As part of the project I was going to do some experiments with video of motion. Turned out that the resourcing just wasn’t there in practice. In some ways that has turned out nicely and certainly has turned out well in terms of collaboration twixt me and the scientists. I have been given access to a lot of experimental work related to the perception of human movement - which is my interest - and fascinating stuff it is indeed. There is the strange disjunction in science between the depth and breadth of the conceptual domain and the operationalising of that domain for the purposes of investigation. Here’s an example - the conceptual domain is movement planning and the perception of the body. Here’s a little video of an experiment (one I look forward to using).

grasp_1

This experiment looks at interference between visual input of motion and motion planning. That’s profoundly linked to being in the world, the whole idea of autonomy and agentive behaviour. How do we move toward and away within complex environments?  What are the intrinsic and extrinsic components of goals?

Which gets boiled down to a dinky little plastic hand grabbing a peg.

This is one of the fascinating aspects of experimental science - the reduction of phenomena to some idealised simplicity.

Aside: I like the Tatum - named after the great pianist Art tatum, and representing the smallest perceptual time unit in music. Art Tatum was an incredibly fast pianist who could maintain swing at high speed. Most musicians switch to regular rhythm when they get fast - If you listen to lots of jazz guitarists it’s pretty obvious that their rhythm becomes symmetrical or regular  once they speed up.

I promised some EEG and here it is (albeit a little anaemic)

this is just a couple of seconds - the blue trace is from (jumps back to Matlab to read the electrode numbers) is from the left hand front - that big spike is a blink (probably) which means it is not a good data segment. The red and green traces are from further back and on top of the head - blinks tend not to propagate back that far.

I’ll add in a blurb about EEG here - we all get so used to ‘Science’ that we forget how amazing all this shit is. What that eyeblink spike is registering is that change in electrical potential caused by the movement of the eyelid across the eyeball. (I’m working on my crappy old man memory here, so please correct me if I’m wrong (forgiveness too)) The eyball has a charge relative to some other point of the body (the reference in EEG parlance, typically the mastoid process - the bony bit behind your ear). This charge varies according to the location of the eyelid - in its role as ‘ground’. That’s ridiculous, that we can measure such piddling things. In fact the whole EEG thing is ridiculous - it measures changes in the membrane potential of  dendrites in aligned pyramidal neurons in the cortex.

An age of miracles and wonders!

Here is an example of the sounds that the participants hear. This is a pretty compressed mp3 but it gives an idea

exp_1_second3minutes

This material is constructed from 4, 6, and 8 second segments of rhythmic music composed to reflect 3 levels of predictability - low, medium and high. The individual sounds are quite limited and there is nothing going on that is traditionally tonal. i didn’t want to bring in the whole layer of ‘tune’ which would confuse the analysis. At the same time single sound rhythms - the easiest to analyse - were too boring. So I went for a compromise. Ten rhythms were programmed up in GURU for each of the levels of predictability, then exported as wavs and as midi. The midi files were analysed in Matlab so that I could check and quantify the predictability, which I operationalise as entropy - in particular the sample entropy (Lake, D. E., J. S. Richman, M. P. Griffin, and J. R. Moorman.Sample entropy analysis of neonatal heart rate variability. Am J Physiol 2002; 283(3):R789-R797;)

So the thirty files each have an associated entropy. I split the files into 4,6, and 8 sec segments (in the very neat DAW, Reaper) and then shuffled them together into 3 * 3-minute ‘pieces’.

I’m quite keen on using an entropy measure throughout the experiment - in the design of the music used as the experimental stimulus and in the analysis of the brain dynamics. Entropy measures (and information theoretic measures generally) map nicely into probabilistic models. That’s nice because it’s consistent with the ‘resonance’ model of brain dynamics I’m using for this project - and generally it is a model I favour for the way the brain is ‘in’ the world. We are active in the environment, which means that the sensory field is always changing and can be thought of as a time-varying probability field. The sensorium, the unfolding response of our sensory sytems to the  entire sensory field, is then reflected in our brain dynamics - which we can measure via the EEG.

I like to think of the brain as encoding the probabilities of experience (ie the history of the field) and using these probablilities to predict what is going to happen next, given the current state of experience. So the brain dynamics should reflect the input dynamics. And music can be thought of as a structuring of the probabilities of of a particular input modality - hearing. It all fits together nicely and means that a consistent framework can be used to generate the music and analyse the resultant effect it has on people - and then subsequently be used again to generate new music based on that repsonse. And even though this project is looking at music as the stimulus - the dominant structural component of the sensory field - it seems to me that any input can be analysed in terms of structure, it’s effects on the brain investigated and then used as a generative mechanism for the production of further art works.

That’s the dream anyway

Will post up some EEG traces as soon as I get Matlab to read the Biosemi data

On the incredibly good news front Daryl Buckley from Elision has agreed for members of Elsion to take part and perform a score derived from their EEG traces using the algorithms I’m developing. That’s very exciting for me as they are fabulous musicians. Of course the other musicians have to agree as well

And pretty much as big a hooray for debugging the experimental design software and finally recording some EEG. Next is to read the EEG into Matlab and write the analysis software (already got most of it), and automate the whole process of getting good sections of data and chucking out crummy bits. EEG recording is mega sensitive, so sensitive that the person sits in a Faraday cage to keep out stray electrical fields - Times when a person blinks or swallows, or moves an eyebrow etc etc have to be identified and thrown away. It’s tedious to do this for 64 electrodes for hours of EEG so I wrote software to automate a lot of this years ago which I’ll resurrect. Also there are log files generated by the experimental design software that track what the person is doing and I have to write something that will read through the logs and extract specific bits of info from reams of text.

Recruitment of people to have their EEG done is underway. The more the merrier, onward and upward!

Well the software that worked on my ancient computer at the uni does not work on the new computer in the EEG lab.

Back to the drawing board.

UPDATE - it appears that the marvellous Paul has found the problem with this iteration and tomorrow I check it out on the lab machines. Fingers are crossed , and by the sounds they are making perhaps crossed a little too much.

and in further news I have a meeting about accesing the human movement and emotion experimental video and still data on thursday. Has the great uni conveyor really changed direction?

helped out with an experiment today - very valuable experience as it gives me an idea of the reality of being hooked up to all the gear. As I am wanting to get both an aesthetic experience and an experimental outcome I have to keep in mind that plugging in is not that comfortable or thrilling if done over an extended period.

Had a demo of the EEG labs, putting on the caps the hold the electrodes, setting up the software - much easier than when I did my PhD. I’ve volunteered to be a participant in one of the lab experiments for the very helpful Dana, who is instructing me in the apparatus. I’ll do quite a few different experiments I hope as it is all fascinating stuff and also the lab people deserve a little reciprocating from me for all their help.

Got the experiment working this morning - or more precisely i have got the stimulus presentation working. Some great help from one of the programmers in Psychology and one of the post-docs in the lab. The solution is one that often happens I think. We shifted the complexity load away from the experimental design software and toward stimulus presentation  and post experimental analysis - both of which are much easier to handle.
About to go off to the EEG recording room and learn how to put on the cap and set up a recording session. Here’s the gear http://www.biosemi.com/products.htm
Wheeeee….

well another frustrating day with my hands deep in the bowels of experimental design software. What I have been spending days trying to do is write a little program that goes

1. play a sound until it finishes
2. show a picture every 500 msecs until the sound finishes
3. repeat with new sound

Somehoworother I can’t even get that done
What I tried today

1. loaded sound (as wavefile)
2. gave the sound two names (no idea why the two, but that’s what the example had though there was nothing about it in the documentation)
3. put the sound in an array and gave the array a name

at this stage I thought I might be able to do something like play(array[index of sound])….how wrong I was. I still hadn’t made a trial and given that trial a name, nor had I declared something called a stimulus event and given that a name

Of course I intuitively grasped the importance of assigning a sound event to that stimulus event. But then, with a rush of blood, thought I could play that stimulus event in it’s guise as a sound event. Oh the follies of the aged. If only i was in utero, then I would understand computers as naturally as using a hammer

Anyway there I was thinking i might be able to make a sound, not realising that i hadn’t declared yet another trial which takes as a parameter the incredibly informative….{nothing}

soon after I gave up. It ain’t rocket science, it’s harder.

More programming for the experiment - it is a very simple idea - play some music files sequentially and also play some graphics - but is proving quite difficult for the software I am using. The package is designed by psychologists and heavily biased to serial presentation of a single stream of data - plus logging user responses and so forth. Once you go to concurent streams (ie what we are used to doing with media all the time) it is very much kludge kludge kludge. The interface is truly awful, which has made me think there is a big opportunity for a media designer type to design up one of these experimental delivery packages. For example, experimental psychologists are constantly dealing with timelines of presentation, event triggering , repetition and so forth, but the most common experimental design software has no representation of time (a la ProTools or Final Cut etc etc) in the interface. I shall pursue this further in my quest for ‘concurrent income streams’