Datasets in SPSS/Excel

One of the golden rules of working with datasets in SPSS is that you need to have one row for each participant. I know there are some exceptions to this, but it’s an important general rule for SPSS.

The main consequence of this is that, when you’re dealing with any form of within-subjects data, your dataset quickly becomes very wide indeed. Let’s look at an example below. Here, we have 10 participants, involved in two experimental sessions. For each session, we’ve measured the Reaction Time (RT).

That’s not too messy (note that I just pasted in 1200 for the values as this is just an illustration). But let’s make things worse. Let’s have 10 experimental sessions, each with three different blocks of trials, each representing a different within-subjects condition. What does it look like now?

Well, we can’t fit it all into a single screenshot, as the dataset has a large number of columns. This is an illustration of what gets referred to as a wide data format – you have a large number of columns mapping on to various factors, variables, etc.

R does things differently, for most of the statistical tests that I’ll be discussing: it uses the long data format instead.

Long Datasets in R

When you think about it, wide datasets can be a real pain. I’ve seen people spend hours running pivot tables and then having to drag columns around to get their datasets in a format that SPSS will be happy with.

With R, things are significantly easier: for many tests, such as t-tests and ANOVAs of various forms, you only need to use a single layout: the long data format. You can probably guess what this is already, but let’s do a direct comparison using the first example dataset described above.

Again, let’s say we have Reaction Times (RTs) for 10 participants involved in two sessions of experimental trials. In wide format, these data look like this:

In the long format, these data look like this:

Here you can see the difference: in the long format, the one row per participant rule does not apply. Instead, you have one row for each combination of factors under examination.

What if your Datasets are all in the Wide Format?

There are a number of options that you can use to convert between the two different formats. I’ve covered perhaps one of the easiest methods, in the form of the reshape package, in a previous post. You’ll need to install the reshape2 package to do this, using the package installation guide I presented previously.

Just to give an example, let’s work through the dataset I’ve been describing above.

First, let’s create some data:

session1 <- rnorm(mean=1500, sd=250, 10)

session2 <- rnorm(mean=1000, sd=250, 10)

ppt <- seq(1:10)

wide<- data.frame(ppt, session1, session2)

That gives us a dataframe called wide. How do we reshape the dataset to the long format that we want? Simple, by using the following:

long<- melt(wide, id=c("ppt"))

This then gives us a dataframe called long, arranged in the format we want.

In many cases, if you want to avoid having to do this, it’s best to make sure your datasets are in the long format beforehand – it’s a simple case of planning ahead and knowing that you can do things differently.

Summary and Next Steps

This post illustrated how to get your data organised for use in R for those who are used to using SPSS/Excel. There are many useful ways to re-organise your data, and I’ve covered one of them here (the reshape package). The next steps include aggregating your data and then running statistical tests.

Godwin, H., Benson, V., & Drieghe, D. (2011). Using visual interruptions to explore the extent and time course of fixation planning in visual search. Poster presented at the European Conference in Eye Movements, Marseille, France.

The poster can be downloaded via the following link:

Click here :ecem_interruption_poster_final

Normally, summers involve a slight easing up of the workload, thanks to there being no students around, allowing people to catch up with things…but not this time! It’s been fun though, it’ll just be a little while before I’m back.

See you…out there…

Click on the image to see an enlarged version.

Packages in R

With SPSS/Excel, you pretty much get everything you could ever want to use, and more, installed with the default installation. This leads to a simple question. How many of the many hundreds of buttons, boxes and options in these programs have you used in total?

R is different. The basic installation of R comes with a large number of packages and commands. However, with R, people have been able to share their own packages which can help out, extend, and implement other useful things to make R even more funky and powerful. This is beneficial for a number of reasons, but, for the new user, it might seem a bit strange. Why doesn’t R just come with all the packages installed right away? Well, the chances are you won’t need all of the packages in existence, so there’s little point in installing them all by default. Doing so also reduces the size of an R download, saves hard drive space, and so on.

People are adding new and useful packages all the time, so let’s install a couple of popular ones that I use all the time.

Installing Packages in R

To get to the list of packages you have installed, go to the packages tab using RStudio:

Packages are often updated, so you can use the Check for Updates button to update your packages.

To install a new package, you can either run the following command via the script tab or console window:

install.packages("PACKAGENAME")

Where PACKAGENAME is the name of the package. Alternatively, using RStudio, you can hit the Install Packages button in the Packages window. You’ll be greeted with something like the following:

In this window, just type the name of the package you want to install in the Packages text box. Here, I’ve gone for ggplot2 and plyr.

Once you hit the install button, the packages will be installed. It’s best to leave Install Dependencies checked because some packages need others to function. For example, ggplot2 uses plyr.

Loading Packages in R

The packages you have installed won’t be loaded straight away. If R loaded all the packages you had installed, then you would often end up with packages loaded that you don’t need to use. To load your packages, you can do one of two things. First you can run the command:

library(PACKAGENAME)

Where PACKAGENAME is the name of the package that you want to load.

An alternative method is to select the package using RStudio’s Package window. To load the package(s) that you want, all you need to do is click the checkbox next to the package name. See below.

There we go, ggplot2 has now been loaded! It’s also loaded plyr as ggplot2 needs plyr to function, as well as reshape.

Getting Help with Using a Package

Packages come with helpful documentation to get you started with using them. Again, you have two options in terms of accessing the documentation. First, you can type the command:

?PACKAGENAME

Where PACKAGENAME is the name of your package.

Alternatively, you can click the name of the package in RStudio’s Packages window, as below.

Whichever method you use, you’ll be presented with the documentation in your packages window, which you can browse to work out what you need to do to use the package.

Which Packages should you Install?

One of the daunting aspects of getting started with R is choosing how to use it, and what packages to install. I’ll cover some suggested packages in future guides, but for the eager, there’s a great list of popular packages that has been put up online by Matthew Dowle, and is available at this link. The list is also part of his unknownR package, which is worth trying out if you are new. When learning R, I used that list to inspire me in terms of which packages I should learn.

You should also keep an eye on community sites such as R-Bloggers, as you’ll often read about packages, as well as other tips and tricks, that you can use and learn from.

UPDATE: Thanks to Tal Galili’s comment, readers may also want to check out CRAN task views, which has detailed info on a huge range of packages.

Next Steps

In the next guide, I’ll get into the interesting stuff: importing and manipulating data, and how doing so differs from SPSS/Excel.

Click the image below to enlarge.

Before you Start

Before we get into any details, I need to point out something important. Whatever you do, DON’T sit down with R in front of you, clutching a dataset in your hand, and expect to be able to do everything you could in SPSS/Excel straight away. I’ve seen many people try this, expecting R to be identical to what they are used to from SPSS/Excel, and they, not surprisingly, give up very quickly. Just remember this: you’ve probably spent years working with SPSS and Excel, beginning as an undergrad, so this process will take time – but that doesn’t mean you should give up, like I’ve seen many people do. The investment of time is more than worth it!

First Steps

First off, download R to get started. Here is a direct download link if you don’t want to navigate the R websites.

After you download R and have it installed, I recommend picking up RStudio and installing that. Load up RStudio. It will look something like this (click image to enlarge):

Orienting Yourself

Now, there are several points worth considering when you are faced with your new setup. First, you can now officially forget about the annoyances of having to switch between programs to do different things: gone are your days of using Excel to manipulate data, then importing it to SPSS, then graphing it, and then realising there’s something horribly wrong with it. I’ve had that happen plenty of times – spotting an error at the very end of the long chain of processes. The worst thing about when this happens is that you then need to go back to step #1 and start from scratch. It’s no fun at all.

R changes all of that. You have SPSS, Excel and Sigmaplot all rolled into one – with oodles of other features packed in besides (plus you have the option of adding more features and packages yourself). More importantly, as you will be writing scripts to do all your data processing and graphs, if you do detect an error, it’s a case of tweaking your script and then re-running it. Gone will be the days when you had to laboriously click through pivot tables, pasting into loads of new worksheets, losing your place and getting confused…and so on. You’ll find yourself wondering why you ever did things like that in the first place.

I’ll now go through what you have in front of you – taking each part step by step.

The Scripting/Data Tabs and Console Window

In the scripting tab(s) area (yellow), you have access to scripts that you write. Scripts consist of commands that you run and send to R to do things.

The Console (highlighted red) is a bit like the output window you get from SPSS. However, unlike the SPSS output window, it’s interactive, so you can type commands to it directly rather than just seeing buckets of text spewed out. This is useful if you want to test something out or just run something once. For example, type:

>40+2

in the console and press Return. It nicely gives you the answer.

[1] 42

To see what the script tab can be used for, and for what scripts can be used for in general, type the same command in again into the script. All you need is to type 40+2. Then, click anywhere in the row where you have typed your command, and click the Run button (see image below). This sends the command from the line in your script directly to the console. You’ll see other buttons nearby that have other, related functions – hover over them for tooltips.

If you find yourself clicking run, and noticing the console output just adding more ‘>’ signs to the output, it means you haven’t clicked the line containing your command. Try clicking the line containing 40+2 with your mouse and running the command again.

Next up, try adding some additional lines of basic maths to your script, highlight the lines with your mouse, and then hit run again. If you have multiple lines of commands in your script to run, then you need to highlight them with your mouse. When you run them, you’ll see that these lines have all been sent to the console, with the answers coming out at the appropriate times.

The Workspace

In Excel, you were used to having multiple worksheets with different datasets in them. These quickly become difficult to use when you have large numbers of worksheets to sift through. In SPSS, you need to have one main SPSS window open for each dataset. This could rapidly lead to having many SPSS windows open, causing annoyance and confusion about which one to work from.

R and RStudio takes a different approach entirely. In R, you have a workspace (users of Matlab will be familiar with this), and RStudio allows you to get an overview of your workspace in a glance. Gone are the days of many worksheets from Excel or many data windows in SPSS. The workspace in RStudio is highlighted below.

Let’s begin by demonstrating the purpose of the workspace. Type the following into a script and run it, or into the console itself:

answer <- 42

You will then see answer appear in the workspace with a value of 42.

But, this works not only with individual values – you can access, store and manipulate entire sets of data like this as well. Let’s create ourselves a little data set to begin with. Typically we work with dataframes in R, so we’ll call them that from now on. Just think of them a bit like a data window in SPSS, or a worksheet in Excel.

Next, run the following command:

testdf <- data.frame(“values”=rnorm(mean=5,sd=1,100))

What this does is create a dataframe called testdf - don’t worry about the details too much just yet, as I’ll get to handling data in the next post in this series. What this dataframe contains is a list of 100 values randomly selected from a normal distribution with a mean of five and standard deviation of one. When you run the command, you’ll notice that testdf gets added to your workspace:

One of the useful features here is that it gives us information about the number of rows (“100 obs.”) and the number of columns (“1 variables”) in our dataframe. This is often helpful for diagnostics and making sure we know what we’ve done with our data when it’s been copied/merged/transformed/etc.

Finally, when you want to take a look at your dataframe, all you need to do is click the name of it in the workspace – a single-click is all you need. That will then open a new tab showing your data, like so:

This now allows you to look at all of your data – though unlike SPSS and Excel, it does mean that you have open in tabs the data you need to look at, rather than everything. Personally I feel this makes everything seem cleaner and less cluttered, but that may just be me! I know that everyone has their own personal way of dealing with things like this.

Plots

Let’s end with making a very simple plot. For now, we’ll make a histogram of our dataframe. To do that, enter the following command:

hist(testdf$values)

You can see the plot shown in the Plots area in the bottom-right-hand corner of the display – shown in the image below. My randomly-selected values show a reasonably normal distribution!

Next Steps

That’s it for now. Hopefully this will serve as a basic orientation for using R with RStudio. For the next posts in the series, I’ll finish a tour of R, and then there will be a discussion on how to organise your data for R – in some cases, it’s a slightly different approach to SPSS, so is worth discussing in detail.

UPDATE: As Ian Fellows kindly points out in the comments, there are other GUIs available for R besides RStudio – including the excellent Deducer. I also discussed some of the other GUIs available in a previous post from when I was just starting to learn R.

This week, a contingent of plucky individuals from my lab have been presenting at the Royal Society’s Summer Science Exhibition. Sadly I couldn’t make it as I was on the one holiday I take each year! Now that I’m back, I thought it would be worth discussing their exhibit, and encouraging anyone who hasn’t been yet to go!

The exhibit covers details of the work we’ve been doing for years on airport security screening (first publication was back in 2004). You can see an introduction into the research in the video below. Apparently they are working on upping the sound a bit.

Our experiments are still alive and kicking, and we’ll be doing more work on it for (at least) the next 4-5 years – so watch this space! Aside from the practical benefits that this type of research has on offer, it’s turned out to be a very significant and useful source of inspiration for developing current models and theories of how humans search their environments for targets of various types. I wrote some more detailed stuff on my website here a while back. 

More information is available at the Royal Society’s website: click here. By the way, the X-ray picture of a bag that they have used has nothing naughty in it. You can also play some online games developed for the exhibition here and here.

Publications that are relevant to this can be found listed here and here.

—

Finally, I’d like to dedicate this post to the computer used to do the eye tracking in that video above. It died on us a few days after the video was recorded. RIP.

Looking at it more closely, I realise I’ve unintentionally done something quite sneaky. Given that humans naturally follow the gaze of other humans, when you visit the site and see me looking over towards the content of the page, you should follow my gaze and look over here as well. Some people suggested that people with autism fail to do this, but that’s not correct (sly link to some of my colleague’s research on just that topic).

Imagine if I used that picture for nefarious purposes, you could be forced to look at something evil.