Debugging a strange iTunes permissions problem with DTrace

November 27, 2011

The other day, I noticed that one of my media files wouldn’t play in iTunes because it decided that my computer wasn’t authorized to play it. I could not authorize my computer to play that song, however, because the iTunes account name associated with that song no longer existed—I had changed the email address of my account in between the time that that was purchased and when I transferred it to my current computer (it was purchased prior to iTunes’ releasing all of their music DRM-free; I strongly oppose DRM).

When I attempted to re-download the track from the iTunes Store, it gave me this error message which would ordinarily be pretty helpful:

iTunes couldn’t download your purchase.

You don’t have write access for your iTunes Media folder or a folder within it. Change permissions (in the Finder) and then choose Store > Check for Available Downloads

Alas, that was not the problem. Nothing I can think of would have messed up the permissions, and find(1) confirms that all of the subdirectories there are owned by me and are readable, writable, and executable:

$ cd ~/Music/iTunes
$ find . \! -user $USER
$ find . -type d \! -perm -0700
$ # No output from the above commands

What’s going on here?

Time to dig deeper with DTrace. DTrace is a powerful debugging tool, useful for answering such questions as “What system calls is this process calling?”, “Why is the performance of my server so horrendous?” and many more. It’s like strace on crack.

But with great power comes great complexity. In order to use DTrace, you need write a short program in the tool’s D programming language (not to be confused with that other D programming language). The program can be written on the command line or in a separate text file, but it’s still non-trivial. Some really useful examples can be found here, in addition to the various examples in the documentation.

The error message from iTunes strongly smells like a call to open(2) is failing with EACCES when iTunes tries to create the re-downloaded media file. Let’s see if that’s the case:

/pid == $target/
  printf("%s %s 0x%x 0x%x", execname, copyinstr(arg0), arg1, arg2);

/pid == $target/
  printf("errno=%d", errno);

Get the PID of iTunes, start tracing it with sudo dtrace -s open.d -p $PID, and try to download the file again. Unfortunately the output is not expected—errors like this get printed many times:

dtrace: error on enabled probe ID 6 (ID 120: syscall::open:return):
 invalid user access in action #2 at DIF offset 24

After a little more digging, I discovered that iTunes does not like getting debugged, which probably means it also doesn’t like getting itself traced—it just makes the debugger segfault instead. Fortunately, it’s not too hard to get around this: just turn ptrace into a no-op in iTunes when it tries to make itself undebuggable with ptrace(PT_DENY_ATTACH). Charlie Miller provides a nice gdb script for doing so:

break ptrace
condition 1 *((unsigned int *) ($esp + 4)) == 0x1f
commands 1

Ok, so we’re past that hurdle. Quit iTunes, restart it under gdb with this anti-anti-debugging technique, fire up DTrace again, and try to re-download the file:

CPU     ID                    FUNCTION:NAME
  0    119                       open:entry iTunes /.vol/234881026/1516872/SC Info.sidb 0xa00 0x1b6
  0    120                      open:return errno=13

Hmm. Error 13 is indeed EACCES, but what is this strange file under /.vol? Why, it’s the Volume Management file system, used by the Carbon File Manager. Using ls -al /.vol, it appears that that directory is completely empty, yet somehow other file accesses within there succeeded, as indicated in the DTrace output.

I’m not sure if there’s an easy to figure out which directory in the real file system that /.vol/234881026/1516872 refers to, but a quick search for a file named “SC Info” yields two likely candidates:

$ locate -i "SC Info"
/Users/Shared/SC Info
/Users/Shared/SC Info~orig

Let’s see what those directories look like:

$ ls -la /Users/Shared/SC\ Info*
/Users/Shared/SC Info:
total 0
drwxr-xr-x   2 root  wheel   68 Jul  2  2010 .
drwxrwxrwt  10 root  wheel  340 Oct 30 22:54 ..

/Users/Shared/SC Info~orig:
total 0
drwxrwxrwx   2 adam  wheel   68 Jul  2  2010 .
drwxrwxrwt  10 root  wheel  340 Oct 30 22:54 ..

Aha! So iTunes is trying to create a file named SC Info.sidb in /Users/Shared/SC Info, but it’s failing because I don’t have write access to that directory.

The solution:

sudo chmod a+w /Users/Shared/SC\ Info

Bingo! The song now downloads successfully.

Of course, you probably could have skipped all this, googled the error message, and found this knowledge base article explaining how to fix it without too much trouble, but that’s boring. Using DTrace to debug the problem is much more fun and exciting!

Do you have any great success (or failure) stories involving DTrace?

One Response to “Debugging a strange iTunes permissions problem with DTrace”

  1. Thanks for the helpful article. It solved my problem. It’s nice to find info on debugging MacOS X