Exploring Airplay Mirroring Internals

August 20th, 2011
Spencer Nielsen Follow snielsen42 on Twitter

AirPlay Mirroring was without a doubt one of the coolest iOS 5 features announced to the public at the 2011 WWDC Keynote. This amazing technology allows you to display the screen of your iOS device on an Apple TV 2. At the moment, the only supported mirroring device is the iPad 2 and the receiving device is limited to only the Apple TV 2. Airplay Mirroring is the latest in a pedigree of Apple media streaming technologies. The previous entry in the family was Airplay Video and is a slightly different technology that allows specific content to be broadcast from a variety of iOS devices and iTunes to an Apple TV 2. The precursor to that technology was originally called AirTunes and is used to broadcast audio content from iOS devices and iTunes to the AirPort Express and Apple TV (either 1 or 2). One peculiar thing that people noted about AirPlay Video was that it was wide open and unencrypted! This was surprising in light of the fact that AirTunes was most definitely not. Because of AirPlay Video’s relatively easy hackability, there quickly sprang up all manner of apps for both Mac OS X and iOS which would act as an AirPlay Video consumer.

So my thought process went like thus:

  • AirTunes – Encrypted
  • AirPlay Video – Unencrypted
  • AirPlay Mirroring – ???

I thought it would be really awesome to whip up a AirPlay Mirroring consumer real quick and I thought the useful applications of such an app would be obvious. At the time I couldn’t find any technical information on the AirPlay Mirroring protocol so I made a guess that it was very similar to the AirPlay Video protocol, maybe even built right on top of it. One of the reasons for this dearth of information was that the setup to test the hypothesis is not a trivial one to get a hold of. First of all, hardware-wise you need:

  • An iPad 2
  • Apple TV 2
  • An HDMI compatible TV along with an HDMI cable (My TV at the time was made in 1982. It did not have an HDMI input).
  • A micro USB cable (not common).

Software-wise you need to install an iOS 5 beta on the iPad 2 and the corresponding iOS update for the Apple TV 2 (that is what you need the micro USB cable for). That setup will allow you to use AirPlay Mirroring but then you need the wherewithal to sniff the network packets and try talking on the right ports to find out the real story.

Say Hello, I mean, Bonjour

I acquired the all the necessary components and starting hacking. The first thing I did was to write a program that would appear on the network to be an AirPlay Mirroring capable device. Perusing the available services with Bonjour Browser made this easy enough. There it was clear as day, the AirPlay service (“_airplay._tcp“). Looking in the TXT record fields there was a “features” section with a bit field of value 0x39f7. I didn’t know what the individual bitfield values indicated but I was sure that one of them was AirPlay Mirroring because the previously advertised value in that field on older versions of the OS was 0x7. So I wrote a program to register a similar looking Bonjour service and lo and behold, it showed up in the iPad 2’s AirPlay menu. Obviously nothing would happen if you selected it because I did not have the advertised AirPlay port (7000) open and running. However, it did not show up as being a “Mirroring” capable device. After some head scratching I tried also registering an AirTunes service (“_raop._tcp“). Now my laptop showed up as a device that could be mirrored to.

So now my program appears on the network to be a AirPlay Mirroring capable device. It was time to see what the iPad 2 was going to say to me on port 7000 after it connected. Interestingly enough, it didn’t say anything. It never even connected to it! This was odd because 7000 was the clearly advertised port in the Bonjour service. I decided that I should know what a real mirroring session is supposed to look like over the network.

Network Olfaction


This means nothing to me.

It had been many years since I had done any significant packet sniffing (back when Wireshark was called Ethereal) and so I had to learn about all the new wireless specific quirks and issues (which was actually quite fun). After some flailing around with Wireshark I found out that the magical mode I needed to put my wireless card in is called “monitor mode” which is also known as “passive mode” (not even “promiscuous mode” would do the trick). Unfortunately Wireshark cannot enable monitor mode and so I turned to KisMAC. The interface was a bit confusing but I got the hang of it after a little bit. Our neighborhood is pretty 802.11 noisy and so I had to filter a lot of the other devices out by MAC address so that the capture log would only be my own wireless network. I also noticed that the interface was tuned to sniffing 2.4 GHz network channels as opposed to the 5 GHz ones that my network ran on. So I switched my network to 2.4 GHz and restricted it to only one channel so that KisMAC would not have to perform channel hopping and potentially lose a packet in the meantime (though I think this may have caused more transmission errors and re-transmissions by doing so).


Hmm, I guess the audio is going over AirTunes…

I captured some packets into a dumplog and opened it up with Wireshark for examination. Unfortunately all I got were generic IEEE 802.11 packets with nothing but header and opaque data. It then occurred to me that the data obviously appeared opaque because it was encrypted (WPA2). I then took the security off of the network and captured another session. This time the higher level protocols were exposed and I could now observe a legitimate mirroring session in action.

The first thing I noticed was that the AirPlay session wasn’t even running on port 7000, instead it was running on port 7100! I couldn’t see this port number mentioned in any of the Bonjour advertisements or any of the earlier network traffic (it is possible that I just missed it). I am guessing that you might just have to know that mirroring sessions are conducted on port 7100 instead of the usual port 7000.

The second thing I noticed was that before the mirroring session started, an AirTunes stream was established and connected. I wasn’t sure that I would be required to do this and so I decided to leave that aspect alone for the time being.

“AIRP”!? That looks suspcious.

I also noticed some odd UDP communication between the iPad 2 and the Apple TV which appeared to possibly be related to the AirPlay session. The second one of which was particularly interesting because it contained the string “AIRP“. In both exchanges each packet appeared to be sent twice. This is presumably because the reliability of the recipient receiving the message was important and UDP of course has no built-in mechanism to ensure delivery.

Getting Down To Business


The conversation of interest.

Looking at the volume of traffic and where things spiked it was obvious that all the video was being shuttled over port 7100 and so it was time to examine the conversation over there. It starts easy enough with a “GET /stream.xml” and a plist response containing version and screen size information. However, all my AirPlay Mirroring dreams were shot down when I arrived at “POST /fp-setup“. This is a FairPlay DRM initiation which means that the mirroring video is indeed encrypted. I observed two back-and-forths between the consumer and producer most likely though an RSA keypair. This was to presumably to perform a challenge-response and for the iPad 2 to send the video’s encryption key over to the Apple TV (I would guess that would mostly likely be a 128-bit AES key used for CBC). At this point I was pretty much dead in the water. I sniffed a number of legitimate sessions and inspected the data fields for patterns and fields that remained constant. Just for kicks I tried having my program replay responses from legitimate sessions which were obviously rejected by the iPad 2 with a promptly closed port. At that point I was just grasping at straws.

The private key that I would need to create an AirPlay Mirroring consumer is hiding somewhere inside the Apple TV 2. I don’t have any idea whether it is in hardware or software (Perhaps present in the iOS restore images? Probably not though). I know that fairly recently someone cracked open an AirPort Express and extracted the AirTunes private key which allows people to now create AirTunes consumers (Hmm, could it be that the AirTunes and AirPlay Mirroring keys are the same?). Even if someone with more leet skills than me was able to retrieve the private key out of an Apple TV 2, I would still have to reverse-engineer the fp-setup protocol and figure out what needs to be encrypted and where which would be no small feat.

I have decided to leave my AirPlay hacking for the time being in order to concentrate on other things. After my hacking session was finished I did later stumble upon some web pages that confirmed that AirPlay Mirroring is indeed encrypted. I had unfortunately missed that little nugget in my first research pass. It’s ok though. I had a ton of fun learning more about this hot new technology and was able to flex some minor reverse-engineering skills and keep them fresh.

35 Responses to “Exploring Airplay Mirroring Internals”

  1. kalon Says:

    Awesome Spencer! It’s good to read you’re still hacking away at things.

  2. Toni Says:

    I like Airplay :)

  3. zennit Says:

    Hi spencer, any ideas what apple are actually doing to achieve airplay mirroring, is it hardware or software based and is it possible to create something simlar to share a computer screen to another computer? I can see it being usefull with games where frame rate is important and vnc rdp just doesn’t work very well. Are they using some kind or capture and stream type system?

  4. zennit Says:

    Is this on the right track?

    http://arstechnica.com/civis/viewtopic.php?f=19&t=1148093

  5. snielsen Says:

    @zennit Unfortunately I wasn’t able to get past the Fairplay encryption and so I wasn’t able to dig in to the specifics of the video encoding. I think the thread you pointed to is probably correct in it’s assessment that whatever video format is used is probably something that has hardware encoding/decoding support in iOS devices.

    Whatever Apple is doing to access the (system-wide) screen buffer and send it to the video encoding hardware is most likely a mechanism that will never be exposed to developers. As for the latency of AirPlay mirroring, in my experience I havn’t found it to have low enough latency to support any action heavy gaming experiences (slower moving games like RPGs or turn-based games are fine). I havn’t compared it to computer screen sharing solutions like Apple Remote Desktop (which I think is really just vnc) or Microsoft’s Remote Desktop programs. I know that the OnLive gaming service has employed a lot of very specific hardware and software solutions targeted at attacking the latency problem.

  6. Nathan Says:

    I’ve always wondered how Apple was able to do this with relatively low latency. I’ve been in the Wi-Fi chip business before, and to get data small enough (and quickly too) to be transmitted to the Apple TV and then for the box to decode it with minimal lag does require some serious horsepower to encode. Not only that, the encoder on the iOS sending device needs to always grab the screen buffer and constantly feed that data to the encoder. My other theory which I’m completely pulling from thin air is whether or not the application itself is replicated on launch inside the Apple TV – in other words, once a user on the iPad launches an app that supports AirPlay mirroring, it sends that same app to the AppleTV to run and all that is exchanged is the program data so that the AppleTV can do its own rendering locally.

  7. snielsen Says:

    @Nathan Interesting idea. I think that would be much harder to pull off than straight video though. I am absolutely sure that as of right now it is simply sending encrypted video frames taken directly from the screen buffer. Compression artifacts are visible on the Apple TV and the network packet volume noticeably increases when you introduce visual complexity. I think that Apple has made very conscious decisions on what kind of video and encryption hardware they have put in their devices and the inclusion of multi-core CPUs has made this kind of technological feat a reality.

  8. kB Says:

    Can you look into AirServer and Reflection and update us on this? They both seem to have cracked the key.

  9. snielsen Says:

    @kB Yes, they both have extracted the private key and reverse engineered the mirroring protocol. I have been in contact with the author of Reflection/AirParrot and am planning on writing a follow-up article about the processes and techniques he used once he decides to make them public. Suffice it to say that the reverse engineering involved there was much more extensive than what I attempted :)

  10. kB Says:

    Thanks. Please post the new article as soon as you can. I am very desperate about it. :)

  11. James Clarke Says:

    @snielsen Any news on this? Theoretically, could you not extract the private key from AirServer and/or Reflection?

  12. snielsen Says:

    @James Clarke Yes the private keys are present inside the binaries and so if you wanted to extract it you could also target those apps in addition to airtunesd. I am guessing that we will not get the full scoop on the protocol until late this year at the earliest (ie until Apple implements all the functionality of Reflection/AirParrot in their products).

  13. James Clarke Says:

    @snielsen I’ve done some digging around. http://nto.github.com/AirPlay.html#screenmirroring is very detailed. I’m not sure about the significance of the /fp-setup requests, except that the second response ends with the final 20 bytes of the second request. Also there is the “FPLY” header, followed by 0x03, 0x01 and then either 0x01, 0x02, 0x03 or 0x04 which indicates whether it is the first request, first response, second request or second response. The “POST /stream” request is a binary plist, and contains the AES key and IV. The IV appears to be unencrypted (just in base 64), but I’m not sure how to decrypt the AES key. It isn’t RSA encrypted as it is too short, and it starts of with a “FPLY” header with a lot of zeros and a “<" in the middle of them. It is worth noting that it must be possible to decrypt this key without the prior requests (i.e. the decryption key or deobfuscation method must be fixed) because the newer AirPlay audio streaming protocol which is used by the Apple TV 2 has the AES key used transmitted in the same format, but just in the header of the request (http://nto.github.com/AirPlay.html#audio-rtsprequests). Do you have any idea how to progress from here?

    P.S. I obtained my data with HTTPScoop (http://tuffcode.com) – it has a very simple interface but gets the job done.

  14. Charlie Says:

    Hi… @snielsen and @James Clarke…
    I would be very interested if either of you have progressed at all with your findings since this article was written?
    It seems that plenty of products eg. Reflections / Airserver and others now have got a workable solution… it would be great to hear if either of you have got anywhere.

    Many thanks in advance…
    Charlie

  15. Chang Says:

    @snielsen and @James Clarke Hi everyone, I’m learning Airplay Mirroring for a few days, And I tried to decrypt the video stream as like Shairport4W, but pity failed. The Shairport4W private key is 256 bytes(after decode from base64) while the Mirroring is 72bytes. I don’t know why Apple curtail the private (and public)key(s). Moreover, the length of Airturns public keys are the same between the Shairport4W and Mirroring.
    It’s really wonderful if the Mirroring Private key can be cracked from the reflection or others, but it’s rather a BIG project to reverse engineering.
    I wonder glad to hear you have got any news about this? And I have done some digging on the internet packages and AES keys. So if any thing I can help to do, please mail me : [email protected]

    Thanks!

  16. snielsen Says:

    @Charlie, @Chang I haven’t looked into AirPlay mirroring myself lately. So far the developers behind AIrServer and Reflection have kept the details and encryption keys close to their chest. I thought that at this point Apple would have duplicated their functionality but they seem to be content to let these 3rd parties pick up the slack.

    I would expect that down the road after Apple has implemented more AirPlay mirroring functionality that these developers will make their work public. But who knows how long that could be.

  17. James Clarke Says:

    I gave up after a while – I couldn’t work out exactly what was going on. I might revisit it if I have time though…

  18. Chang Says:

    It ‘s not so easy for apple to prevent the 3rd parties from cracking the key, for I think the private key is fixed in iDevices(maybe included in hardware or software), so if the key changed, the iDevices in the hand of consumers can not work any more with this function. I am not sure am I right?
    I have looked into the files of reflection and there are only the .exe and one .dll files are noteworthy. I guess the key maybe hidden in this two and whether it’s a little easier to crack them than AES? :D

  19. JM Robles Says:

    Hi! What’s about this, https://github.com/nto/AirSpeaker/blob/master/AirTunes/CryptoController.m (setPrivateKey) ?

    Is it the private key was looking for? Or only for AirTunes?

    Thanks so much for you work!

  20. Chang Says:

    @JM Robles

    Thanks Robles, I’m afraid that this private key is for AirTunes only, we can get this key from the open source project shairport4w. I have tried using it to decrypt video but failed.

  21. JM Robles Says:

    @Chang

    Yes, sorry, I saw it after further investigation.

    Now I’m studying AirPlay protocol, if I understand well, the iOS device do a ANNOUNCE to “AirPlay server” with a SDP content where it specifies the session AES key ciphered (with AES IV), not?

    Regards!

  22. Chang Says:

    @JM Robles ,
    yeah, right. The iDevice will send the AES IV and the encoded AES key (base64) to the server in the ANNOUNCE package.

  23. JM Robles Says:

    @Chang

    Good!

    The decode “AES Key” is really “FPLY\x01\x02x01\x00″ + 64 byte data.
    But I don’t know if that 64 other bytes is the key or no.

  24. Chang Says:

    uh…sorry, I’m not very clear what you mean.

    you can refer to the website:
    http://nto.github.io/AirPlay.html#screenmirroring
    and , in the section 6.1, you will see “The client sends a binary property list with information about the stream, immediately followed by the stream itself. ”
    And in the property list , param1 is the AES key, encrypted with FairPlay, 72 bytes.

  25. dm4 Says:

    Hi @Chang,

    Since the AES key (param1) is encrypted by Fairplay, I think the ‘/fp-setup’ requests may be kind of key exchange in Fairplay protocol? I’ve reverse engineered AirServer a little bit but couldn’t find suitable private key to decrypt the AES key :(

  26. Chang Says:

    hi, @dm4

  27. Chang Says:

    hi, @dm4

    I DO agree that the ‘/fp-setup’ requests exchanged private key, what we need is THE private key .

  28. unicorn Says:

    We have already cracked the AirPlay mirror encryption algorithm, the ‘/fp-setup’ encrypted by Fairplay with private key,than the video steam also encrypted by Fairplay with different private key.
    the demo video show:
    http://v.youku.com/v_show/id_XNTgwMjQzMTk2.html

    mail: [email protected]

  29. Chang Says:

    Wow, it’s so cool, would you like to share it? :)

  30. CarpeDiem Says:

    Any update on the solution to Airplay Mirror Encryption Algorithm?

  31. stonexing Says:

    I have alerady work out it on iPhone/ipad
    this is my link video
    http://www.tudou.com/programs/view/kNTG-q0qtRY/?lvt=&resourceId=0_07_10_28

    my email:[email protected]

  32. RestlessB Says:

    Hi everyone!

    I´m trying to reverse engineering AirServer in order to understand how AirPlay Mirroring Protocol works. Anyone has idea on how fp-setup communication works?

    If anyone could help me to understand the how the keys transmitted this is my email: [email protected]

  33. indiekiduk Says:

    That first video is questionable if it’s not just an apple tv. And the second video is using photo sharing airplay which is already open, not mirroring.

    I’ve managed to pass ft-setup and can get it to start streaming but not figured out how to decrypt param 1 to the AES key.

  34. Arb Says:

    No change in this?

  35. Droid Says:

    The guys at WaxRain who built AirPin PRO have figured this out as well and have the functionality on their android client which works flawlessly with both iOS and OSX AirPlay Mirroring clients.

Leave a Reply

Entries (RSS) and Comments (RSS).