Exploring Airplay Mirroring Internals | |
| August 20th, 2011 | |
 Spencer Nielsen   |
AirPlay Mirroring was without a doubt, one of the coolest iOS 5 features announced to the public at the 2011 WWDC Keynote. This amazing technology allows you to display the screen of your iOS device on an Apple TV 2. At the moment, the only supported mirroring device is the iPad 2 and the receiving device is limited to only the second generation Apple TV. Airplay Mirroring is the latest in a series of Apple media streaming technologies. The previous entry in the family was Airplay Video, which is a slightly different technology that allows specific content to be broadcast from a variety of iOS devices and iTunes to an Apple TV 2. The precursor to that technology was originally called AirTunes, and is used to broadcast audio content from iOS devices and iTunes to the AirPort Express and Apple TV (either first gen or second gen). One peculiar thing people noted about AirPlay Video was that it’s wide open and unencrypted! This was surprising in light of the fact that AirTunes was most definitely not. Because of AirPlay Video’s relatively easy hackability, there quickly sprang up all manner of apps for both Mac OS X and iOS which would act as an AirPlay Video consumer.
So my thought process went like thus:
- AirTunes – Encrypted
- AirPlay Video – Unencrypted
- AirPlay Mirroring – ???
I thought it would be really awesome to quickly whip up an AirPlay Mirroring consumer and I thought the useful applications of such an app would be obvious. At the time, I couldn’t find any technical information on the AirPlay Mirroring protocol so I made a guess that it was very similar to the AirPlay Video protocol. Maybe even built right on top of it. One of the reasons for this dearth of information was that the setup to test the hypothesis is not a trivial one to get a hold of. First, you need the following hardware:
- iPad 2
- Apple TV 2
- An HDMI compatible TV along with an HDMI cable (My TV at the time was made in 1982. It did not have an HDMI input).
- A micro USB cable (not common currently).
Software-wise you need to install an iOS 5 beta on the iPad 2, as well as the corresponding iOS update for the Apple TV 2 (that’s what you need the micro USB cable for). That setup will allow you to use AirPlay Mirroring, but then you need the wherewithal to sniff the network packets and try talking on the right ports to find out the real story.
Say Hello, I mean, Bonjour
I acquired the all the necessary components and starting hacking. The first thing I did was to write a program that would appear to be an AirPlay Mirroring capable device on the network. Perusing the available services with Bonjour Browser made this easy enough. There it was clear as day, the AirPlay service (“_airplay._tcp“). Looking in the TXT record fields, there was a “features” section with a bit field of value 0x39f7. I didn’t know what the individual bitfield values indicated, but I was sure that one of them was AirPlay Mirroring because the previously advertised value in that field on older versions of the OS was 0x7. So I wrote a program to register a similar looking Bonjour service and lo and behold, it showed up in the iPad 2’s AirPlay menu. Obviously nothing would happen if you selected it, because I didn’t have the advertised AirPlay port (7000) open and running. However, it did not show up as being a “Mirroring” capable device. After some head scratching, I tried also registering an AirTunes service (“_raop._tcp“). Now my laptop showed up as a device that could be mirrored to.
Now my program appears on the network as an AirPlay Mirroring capable device. It was time to see what the iPad 2 was going to say to me on port 7000 after it connected. Interestingly enough, it didn’t say anything. It never even connected to it! This was odd because 7000 was the clearly advertised port in the Bonjour service. I decided that I should know what a real mirroring session is supposed to look like over the network.
Network Olfaction
It had been many years since I had done any significant packet sniffing (back when Wireshark was called Ethereal). So I had to learn about all the new wireless-specific quirks and issues (which was actually quite fun). After some flailing around with Wireshark, I found out that the magical mode I needed to put my wireless card in is called “monitor mode“. This is also known as “passive mode” (not even “promiscuous mode” would do the trick). Unfortunately, Wireshark cannot enable monitor mode and so I turned to KisMAC. The interface was a bit confusing, but I got the hang of it after a little bit. Our neighborhood is pretty 802.11 noisy, and so I had to filter a lot of the other devices out by MAC address so that the capture log would only be my own wireless network. I also noticed that the interface was tuned to sniffing 2.4 GHz network channels as opposed to the 5 GHz ones my network ran on. So I switched my network to 2.4 GHz and restricted it to only one channel so that KisMAC would not have to perform channel hopping and potentially lose packets in the meantime (though I think this may have caused more transmission errors and re-transmissions by doing so).
I captured some packets into a dumplog and opened it up with Wireshark for examination. Unfortunately, all I got were generic IEEE 802.11 packets with nothing but header and opaque data. It then occurred to me that the data obviously appeared opaque because it was encrypted (WPA2). I then took the security off of the network and captured another session. This time the higher level protocols were exposed and I could now observe a legitimate mirroring session in action.
The first thing I noticed was that the AirPlay session wasn’t even running on port 7000, instead it was running on port 7100! I couldn’t see this port number mentioned in any of the Bonjour advertisements or any of the earlier network traffic (it’s possible that I just missed it). I am guessing that you might just have to know that mirroring sessions are conducted on port 7100 instead of the usual port 7000.
The second thing I noticed was that before the mirroring session started, an AirTunes stream was established and connected. I wasn’t sure that I would be required to do this, and so I decided to leave that aspect alone for the time being.
|
 |
“AIRP”!? That looks suspcious.
I also noticed some odd UDP communication between the iPad 2 and the Apple TV which appeared to possibly be related to the AirPlay session. The second one of which was particularly interesting, because it contained the string “AIRP“. In both exchanges, each packet appeared to be sent twice. This is presumably because the reliability of the recipient receiving the message was important, and UDP of course, has no built-in mechanism to ensure delivery.
Getting Down To Business
Looking at the volume of traffic, and observing where things spiked, it was obvious that all the video was being shuttled over port 7100. It was time now to examine the conversation over there. It starts easy enough with a “GET /stream.xml” and a plist response containing version and screen size information. However, all my AirPlay Mirroring dreams were shot down when I arrived at “POST /fp-setup“. This is a FairPlay DRM initiation which means that the mirroring video is indeed encrypted. I observed two back-and-forths between the consumer and producer, most likely though an RSA keypair. This was to presumably to perform a challenge-response, and for the iPad 2 to send the video’s encryption key over to the Apple TV (I would guess that would mostly likely be a 128-bit AES key used for CBC). At this point I was pretty much dead in the water. I sniffed a number of legitimate sessions and inspected the data fields for patterns and fields that remained constant. Just for kicks I tried having my program replay responses from legitimate sessions which were immediately rejected by the iPad 2 with a promptly closed port. At that point I was just grasping at straws.
The private key needed to create an AirPlay Mirroring consumer is hiding somewhere inside the Apple TV 2. I don’t have any idea whether it’s in hardware or software (Perhaps present in the iOS restore images? Probably not though). I know that fairly recently someone cracked open an AirPort Express and extracted the AirTunes private key which allows people to now create AirTunes consumers (Hmm, could it be that the AirTunes and AirPlay Mirroring keys are the same?). Even if someone with more leet skills than me was able to retrieve the private key out of an Apple TV 2, I would still have to reverse-engineer the fp-setup protocol and figure out what needs to be encrypted and where, which would be no small feat.
I’ve decided to leave my AirPlay hacking for the time being in order to concentrate on other things. After my hacking session was finished, I did later stumble upon some web pages that confirmed that AirPlay Mirroring is indeed encrypted. I had unfortunately missed that little nugget in my first research pass. It’s ok though. I had a ton of fun learning more about this hot new technology, and was able to flex some minor reverse-engineering skills and keep them fresh.
August 23rd, 2011 at 6:54 pm
Awesome Spencer! It’s good to read you’re still hacking away at things.
September 12th, 2011 at 10:53 am
I like Airplay 🙂
September 18th, 2011 at 4:07 am
Hi spencer, any ideas what apple are actually doing to achieve airplay mirroring, is it hardware or software based and is it possible to create something simlar to share a computer screen to another computer? I can see it being usefull with games where frame rate is important and vnc rdp just doesn’t work very well. Are they using some kind or capture and stream type system?
September 18th, 2011 at 4:32 am
Is this on the right track?
http://arstechnica.com/civis/viewtopic.php?f=19&t=1148093
September 18th, 2011 at 7:17 am
@zennit Unfortunately I wasn’t able to get past the Fairplay encryption and so I wasn’t able to dig in to the specifics of the video encoding. I think the thread you pointed to is probably correct in it’s assessment that whatever video format is used is probably something that has hardware encoding/decoding support in iOS devices.
Whatever Apple is doing to access the (system-wide) screen buffer and send it to the video encoding hardware is most likely a mechanism that will never be exposed to developers. As for the latency of AirPlay mirroring, in my experience I havn’t found it to have low enough latency to support any action heavy gaming experiences (slower moving games like RPGs or turn-based games are fine). I havn’t compared it to computer screen sharing solutions like Apple Remote Desktop (which I think is really just vnc) or Microsoft’s Remote Desktop programs. I know that the OnLive gaming service has employed a lot of very specific hardware and software solutions targeted at attacking the latency problem.
October 4th, 2011 at 6:21 pm
I’ve always wondered how Apple was able to do this with relatively low latency. I’ve been in the Wi-Fi chip business before, and to get data small enough (and quickly too) to be transmitted to the Apple TV and then for the box to decode it with minimal lag does require some serious horsepower to encode. Not only that, the encoder on the iOS sending device needs to always grab the screen buffer and constantly feed that data to the encoder. My other theory which I’m completely pulling from thin air is whether or not the application itself is replicated on launch inside the Apple TV – in other words, once a user on the iPad launches an app that supports AirPlay mirroring, it sends that same app to the AppleTV to run and all that is exchanged is the program data so that the AppleTV can do its own rendering locally.
October 5th, 2011 at 9:04 am
@Nathan Interesting idea. I think that would be much harder to pull off than straight video though. I am absolutely sure that as of right now it is simply sending encrypted video frames taken directly from the screen buffer. Compression artifacts are visible on the Apple TV and the network packet volume noticeably increases when you introduce visual complexity. I think that Apple has made very conscious decisions on what kind of video and encryption hardware they have put in their devices and the inclusion of multi-core CPUs has made this kind of technological feat a reality.
April 3rd, 2012 at 10:56 am
Can you look into AirServer and Reflection and update us on this? They both seem to have cracked the key.
April 4th, 2012 at 9:48 am
@kB Yes, they both have extracted the private key and reverse engineered the mirroring protocol. I have been in contact with the author of Reflection/AirParrot and am planning on writing a follow-up article about the processes and techniques he used once he decides to make them public. Suffice it to say that the reverse engineering involved there was much more extensive than what I attempted 🙂
April 4th, 2012 at 9:51 am
Thanks. Please post the new article as soon as you can. I am very desperate about it. 🙂
May 15th, 2012 at 12:45 pm
@snielsen Any news on this? Theoretically, could you not extract the private key from AirServer and/or Reflection?
May 16th, 2012 at 8:26 am
@James Clarke Yes the private keys are present inside the binaries and so if you wanted to extract it you could also target those apps in addition to airtunesd. I am guessing that we will not get the full scoop on the protocol until late this year at the earliest (ie until Apple implements all the functionality of Reflection/AirParrot in their products).
May 19th, 2012 at 7:46 am
@snielsen I’ve done some digging around. http://nto.github.com/AirPlay.html#screenmirroring is very detailed. I’m not sure about the significance of the /fp-setup requests, except that the second response ends with the final 20 bytes of the second request. Also there is the “FPLY” header, followed by 0x03, 0x01 and then either 0x01, 0x02, 0x03 or 0x04 which indicates whether it is the first request, first response, second request or second response. The “POST /stream” request is a binary plist, and contains the AES key and IV. The IV appears to be unencrypted (just in base 64), but I’m not sure how to decrypt the AES key. It isn’t RSA encrypted as it is too short, and it starts of with a “FPLY” header with a lot of zeros and a “<" in the middle of them. It is worth noting that it must be possible to decrypt this key without the prior requests (i.e. the decryption key or deobfuscation method must be fixed) because the newer AirPlay audio streaming protocol which is used by the Apple TV 2 has the AES key used transmitted in the same format, but just in the header of the request (http://nto.github.com/AirPlay.html#audio-rtsprequests). Do you have any idea how to progress from here?
P.S. I obtained my data with HTTPScoop (http://tuffcode.com) – it has a very simple interface but gets the job done.
November 19th, 2012 at 9:07 am
Hi… @snielsen and @James Clarke…
I would be very interested if either of you have progressed at all with your findings since this article was written?
It seems that plenty of products eg. Reflections / Airserver and others now have got a workable solution… it would be great to hear if either of you have got anywhere.
Many thanks in advance…
Charlie
March 20th, 2013 at 6:35 am
@snielsen and @James Clarke Hi everyone, I’m learning Airplay Mirroring for a few days, And I tried to decrypt the video stream as like Shairport4W, but pity failed. The Shairport4W private key is 256 bytes(after decode from base64) while the Mirroring is 72bytes. I don’t know why Apple curtail the private (and public)key(s). Moreover, the length of Airturns public keys are the same between the Shairport4W and Mirroring.
It’s really wonderful if the Mirroring Private key can be cracked from the reflection or others, but it’s rather a BIG project to reverse engineering.
I wonder glad to hear you have got any news about this? And I have done some digging on the internet packages and AES keys. So if any thing I can help to do, please mail me : [email protected]
Thanks!
March 20th, 2013 at 8:39 am
@Charlie, @Chang I haven’t looked into AirPlay mirroring myself lately. So far the developers behind AIrServer and Reflection have kept the details and encryption keys close to their chest. I thought that at this point Apple would have duplicated their functionality but they seem to be content to let these 3rd parties pick up the slack.
I would expect that down the road after Apple has implemented more AirPlay mirroring functionality that these developers will make their work public. But who knows how long that could be.
March 20th, 2013 at 9:30 am
I gave up after a while – I couldn’t work out exactly what was going on. I might revisit it if I have time though…
March 20th, 2013 at 6:48 pm
It ‘s not so easy for apple to prevent the 3rd parties from cracking the key, for I think the private key is fixed in iDevices(maybe included in hardware or software), so if the key changed, the iDevices in the hand of consumers can not work any more with this function. I am not sure am I right?
I have looked into the files of reflection and there are only the .exe and one .dll files are noteworthy. I guess the key maybe hidden in this two and whether it’s a little easier to crack them than AES? 😀
May 7th, 2013 at 1:24 pm
Hi! What’s about this, https://github.com/nto/AirSpeaker/blob/master/AirTunes/CryptoController.m (setPrivateKey) ?
Is it the private key was looking for? Or only for AirTunes?
Thanks so much for you work!
May 8th, 2013 at 10:39 pm
@JM Robles
Thanks Robles, I’m afraid that this private key is for AirTunes only, we can get this key from the open source project shairport4w. I have tried using it to decrypt video but failed.
May 8th, 2013 at 10:59 pm
@Chang
Yes, sorry, I saw it after further investigation.
Now I’m studying AirPlay protocol, if I understand well, the iOS device do a ANNOUNCE to “AirPlay server” with a SDP content where it specifies the session AES key ciphered (with AES IV), not?
Regards!
May 9th, 2013 at 12:33 am
@JM Robles ，
yeah, right. The iDevice will send the AES IV and the encoded AES key (base64) to the server in the ANNOUNCE package.
May 9th, 2013 at 1:01 am
@Chang
Good!
The decode “AES Key” is really “FPLY\x01\x02x01\x00” + 64 byte data.
But I don’t know if that 64 other bytes is the key or no.
May 9th, 2013 at 1:24 am
uh…sorry, I’m not very clear what you mean.
you can refer to the website:
http://nto.github.io/AirPlay.html#screenmirroring
and , in the section 6.1, you will see “The client sends a binary property list with information about the stream, immediately followed by the stream itself. ”
And in the property list , param1 is the AES key, encrypted with FairPlay, 72 bytes.
July 4th, 2013 at 4:00 am
Hi @Chang,
Since the AES key (param1) is encrypted by Fairplay, I think the ‘/fp-setup’ requests may be kind of key exchange in Fairplay protocol? I’ve reverse engineered AirServer a little bit but couldn’t find suitable private key to decrypt the AES key 🙁
July 4th, 2013 at 7:06 pm
hi, @dm4
July 4th, 2013 at 7:08 pm
hi, @dm4
I DO agree that the ‘/fp-setup’ requests exchanged private key, what we need is THE private key .
July 19th, 2013 at 1:35 am
We have already cracked the AirPlay mirror encryption algorithm, the ‘/fp-setup’ encrypted by Fairplay with private key,than the video steam also encrypted by Fairplay with different private key.
the demo video show:
http://v.youku.com/v_show/id_XNTgwMjQzMTk2.html
mail: [email protected]
July 21st, 2013 at 6:17 pm
Wow, it’s so cool, would you like to share it? 🙂
September 6th, 2013 at 10:12 am
Any update on the solution to Airplay Mirror Encryption Algorithm?
October 11th, 2013 at 3:28 am
I have alerady work out it on iPhone/ipad
this is my link video
http://www.tudou.com/programs/view/kNTG-q0qtRY/?lvt=&resourceId=0_07_10_28
my email:[email protected]
November 2nd, 2013 at 2:17 am
Hi everyone!
I´m trying to reverse engineering AirServer in order to understand how AirPlay Mirroring Protocol works. Anyone has idea on how fp-setup communication works?
If anyone could help me to understand the how the keys transmitted this is my email: [email protected]
November 8th, 2013 at 12:12 pm
That first video is questionable if it’s not just an apple tv. And the second video is using photo sharing airplay which is already open, not mirroring.
I’ve managed to pass ft-setup and can get it to start streaming but not figured out how to decrypt param 1 to the AES key.
December 21st, 2013 at 12:34 pm
No change in this?
September 6th, 2014 at 10:59 am
The guys at WaxRain who built AirPin PRO have figured this out as well and have the functionality on their android client which works flawlessly with both iOS and OSX AirPlay Mirroring clients.
October 1st, 2015 at 9:45 pm
@indiekiduk, could you explain how to pass ft-setup step?,