AVRCP Metadata

Quite some time has passed since last post and I want to update you on what have been making me busy lately. I’ve got accepted for GSoC2010 with BlueZ again this year I just passed the midterm evaluation. Following is the project I’m working on:

Add metadata and player status information exchange to bluetoothd’s AVRCP plugin

– Abstract –

Today the AVRCP — Audio and Video Remote Control Profile — plugin in bluetoothd supports only basic remote control commands (play, next, stop, etc.). Support for metadata and player status information about the stream being played over bluetooth is missing to fully comply with AVRCP 1.3. In addition, AVRCP 1.4 adds the ability of browsing content over bluetooth, so controlling a media player remotely becomes a richer experience for the end-user.

All of this is specially useful for devices with displays and more elaborated controls, like fancy headsets, cellphones, and car-kits. One of the most illustrative use-case of these technologies is a user which goes into its car with a bluetooth-enabled cellphone with a media player and his music/video collection in it and have a bluetooth car-kit with wheel-integrated controls. The user can then browse and play music/videos on the cellphone — which is possibly attached to a car-stand which also charges the phone’s battery and holds the phone in a position suitable for watching videos — and have meta-information about the media showing on the car-kit’s display without moving his hands from the wheel.

– Project Details –

AVRCP has two roles defined: Controller (CT) and Target (TG). Command frames are sent from the CT to the TG in order to remote control it. Metadata is sent from the TG to the CT. This project aims to add metadata and player status information support to bluetoothd, so it fully complies with AVRCP 1.3 TG specification. To accomplish this goal, we need some level of integration between the media players and bluetoothd.

Most media players supports or are in the process of adoption of a standard control interface via D-Bus called MPRIS — Media Player Remote Interfacing Specification [1] — which current version is 1.0. Discussion about its definition is happening on mpris mailing list [2]. Some players known to implement it, natively or via plugins, are Amarok, Audacious, BMPx, Corn, Dragon Player, Exaile, MPD, QMMP, Songbird, Totem, VLC, XMMS2. MPRIS defines a /Player object with methods not only for controlling players (play, next, pause, etc.) but also for retrieving metadata information (GetMetadata) and player status (GetStatus). Two signals provide this information too (TrackChange and StatusChange). In order to obtain metadata information and player status bluetoothd could call those methods or listen to those signals, and send the info for it AVRCP CT peer.

Although, this model has some limitations. The most obvious one comes from the D-Bus design: media players are started by users, so it’s D-Bus services lives on the session bus; ITOH bluetoothd is a system-wide daemon and can only connect to the system bus. Another issue is that bluetoothd can handle multiple devices concurrently. This means that the following scenario can happen: more than one audio sink connected to bluetoothd receiving different audio streams and willing to receive it’s respective metadata and player status at the same time. bluetoothd would have to map each media application and it’s MPRIS service to the respective audio stream to send the correct information to its peers.

Audio streaming from media applications to bluetoothd, in it’s recommended configuration, is handled by PulseAudio [3]. PulseAudio is a modern sound server which promotes a very complete integration between media applications and sound devices. Also, PulseAudio is a de-facto standard of how to access sound-devices on POSIX OSes, being an integral part of all relevant modern Linux distributions and used in various mobile devices by multiple vendors. PulseAudio is implemented as a user-wide daemon, meaning it can access methods and listen to signals on both the session and system buses. Also, it has information about which application is playing to which sound device. These characteristics makes it a very appropriate solution for the issues pointed before.

The proposed approach to implement this project is to have PulseAudio intermediating data exchange between media players and bluetoothd. For each audio stream going to a bluetooth audio sink it would listen to TrackChange and StatusChange signals coming from the application generating the stream going to that device and send this information to the remote peer through AVRCP calling a method exposed on bluetoothd’s org.bluez.Control interface. This integration could latter be extended for basic player controls, having PulseAudio to listen to bluetoothd signals and call methods on media players MPRIS interfaces to trigger the matching action. Also, a similar approach could be modeled for adding content browsing capability, as defined by AVRCP 1.4 specification.

– Have you worked on a Linux system before? –

I’ve been working with Linux since 2002, when I’ve started the Computer Engineering undergrad. In the beginning I just did the school projects on Linux, but since the middle of 2002 it’s my main system. By that time I had Conectiva on my home computer, RedHat on the school, and Slackware on my home router. After a few years I got in touch with Gentoo, and I liked so much the idea that it became my main distro. In addition, I’ve also started working on a project at University of Campinas, to make a Linux distribution for Itautec, a brazilian IT company, which was Gentoo based. Nowadays I’m using Ubuntu in my laptop, because of the convenience to install and no need of much tweaking, so I have more time to work with other projects, but I’m comfortable using and administrating any distribution.

– Have you contributed to a open source project? if yes, please provide the details –

My first contribution was a little bit frustrating, since I never got any answer (a two lines patch to python-dialog — http://sourceforge.net/projects/pythondialog/ — to support one dialog that was missing). After that I’ve made some performance improvements in ImageMagick and DevIL using OpenMP, both of them being accepted upstream. I’ve also completed two GSoC projects with BlueZ (PulseAudio A2DP modules and A2DP Sink). After that I’ve done the HFP integration into PulseAudio and I’m currently helping Gustavo Padovan with Enhanced Retransmission and Streaming modes into L2CAP, mostly reviewing and testing his implementation. Not related to bluetooth, lately I’ve done some contributions to the Enlightenment Foundation Libraries and E17 WM (fixes on ethumb and emotion, ofono module to E17), and to LightMediaScanner, a lightweight metadata scanner for media files.

– What is your educational Qualification (grad/under-grad) ? –

I’m a Computer Engineer, majored on University of Campinas (Unicamp), Brazil. I’ve started my master degree on Computer Science studies in 2008, on the same university. – Why do you want to do a project involving Bluetooth/BlueZ ? – I’ve already been a GSoC student with BlueZ and was a very good experience. I’ve worked with audio profiles, with most of the times are the profiles that work together with AVRCP, so it would be a kind of continuation from my previous projects, from the user-experience point of view. The initial motivation to choose BlueZ last year was because it’s something that I use on my daily activities and because I’m very enthusiastic about wireless devices and mobility. Besides that, I always had interest about network-related stuff.

– If your application is accepted will it be a part of your graduation process or it will it be just for hobby? –

If I get accepted to this project, it will not be part of my graduation process.

– Give us an *estimative* of your schedule (exams period, etc.) and how much time you would be able to dedicate to the project. –

The project has 2 main parts: integration with media applications, which consists on obtaining metadata and player status from the applications and passing this to bluetoothd, and bluetooth communication, which consists in implementing the “Procedure of Metadata Transfer” and its PDU, as defined in the AVRCP specification, inside bluetoothd’s AVRCP plugin. Both parts can be implemented independently, since the integration between both is going to be done through bluetoothd’s D-Bus API. I plan to start by the bluetooth communication part, so when doing the media applications integration I can call the already implemented AVRCP D-Bus methods. Then I’ll have the bluetooth communication task as the midterm milestone and the media applications integration as the final milestone.

[1] http://xmms2.org/wiki/MPRIS

[2] http://mailman.videolan.org/listinfo/mpris

[3] http://pulseaudio.org/