[MeeGo-ivi] audio architecture example

[Marco Ballesio]

Hi Laci,

On Sat, Feb 19, 2011 at 1:18 PM, Jalics, Laci <laci.jalics at delphi.com> wrote:
> Thanks Marco.
> Comments below.
>
>> -----Original Message-----
>> From: Marco Ballesio [mailto:gibrovacco at gmail.com]
>> Sent: Thursday, February 17, 2011 3:16 AM
>> To: Jalics, Laci
>> Cc: meego-ivi at meego.com
>> Subject: Re: [MeeGo-ivi] audio architecture example
>>
>> Hello,
>>
>> On Wed, Feb 16, 2011 at 6:11 AM, Jalics, Laci <laci.jalics at delphi.com> wrote:
>> >
>> > I've created a thought starter slide here on audio routing/policy management.
>> > http://wiki.meego.com/images/Audio_arch1.pdf (couldn't attach it do to posting
>> size limitations)
>>
>> Great doc, I have a few questions / suggestions (maybe I'll have more
>> in the future, but it's just to begin ;) ).
>>
>> > Essentially all the architectures that I've seen have a dual processor solution:
>> a low speed micro to handle power moding, CAN bus, and early audio in vehicles.
>>  DSPs are common in automotive systems as well.  So, I've created a diagram that
>> starts to capture the hw/sw architecture relating to audio routing and control.
>>
>> - Are audio/video codecs running on the DSP? In such a case you need
>> to draw a connection bw GStreamer and the core, possibly showing the
>> presence of DSP-specific plugins (for instance, something like
>> gst-openmax or gst-dsp). It should be up to the adaptation layer to
>> deliver them.
>>
>
>
> Two thoughts here. The SOC or GPU may provide main CPU support for decoding(which is not shown in diagram).  Also, I'm not sure, but it may make sense to pass the compressed audio instead of uncompressing it with the CPU.

That's exactly what I meant: both gst-dsp and gst-openmax are just
"wrappers" used to interface the GStreamer pipeline with the hw used
to handle encoded data. Al the embedded chipsets have their own
strategy: on TI omap it's an embedded DSP, Atom architecture is using
its own HW accelerated codecs and it's more or less the same for
QualComm and Freescale. Some of them even deliver openmax wrappers
that can be (if we're lucky) plugged automagically in gst-openmax.

In case there's nothing suitable for your architecture it's usually
fairly simple to design and implement custom wrappers (the N900
teaches something with Felipe Contreras' gst-dsp).

One more note: once decoded, data will anyway have to come back to the
CPU for audio mixing/routing/spectrum flattening/ etc.. so a properly
designed set of elements will enable you to easily interface with the
audio server (pulseaudio) or whatever routing strategy (RTSP?) has
been decided to use.

>
>>
>> - HTTP streaming is not effective for audio latencies (I really don't
>> suggest you to use it if it can be avoided). RTP is definitely better,
>> as it allows standard NTP syncronisation across the network entities
>> through RTCP packets (not handled by pulseaudio so far, but from
>> GStreamer). Depending on the degree of reliability and abstraction you
>> may want to connect the "rear seat" with either GStreamer or
>> pulseaudio, but RTP is definitely the transport you want to use.
>>
> Thanks for the correction.
>>
>> - You could expand "Policy Manager" (I'd suggest Resource Policy
>> Framework) into decision point and enforcement points. Of the latter,
>> you'll need at least two separate ones:
>>   - VEH enforcement point.
>>   - Pulseaudio enforcement point.
>
> Could you expand on this.  I see the Resource Policy Framework making and passing the control decisions to the various blocks where routing occurs.

Again, exactly ;) . If routing has to be transparent for applications,
the "blocks where routing occurs" must be handled by appropriate
components named the "enforcement points". See here if you haven't
already:

http://conference2010.meego.com/session/policy-framework-flexible-way-orchestrate-multiple-functionalities-meego-devices

>
>>   - What is the MOST driver supposed to do that cannot be done in
>> ALSA/pulseaudio with the MOST plugin? In case it's not possible to
>> aggregate the layers, you'll probably need a third enforcement point
>> here, otherwise you can handle the thing through the pulseaudio
>> enforcement point.
>>
>
> I'm not an expert on MOST, but my assumption is that ultimately the MOST driver delivers audio and control information to the MOST bus.  We need to pass that on from Pulse and the "Resource Policy FW"

So you'll very likely need a custom enforcement point, unless it
cannot be handled with ALSA/pulseaudio.

>
>> > The vehicle processor on the left controls the DSP as well as manages the
>> dedicated hardware blocks relating to early audio and external blocks (tuner,
>> chime, CD, HD radio, ...).
>>
>> So if understood well the architecture, FM radio data never goes
>> through the CPU (direct I2S connection)... how are you planning to
>> route it to the "Rear seat" then?
>>
> One strategy I can think of is putting the rear seat on the MOST bus and routing it out the DSP onto the MOST bus.

It's possible, depending on the target cost of design and
implementation for such an architecture + HW wrt a plain ethernet-like
network (e.g. IP-over-CAN). Not knowing how common is -in the IVI
market- the approach you're proposing, I assume the latter being less
expensive as BOM and less a pain-in-the-neck to design, implement and
maintain.

Indeed, there may be latency issues opening a can of worms, but RTP
should grant very low ones (ten of milliseconds for a very well tuned
system), and I don't really know any reasons for network jitter here..

Regards

>
>> > Once the main processor wakes up becomes the master audio manager by controlling
>> the vehicle processor via an SPI interface.  Within the main processor exist the
>> policy manager that needs to manage PulseAudio, the Vehicle Processor and the Most
>> bus. I tried to identify the control lines and the audio data lines.
>> >
>> > I'm particularly interested in how the MeeGo Policy Framework and PulseAudio
>> support this type of architecture.
>> >
>> > The diagram is not perfect, but I'm hoping it will stimulate discussion.
>>
>> It's already an excellent starting point.
>>
>> Regards
>>
>> > No worries,
>> > laci
>> >
>> > Laci Jalics
>> > Engineering Group Manager (Platform Group) Delphi M/C 483-300-130 3000
>> University Drive Auburn Hills, MI 48326
>> > Phone: 248-732-1825
>> > Fax: 248-836-1774
>> >
>> >
>> >
>> >
>> **********************************************************************************
>> ****** Note: If the reader of this message is not the intended recipient, or an
>> employee or agent responsible for delivering this message to the intended
>> recipient, you are hereby notified that any dissemination, distribution or copying
>> of this communication is strictly prohibited. If you have received this
>> communication in error, please notify us immediately by replying to the message
>> and deleting it from your computer. Thank you.
>> **********************************************************************************
>> ******
>> > _______________________________________________
>> > MeeGo-ivi mailing list
>> > MeeGo-ivi at lists.meego.com
>> > http://lists.meego.com/listinfo/meego-ivi
>> >
> **************************************************************************************** Note: If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. ****************************************************************************************
>