[MeeGo-community] MeeGo Community Development: Apps, Surrounds and the Community OBS

David Greaves david at dgreaves.com
Wed Jan 26 17:31:26 PST 2011 

This is not a short email ... again. Please trim any replies and consider 
changing the subject too.

It's posted here:
   http://mer-l-in.blogspot.com/2011/01/meego-community-development-apps.html

So... lets say we have some device that run MeeGo; what else can it do? Where 
are the apps?

Apps - the area formerly known as "Extras"
---
Borrowing Andrew Flegg (Jaffa)'s mission statement [1] for MeeGo Community Apps: 
we need to ensure a vibrant and quality area for community open source software.

So MeeGo Apps is the community app store and follows on from the succesful Maemo 
Extras[2].

It allows app developers to build MeeGo compliant (and other) apps and 
distribute them to users.

What makes Apps different from a 'random repository' is the community QA process 
that is applied; the objective of this QA process is to permit users to trust 
the apps and to deliver a collection of apps that a commercial vendor could 
enable on a shipping device (as Nokia did with 'Maemo Extras' on the N900).

Clearly then, there needs to be some processes (and automation) to define how QA 
checks are carried out and what to do when certain events occur. We already have 
a process automation sytem (called BOSS - it's is being developed by Nokia and 
is essentially an integration of various OSS libraries) - now we need to decide 
what processes and criteria we use to manage and promote apps. Initially I 
suggest we start with the Maemo Extras process[3] and refine that.

When we talk about process we are also talking about policy. I've discussed this 
below and quite a few of the points apply to Apps.

Now, it's not yet clear what OBS targets Apps should build against. Currently we 
have[4]:

     * MeeGo:1.1:Core
     * MeeGo:1.1:Core:Handset
     * MeeGo:1.1:Core:IVI
     * MeeGo:1.1:Core:Netbook

These correspond to the MeeGo Core and then a number of UXes. However there are 
plans to consolidate them in the main MeeGo OBS - but how should the Apps area 
look? Should a handset user be presented with apps which were only intended to 
run on a netbook? OTOH is this anything to do with the repo and build target - 
should it be determined by metadata and selectively presented by the download 
application?

Well, until this is resolved, an app developer simply needs to enable each 
target that the app supports to make it available in that repository.

I think the discussion up to this point is fairly simple and uncontentious; we 
can start to address the issues mentioned and just get on with it [5] and start 
to deliver MeeGo Apps :)

However, as mentioned, the Apps area allows both MeeGo-compliant and MeeGo-extra 
applications. MeeGo-compliant apps are those which only use libraries in MeeGo 
core/UX; MeeGo-extra apps have a little extra open-source goodness sprinkled 
in... Surrounds.

MeeGo: The Linux Distro
---
Before I tackle 'Surrounds' lets step back a little:

Since MeeGo was announced around a year ago I've been interested in how 
development happens around MeeGo as well as in the centre. The following is 
intended to be a discussion document; there are clearly some large gaps and I 
expect some of my proposals to be shot down - but I really hope that only 
happens when better ones are proposed!

In order to address this I think it's important to ask... what is the point of 
MeeGo? Well, IMHO, MeeGo is primarily a modern linux baseline upon which 
commercial mobile computer products can be delivered. MeeGo's main customers are 
not end-users - they are device vendors; *they* should be the focus of our core 
engineering team's design, delivery and QA effort.

So MeeGo clearly has a focused development area with the main OBS supporting the 
development and delivery of the MeeGo core system software and reference UXes.

Of course MeeGo is also capable of being a wonderful and open linux distro in 
its own right. However I personally think that to be a success, the MeeGo core 
needs to focus on the primary goal and allow more of the "linux distro" side to 
be managed as a surrounding project.

Incidentally, in my day job I care a lot about how MeeGo presents itself to us 
as a vendor. I think it would be a good thing for MeeGo to have it's own 
"reference product" built around the core that it offers to vendors. This would 
expose issues like:

     * How does MeeGo communicate significant upcoming changes to it's customers?
     * How do we deal with package naming clashes?
     * How much of MeeGo is needed for compliance vs the parts that are needed 
to make a reference implementation?
     * With more than one community product: how connected are devices?

So, 'Surrounds' is a proposal that aims to provide a collection of open source 
libraries, tools and applications that support the building of collaborative 
bazaar-style applications and yes, maybe even MeeGo distributions.

Whilst I don't think this is an immediate change, I do think it's an interesting 
direction for MeeGo to take that may allow more focus on the core deliverables.

In the meantime lets look at this "extra open-source goodness" I promised:

Surrounds
---
As we know, the open source world is a world of sharing; we regularly stand on 
the shoulders of our peers and it's considered very bad taste to just 'bundle' a 
library into a monolithic application. (Notice how I didn't mention any kind of 
compliance programme by name - aren't I good!)

Surrounds provides a collection of libraries and tools that encourages this best 
practice way of working in MeeGo and delivers MeeGo-extra apps. The most obvious 
area would be for languages like perl, python and ruby. These languages have 
large numbers of useful libs that clearly are not going to be present in core 
MeeGo. Of course there will be many other tools and applications that would also 
be at home in Surrounds.

Over time, packages may migrate into core MeeGo and equally, some are going to 
be deprecated. Hopefully Surrounds makes both of those actions a little easier 
but we do need to decide how we handle deprecation/promotion of 
applications/libs between core and Surrounds.

There are many complexities when dealing with a collection of packages like this:

QA and Releases
---
The fundamental question is "How is Surrounds QA'ed and released?".

This is a significant undertaking and needs a lot of resource and users. This 
alone means that Surrounds should probably start off slowly and ramp up as MeeGo 
users and developers arrive in larger numbers. The important thing is to prepare 
for the main issues we're likely to face.

I propose that Surrounds is actually only populated on an "as needed" basis; 
tools can be submitted directly but libraries have to be needed by a tool or by 
an application submitted to the Apps area.

Upgrades and concurrent installations
---
For example: let's look at a device that has MeeGo 1.2 installed on it and an 
application "Wowee" built against an imaginary libvisual-1.0 which is in the 
Surrounds-1.2-A stable release of Surrounds.

9 months later many libraries have been updated and developers want to use the 
latest versions... it's time to release a new version of Surrounds-1.2. Sadly 
libvisual is up to version 1.0a with a minor tweak which would break Wowee; not 
only that but the Wowee developer has gone awol (so there's no-one to test it) 
and no-one's stepped up to update it. Even worse there are 50,000 users using 
Wowee which is working like a dream.

Looking at the large number of apps in Maemo Extras I think problems like this 
will arise and are beyond the resources of the community to handle at this point.

A technical solution may be that when Surrounds-1.2-A is installed it goes into 
/opt/surrounds/1.2-A/... and any apps use the appropriate path to find their 
dependencies.

This allows apps that use libraries from Surrounds-1.2-A and Surrounds-1.2-B to 
be installed concurrently even if they use different versions.

I'm not sure how Surrounds-1.2 handles upgrades from MeeGo 1.2 to 1.3.

The message however is that there are many issues like this that it would be 
good to identify in advance. Just ask a maemo developer about the 
"optification"[6] hack :)

Porting vs Maintaining : A MeeGo partner
---
When it comes to populating Surrounds we need to look to the larger linux ecosystem.

Some tools and libraries will have comitted maintainers who have time to monitor 
their packages and fix bugs and security issues in a timely manner.

This is the classic distribution 'maintainer' role.

Some libraries won't. Developers will simply need a particular library and 
'grab' a source rpm that looks like it's good-enough. They throw it at the OBS 
and a few minutes later have an installable package for their application. This 
is what happens in Maemo Extras.

I call this activity 'porting'.

The problem arises when another developer does the same thing a few weeks later 
with a slightly different version of the package. This may cause the original 
app to fail.

Equally, none of those developers want to commit to maintaining the library. So 
when a security issue arises, no-one is ready to update the package.

I'm actually a little concerned about prolific porting - and sadly this is the 
bit that gets the fame and glory : "look how many packages he's ported". The 
truth is that this doesn't bode well for MeeGo in the long run.

My suggestion for this group of packages is to nominate an upstream or partner 
distro with a good security record and a wide base of packages. Surrounds 
releases would closely track this distro (and will likely differ from core 
MeeGo). Packages taken from this distro would be fast tracked as 'ported' rather 
than 'maintained' and only minimal packaging changes would be allowed. This 
would allow Surrounds to leverage the partner-distro's security efforts (and 
hopefully offer benefits to the partner in return).

For obvious reasons I propose openSuse as the partner distro. (For the record 
I'm a Debian guy - but lets be sensible here!)

For equally obvious reasons this is a political minefield :)

Some random policy questions:
---
There are a lot of policy considerations for the community; some I wrote in a 
bit of rant (forgive me Shane). Some apply to Surrounds and Apps, some to the 
OBS in general

     * Licenses: OSI only licenses? GPL3 (considering the "promotion to core" 
target).
     * People: What's the process for absent devs? If someone says "hey, lbt, 
let me upload a new version of that library Shane uploaded" ... do I just let 
her? What if there's a security issue in it? What if you've not been seen in 3 
months? Do maintainers need to demonstrate reliability to handle timely updates?
     * Legal : Do we accept mp3 players? libdecss? What about porn? What about 
apps with a rather rude splash screen? Any patent issues? (Don't forget the 
system physically lives in the "land of the free" so is subject to state seizure 
by the RIAA).
     * Commercial: I've had commercial organisations enquire about using the 
community OBS. I've had them ask to sponsor it. How do we deal with this? My 
first reaction is "no commercial work"... but why not?
     * Quality: Do we insist on a valid bugtracker being identified for all 
Surrounds packages? Must it proxy via a bmc#?
     * Packages scope: Where's the ITP?
     * Can we rebuild MeeGo with some different compiler flags (think non-ssse3 
... yay ... I can run MeeGo on my AMD desktop at last!)
     * Do we sign any of the community repos? What does that mean?
     * Acceptable Use Policy : at what point do we ask users to stop doing what 
they're doing?
     * Non-MeeGo targets? When Smeegol becomes Sogmeel will we allow it to be a 
target?

Again the message is "there's a lot to think about".

The Community OBS
---
We have a MeeGo Community OBS[7] already; and it builds apps for MeeGo.

It is currently managed by Niels Breet (X-fade) and I (lbt) - but we need more 
help. In part this whole post is a way to structure what we'd like to achieve 
and to make it easier to identify tasks.

Structure
---
So what project structure do we need? We've not assumed MeeGo at the top level 
namespace to allow us to support additional distros such as Fremantle and 
hopefully others.

One common use for structure is to provide a QA route. Packages go from some 
sub-project in a developer's home: area to a 'Team' area and then into a Testing 
area. Each transition is subject to policy and quality checks. Defining these 
workflows and structures correctly will make life easier.

Team Areas
---
It makes sense for teams to have collaboration for things like KDE, Gnome, 
Python, LibreOffice, Emacs etc. This would allow more thorough testing to take 
place before releasing either to Surrounds or Apps

However, what's needed to form a team? Can anyone just step up and claim 
Team:KDE or should we ask for some relationship with upstream? Do we just want 
to assess a proposal? Who does the assesment? Do we announce this and give time 
for the community to raise objections or considerations?


PPAs
---
The general policy for *home* projects is "OSI, nothing illegal and don't take 
the piss". We may (or may not) need a more formal one :)

Certainly these PPAs offer developers a huge amount of freedom. They can start 
by uploading and building a package in a "home" directory (which can have a 
structure underneath it for multiple projects). This will allow them to build 
against any of the main targets; any group/community projects or even any other 
community member home projects. Oh, and they can use each others code as a 
baseline too. Once built the packages are automatically published to a repo on 
the community downloads server.

AFAIUI this is a lot like the Ubuntu PPA solution (hence the heading).

At that point developers can stop if they want. They have a complete set of 
repositories (1 per subproject). No painful QA processes for them and no 
'fragmentation' for the MeeGo community. But equally these repo(s) will needed 
to be manually added to a device in order for it to appear in any 
apt-get/zypper/yum etc. Oh, and the Apps downloader team really should pay 
attention here.

A huge benefit here is that to for a user to get at a development version of an 
application they use a specific repository, not a mishmash of randomly unstable 
packages (like the Maemo Extras-devel area).

MeeGo Targets
---
What MeeGo targets do we support?

MeeGo 1.0? Really? Does anyone use that?

MeeGo 1.1 and above make sense. But what architectures? As released is fine - 
but what if there are community rebuilds of non-ssse3?

Also, when MeeGo releases distro updates, do we just rebuild all apps? What 
happens to QA at this point?

MeeGo Snapshots
---
One area to look at is how often we import MeeGo snapshots - and incur the 
rebuild cost for any projects targetting them.

MeeGo Rebuilds
---
Frankly I'm a bit fed up that after a year I can't run MeeGo on my desktop. It 
has an AMD phenom chip and an ATI card - both well supported in the opensource 
world. It's time to build a community MeeGo :)

Do we need a community version targetting the DublinBook? What about the O2 
Joggler? Are there any policy issues here? Can anyone ask for this - there's 
potentially a lot of resource tied up doing this kind of rebuild.

Surrounds
---
What projects are needed in the Surrounds area? Certainly some 
unstable->testing->stable cycle makes sense; but this is not likely to be 
finalised until the workflow is understood.

Fremantle
---
The Fremantle migration. Mmmm. Well it might happen at some point :)

Actions and Ideas
---
     * Establish a task-force/group to coordinate and deliver activity and to 
act as a communications hub.
     * Design and implement OBS project structures and targets for Surrounds, 
Apps, Teams and (suggestions for) home areas
     * Design and communicate process/policy for putting packages into these 
projects and migrating them
     * How do packages move from Community OBS into MeeGo core
     * Implement and automate stuff
     * Structure and enrich the outstanding issues outlined above
     * Design project structures
     * Work up a proposal which sees MeeGo split into Core and Distro
     * Copy this list into a Wiki so it's actually useful

[1] http://lists.meego.com/pipermail/meego-community/2010-December/003021.html
[2] http://maemo.org/downloads/Maemo5
[3] http://wiki.maemo.org/Extras_repository_process_definition
[4] https://build.pub.meego.com/project/list_public
[5] http://wiki.meego.com/Community_Application_Support
[6] http://wiki.maemo.org/Opt_Problem
[7] http://build.pub.meego.com/

lbt

-- 
"Don't worry, you'll be fine; I saw it work in a cartoon once..."

More information about the MeeGo-community mailing list