MeeGo Community Development: Apps, Surrounds and the Community OBS

So... lets say we have some device that run MeeGo; what else can it do? Where are the apps?

Apps - the area formerly known as "Extras"

Borrowing Andrew Flegg (Jaffa)'s mission statement for MeeGo Community Apps: we need to ensure a vibrant and quality area for community open source software.

So MeeGo Apps is the community app store and follows on from the succesful Maemo Extras.

It allows app developers to build MeeGo compliant (and other) apps and distribute them to users.

What makes Apps different from a 'random repository' is the community QA process that is applied; the objective of this QA process is to permit users to trust the apps and to deliver a collection of apps that a commercial vendor could enable on a shipping device (as Nokia did with 'Maemo Extras' on the N900).

Clearly then, there needs to be some processes (and automation) to define how QA checks are carried out and what to do when certain events occur. We already have a process automation sytem (called BOSS - it's is being developed by Nokia and is essentially an integration of various OSS libraries) - now we need to decide what processes and criteria we use to manage and promote apps. Initially I suggest we start with the Maemo Extras process and refine that.

When we talk about process we are also talking about policy. I've discussed this below and quite a few of the points apply to Apps.

Now, it's not yet clear what OBS targets Apps should build against. Currently we have:

• MeeGo:1.1:Core
• MeeGo:1.1:Core:Handset
• MeeGo:1.1:Core:IVI
• MeeGo:1.1:Core:Netbook

These correspond to the MeeGo Core and then a number of UXes. However there are plans to consolidate them in the main MeeGo OBS - but how should the Apps area look? Should a handset user be presented with apps which were only intended to run on a netbook? OTOH is this anything to do with the repo and build target - should it be determined by metadata and selectively presented by the download application?

Well, until this is resolved, an app developer simply needs to enable each target that the app supports to make it available in that repository.

I think the discussion up to this point is fairly simple and uncontentious; we can start to address the issues mentioned and just get on with it and start to deliver MeeGo Apps :)

However, as mentioned, the Apps area allows both MeeGo-compliant and MeeGo-extra applications. MeeGo-compliant apps are those which only use libraries in MeeGo core/UX; MeeGo-extra apps have a little extra open-source goodness sprinkled in... Surrounds.

MeeGo: The Linux Distro

Before I tackle 'Surrounds' lets step back a little:

Since MeeGo was announced around a year ago I've been interested in how development happens around MeeGo as well as in the centre. The following is intended to be a discussion document; there are clearly some large gaps and I expect some of my proposals to be shot down - but I really hope that only happens when better ones are proposed!

In order to address this I think it's important to ask... what is the point of MeeGo? Well, IMHO, MeeGo is primarily a modern linux baseline upon which commercial mobile computer products can be delivered. MeeGo's main customers are not end-users - they are device vendors; *they* should be the focus of our core engineering team's design, delivery and QA effort.

So MeeGo clearly has a focused development area with the main OBS supporting the development and delivery of the MeeGo core system software and reference UXes.

Of course MeeGo is also capable of being a wonderful and open linux distro in its own right. However I personally think that to be a success, the MeeGo core needs to focus on the primary goal and allow more of the "linux distro" side to be managed as a surrounding project.

Incidentally, in my day job I care a lot about how MeeGo presents itself to us as a vendor. I think it would be a good thing for MeeGo to have it's own "reference product" built around the core that it offers to vendors. This would expose issues like:

• How does MeeGo communicate significant upcoming changes to it's customers?
• How do we deal with package naming clashes?
• How much of MeeGo is needed for compliance vs the parts that are needed to make a reference implementation?
• With more than one community product: how connected are devices?

So, 'Surrounds' is a proposal that aims to provide a collection of open source libraries, tools and applications that support the building of collaborative bazaar-style applications and yes, maybe even MeeGo distributions.

Whilst I don't think this is an immediate change, I do think it's an interesting direction for MeeGo to take that may allow more focus on the core deliverables.

In the meantime lets look at this "extra open-source goodness" I promised:

Surrounds

As we know, the open source world is a world of sharing; we regularly stand on the shoulders of our peers and it's considered very bad taste to just 'bundle' a library into a monolithic application. (Notice how I didn't mention any kind of compliance programme by name - aren't I good!)

Surrounds provides a collection of libraries and tools that encourages this best practice way of working in MeeGo and delivers MeeGo-extra apps. The most obvious area would be for languages like perl, python and ruby. These languages have large numbers of useful libs that clearly are not going to be present in core MeeGo. Of course there will be many other tools and applications that would also be at home in Surrounds.

Over time, packages may migrate into core MeeGo and equally, some are going to be deprecated. Hopefully Surrounds makes both of those actions a little easier but we do need to decide how we handle deprecation/promotion of applications/libs between core and Surrounds.

There are many complexities when dealing with a collection of packages like this:

QA and Releases

The fundamental question is "How is Surrounds QA'ed and released?".

This is a significant undertaking and needs a lot of resource and users. This alone means that Surrounds should probably start off slowly and ramp up as MeeGo users and developers arrive in larger numbers. The important thing is to prepare for the main issues we're likely to face.

I propose that Surrounds is actually only populated on an "as needed" basis; tools can be submitted directly but libraries have to be needed by a tool or by an application submitted to the Apps area.

Upgrades and concurrent installations

For example: let's look at a device that has MeeGo 1.2 installed on it and an application "Wowee" built against an imaginary libvisual-1.0 which is in the Surrounds-1.2-A stable release of Surrounds.

9 months later many libraries have been updated and developers want to use the latest versions... it's time to release a new version of Surrounds-1.2. Sadly libvisual is up to version 1.0a with a minor tweak which would break Wowee; not only that but the Wowee developer has gone awol (so there's no-one to test it) and no-one's stepped up to update it. Even worse there are 50,000 users using Wowee which is working like a dream.

Looking at the large number of apps in Maemo Extras I think problems like this will arise and are beyond the resources of the community to handle at this point.

A technical solution may be that when Surrounds-1.2-A is installed it goes into /opt/surrounds/1.2-A/... and any apps use the appropriate path to find their dependencies.

This allows apps that use libraries from Surrounds-1.2-A and Surrounds-1.2-B to be installed concurrently even if they use different versions.

I'm not sure how Surrounds-1.2 handles upgrades from MeeGo 1.2 to 1.3.

The message however is that there are many issues like this that it would be good to identify in advance. Just ask a maemo developer about the "optification" hack :)

Porting vs Maintaining : A MeeGo partner

When it comes to populating Surrounds we need to look to the larger linux ecosystem.

Some tools and libraries will have comitted maintainers who have time to monitor their packages and fix bugs and security issues in a timely manner.

This is the classic distribution 'maintainer' role.

Some libraries won't. Developers will simply need a particular library and 'grab' a source rpm that looks like it's good-enough. They throw it at the OBS and a few minutes later have an installable package for their application. This is what happens in Maemo Extras.

I call this activity 'porting'.

The problem arises when another developer does the same thing a few weeks later with a slightly different version of the package. This may cause the original app to fail.

Equally, none of those developers want to commit to maintaining the library. So when a security issue arises, no-one is ready to update the package.

I'm actually a little concerned about prolific porting - and sadly this is the bit that gets the fame and glory : "look how many packages he's ported". The truth is that this doesn't bode well for MeeGo in the long run.

My suggestion for this group of packages is to nominate an upstream or partner distro with a good security record and a wide base of packages. Surrounds releases would closely track this distro (and will likely differ from core MeeGo). Packages taken from this distro would be fast tracked as 'ported' rather than 'maintained' and only minimal packaging changes would be allowed. This would allow Surrounds to leverage the partner-distro's security efforts (and hopefully offer benefits to the partner in return).

For obvious reasons I propose openSuse as the partner distro. (For the record I'm a Debian guy - but lets be sensible here!)

For equally obvious reasons this is a political minefield :)

Some random policy questions:

There are a lot of policy considerations for the community; some I wrote in a bit of rant (forgive me Shane). Some apply to Surrounds and Apps, some to the OBS in general

• Licenses: OSI only licenses? GPL3 (considering the "promotion to core" target).
• People: What's the process for absent devs? If someone says "hey, lbt, let me upload a new version of that library Shane uploaded" ... do I just let her? What if there's a security issue in it? What if you've not been seen in 3 months? Do maintainers need to demonstrate reliability to handle timely updates?
• Legal : Do we accept mp3 players? libdecss? What about porn? What about apps with a rather rude splash screen? Any patent issues? (Don't forget the system physically lives in the "land of the free" so is subject to state seizure by the RIAA).
• Commercial: I've had commercial organisations enquire about using the community OBS. I've had them ask to sponsor it. How do we deal with this? My first reaction is "no commercial work"... but why not?
• Quality: Do we insist on a valid bugtracker being identified for all Surrounds packages? Must it proxy via a bmc#?
• Packages scope: Where's the ITP?
• Can we rebuild MeeGo with some different compiler flags (think non-ssse3 ... yay ... I can run MeeGo on my AMD desktop at last!)
• Do we sign any of the community repos? What does that mean?
• Acceptable Use Policy : at what point do we ask users to stop doing what they're doing?
• Non-MeeGo targets? When Smeegol becomes Sogmeel will we allow it to be a target?

Again the message is "there's a lot to think about".

The Community OBS

We have a MeeGo Community OBS already; and it builds apps for MeeGo.

It is currently managed by Niels Breet (X-fade) and I (lbt) - but we need more help. In part this whole post is a way to structure what we'd like to achieve and to make it easier to identify tasks.

Structure

So what project structure do we need? We've not assumed MeeGo at the top level namespace to allow us to support additional distros such as Fremantle and hopefully others.

One common use for structure is to provide a QA route. Packages go from some sub-project in a developer's home: area to a 'Team' area and then into a Testing area. Each transition is subject to policy and quality checks. Defining these workflows and structures correctly will make life easier.

Team Areas

It makes sense for teams to have collaboration for things like KDE, Gnome, Python, LibreOffice, Emacs etc. This would allow more thorough testing to take place before releasing either to Surrounds or Apps

However, what's needed to form a team? Can anyone just step up and claim Team:KDE or should we ask for some relationship with upstream? Do we just want to assess a proposal? Who does the assesment? Do we announce this and give time for the community to raise objections or considerations?

PPAs

The general policy for *home* projects is "OSI, nothing illegal and don't take the piss". We may (or may not) need a more formal one :)

Certainly these PPAs offer developers a huge amount of freedom. They can start by uploading and building a package in a "home" directory (which can have a structure underneath it for multiple projects). This will allow them to build against any of the main targets; any group/community projects or even any other community member home projects. Oh, and they can use each others code as a baseline too. Once built the packages are automatically published to a repo on the community downloads server.

AFAIUI this is a lot like the Ubuntu PPA solution (hence the heading).

At that point developers can stop if they want. They have a complete set of repositories (1 per subproject). No painful QA processes for them and no 'fragmentation' for the MeeGo community. But equally these repo(s) will needed to be manually added to a device in order for it to appear in any apt-get/zypper/yum etc. Oh, and the Apps downloader team really should pay attention here.

A huge benefit here is that to for a user to get at a development version of an application they use a specific repository, not a mishmash of randomly unstable packages (like the Maemo Extras-devel area).

MeeGo Targets

What MeeGo targets do we support?

MeeGo 1.0? Really? Does anyone use that?

MeeGo 1.1 and above make sense. But what architectures? As released is fine - but what if there are community rebuilds of non-ssse3?

Also, when MeeGo releases distro updates, do we just rebuild all apps? What happens to QA at this point?

MeeGo Snapshots

One area to look at is how often we import MeeGo snapshots - and incur the rebuild cost for any projects targetting them.

MeeGo Rebuilds

Frankly I'm a bit fed up that after a year I can't run MeeGo on my desktop. It has an AMD phenom chip and an ATI card - both well supported in the opensource world. It's time to build a community MeeGo :)

Do we need a community version targetting the DublinBook? What about the O2 Joggler? Are there any policy issues here? Can anyone ask for this - there's potentially a lot of resource tied up doing this kind of rebuild.

Surrounds

What projects are needed in the Surrounds area? Certainly some unstable->testing->stable cycle makes sense; but this is not likely to be finalised until the workflow is understood.

Fremantle

The Fremantle migration. Mmmm. Well it might happen at some point :)

Actions and Ideas

• Establish a task-force/group to coordinate and deliver activity and to act as a communications hub.
• Design and implement OBS project structures and targets for Surrounds, Apps, Teams and (suggestions for) home areas
• Design and communicate process/policy for putting packages into these projects and migrating them
• How do packages move from Community OBS into MeeGo core
• Implement and automate stuff
• Structure and enrich the outstanding issues outlined above
• Design project structures
• Work up a proposal which sees MeeGo split into Core and Distro
• Copy this list into a Wiki so it's actually useful