Want to PUSH... need a kickstart?

So one problem with the PUSH is that it's a bit tough for us software nerds to get into the real world. What would be cool is a nice easy "apt-get install HARDWARE" ... interested? read on...

A few years ago I bought a couple of these boards for about £25.

They provide:

• 5 digital input channels
• 8 digital output channels
• 2 analogue inputs
• 2 analogue outputs (8 bit resolution)
  

You can also have up to 4 of these plugged in via a usb hub.

So, I thought... how hard can it be... and actually it is quite easy. The source is on sourceforge and I'm packaging it so you can pull libvelleman from garage, plug in a board and PUSH :)


As you can see the test program implements K.I.T.T. for you...




So now all we need is someone with an N900 to help me test it...

(Warning: Don't get your hopes up - it appears that the N900 can't (yet?) act as a normal USB host; so no plugging in printers....)

The Summit: Git/Gitorious - Untracked talks: consider adding them...

I proposed a talk at the summit on DVCS and git and it has not been accepted (or rejected) by the talk selection group. This isn't a complaint  :)     . . . . .   however I do worry that it might worth pushing ... what do you think? (Breaking news... sounds like there has been an acceptance - also it may make sense to do a BOF session after any presentations). I still would appreciate feedback - what' important to you?

This message is just to solicit opinions and hopefully will validate their decision. If it causes a rethink then that's fine too.

Heh - I could do without the extra work involved!!

I personally thing think that as a development community with git on garage and gitorious.org we should be making efforts to understand how best to use DVCS processes to collaborate.

Maybe it's just me but I see a lot of devs who are new to DVCS and very few community guidelines on how to use DVCS. Qt uses it but, as we've recently been discussing, it could be going better.

Frankly I don't care which 'good practice' I use - I can go out and find lots of them. But it strikes me that as a community we should at least say "hey, quite a few of us are using this approach - if you don't have any strong preferences then you can use it too"

So why now?

Well, real soon now (I hope) we're going to have 3 different versions to support.

• Fremantle
• Diablo
• Mer

They'll each have different capabilities and some apps just won't care - they'll support Fremantle or Diablo and ignore the others. That's fine.
(Although being open-source you might find patches being sent in to add support for another flavour - how will you cope? Wouldn't you like some help in doing the boring stuff?)

Other apps will decide to embrace that diversity from the start.

So how do we (you) manage the build-variations (ie debian/* may well vary for each of them. Maybe ./configure too)? Do we use branches? How?

Now how do we manage adding features and back-porting simple bug fixes to the stable release whilst you work on that big new feature set.

How are contributions and teams handled?

It sounds horrendous - and it can be!


But actually this is all fairly simple stuff with DVCS once you have it explained and once you grok it - but it's bloody hard to figure out from scratch and it's also very unlikely that you'll arrive at the same solution as another team.

Which means if you're a member of multiple teams you might find they each have different approaches - "whoo hoo!" - not!

So anyway... I thought a talk would be a good idea.

Now at the time no-one had volunteered so I did - some of you may have noticed my name at the bottom of:
  http://www.kernel.org/pub/software/scm/git/docs/git.html
(yes, I'm proud of that contribution)

Now you may think that qualifies me to offer this talk - you'd be wrong ;)

I was hanging around the kernel lists at the time, got interested and acted as an 'editor' pulling together words of wisdom. Since then I've used git a little but it wasn't until we started to need it in Mer that I reviewed the state-of-play and tried to pull in other people's good ideas - so that's what I'd use as a base.

However we also have Johan Sørensen who wrote Gitorious.org - I think having him speak on using git and gitorious would be an opportunity that we shouldn't miss.

Equally there must be developers in Nokia/Trolltech who could say "we know this stuff and this is how we think you might want to do it."

So... speak now...

PS If anyone fancies collaborating and doing a multi-person presentation then I'm up for it.

GrandCentral for Mer... connecting the components

First, I'm sorry but this isn't an N900 post :) instead I've been looking at one of the areas that's holding back Mer. It's fairly simple conceptually: just handling interaction between various parts of the system; and by this I mean things like screen dimming; what happens when you press the power key; shutting down when the battery is low; waking up for an alarm... Things that should just work.  Well, I think I have a potential solution. (Oh, and some of you may be interested... I also found out how far eclipse has come in the UML modelling space too!)

Currently this is handled by powerlaunch; it is really not a bad bit of code and does most of its job well.

Sadly it really sucks at being configurable!

Here's a snippet:
global_key_press_power = timer_set _key_power_long $powerkey_longdelay; timer_set _key_power_medium $powerkey_mediumdelay; set powerkey_short 1; call _key_press_power_short
global_key_release_power = timer_cancel _key_power_long; timer_cancel _key_power_medium; if $powerkey_short _key_release_power_short key_release_power_long


I don't know what it does and the only way I'd be happy changing it would be after extensive research. Frankly it scares me!

Here's what I was thinking we'd do for a 'configuration' syntax in Mer:

# describe some action
def on_wifi_up():
  notify "The wifi is up"
  os.system("mail me@gmail.com -S 'wifi is up'")

connect(wifi, "Connection.up", on_wifi_up)

connect(camera, "Camera.active", takePhotoAfter10Secs)
connect(camera, "Camera.turn",   media.play, "/home/lbt/sounds/camera_click.wav")
connect(powerButton, "Button.longPress", powermenu.show_menu)


This uses a python signal library (like Qt or libsigc++) and provides a set of component objects that emit nice signals like "Connection.up".

The benefits of this approach:

• simple syntax is nice and readable
• interpreted so can easily be edited
• python understood by many so can be easily extended
• re-uses some powerlaunch code (eg powered)

Of course there are drawbacks:

• it's not a config file - it's code (but considering we want to configure functional behaviour then any config syntax needs to be a language... so use a decent one)
• python is quite large (but it's now in memory and .so are available)
• it's another re-write :(  (it only took a few days and I'm using powerlaunch as a prototype for config rules - if I can understand them!!)

So anyway, as I said, I found eclipse Modelling edition...

This says (well, I hope it says):

• There is a system and a user dispatcher
• They're based on a common Dispatcher
• Which contains components (such as Headphone, Menu etc) that can
• send signals (eg when the camera on the N800 is turned)
• take requests (dim the screen)
• provide state information (a headphone is connected)
• be accesed on D-Bus
• The Dispatcher creates Connections which
• listen() for Components to emit signals (also listening on D-Bus)
• run Scriptlets which might
• run arbitary code
• make requests to components
• emit signals that (other) Connections can listen() for
• Scriptlets are part of the Dispatcher

Oh, and the good news is that it's up and running... I'll put the code on gitorious RSN.

Do you take contributions? or is it "Beware of the Leopard"

Open source is about opening things up... right?
Well, yes.
But I think there's more to it than that.

Think about the kernel and how that works. A wide open mailing list (and source repository) with people getting on with it. Fedora? Ubuntu? The same. SGI's XFS filesystem; in fact any number of projects out there.

But what does that even mean? "Getting on with it?"

To me it means discussions, proposals, code, designs, bugs. If you care you can watch; you can even attempt to join in. You can certainly learn about what's happening and why.

So here's the question...

Where's the 'Contribute to Maemo' on maemo.org?

Fremantle is coming out - but it just appears on the download servers. There's no opportunity for the community to contribute. Where are the mailing list archives or the commit logs so I can find who committed that change, when and why?

Opensource means you can't be shy; open doesn't mean "behind closed doors"; it doesn't mean "you get it when we're done".

Open means "this is what we're doing". And "we" includes the community.

I hope we're doing it the right way in Mer.

irc is our main collaboration area and that's open and logged. Not terribly useful as an archive though. We have the chatter mailing list which gives a pretty good summary of key events.

What we do to activate incoming developers to make it easy for them to contribute? Well, we've adopted gitorious as our DVCS and I think the process and UI there makes that a good call. I've also put some effort into describing our 'internal' build processes and what we (ultimately) expect DVCS contributions to look like.
 Carsten has setup a neat little tracker of outstanding tasks so potential contributors can pick up things to work on.

We have some design statements on the wiki and we know that 'track Fremantle and predict Harmattan' are our tenets.

I wondered what Android does?
Hmm : http://source.android.com/roadmap

The Android Open Source Project will publish roadmaps for upcoming releases. The roadmaps will be developed with input from this community, Project Leads, and the Open Handset Alliance

Sounds good.. and the code is up there.

What about contributions?
Well... say no more! A random click got me that gem.

Nokia and Maemo5 .... well, to be frank it looks a lot more like Microsoft.

And that's not that harsh :)

Microsoft provide (I'm told) excellent documentation for developing OSS on their platform.

For Maemo there is some really good developer documentation (seriously - I think this is not shouted about enough); there's superb support for creating OSS on Maemo platforms; and that's absolutely crucial.

Contributing to Maemo... not so much.

Yes, the source is there for the released maemo packages; and there are random directories like this or this   (first one is circa 2005, 2nd April 2008)... browse around, we found some useful stuff in there!
As Carsten said "god knows how i found that first time". There are garage projects too (impossible to find the right ones though), recently we noticed some appearing on gitorious or  git.maemo.org or svn....

However I'm not really aware of anything other than bugzilla for communicating with the development teams; and I don't think that's something the community generally finds to be a 'shining example' :)

We recently added a note by way of an apology for Mer's lack of "apt-get source" but I think Maemo deserves it too:

`...You hadn't exactly gone out of your way to call attention to them had you? I mean like actually telling anyone or anything.'
`But the plans were on display...'
`On display? I eventually had to go down to the cellar to find them.'
`That's the display department.'
`With a torch.'
`Ah, well the lights had probably gone.'
`So had the stairs.'
`But look you found the notice didn't you?'
`Yes,' said Arthur, `yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying "Beware of The Leopard".'

I really don't think it's doom and gloom though - Qt has opened up a lot since Nokia took over. The development has moved out into the open and there's a process to contribute. Gitorious is  being sponsored by Nokia/Qt and I hope the maemo-af team are leading the way.

I really hope that when Harmattan arrives the basement is light and airy; very, very noisy and the leopard has relocated to the lobby!

Accelerating OBS .... !scratchbox

So I've been working on accelerating the Open Build Service recently.

It's a great build environment for Mer and we love it. It uses qemu to emulate arm architectures and provides a pristine chroot build environment using native packages. This approach of using distribution tools rather than a toolchain provides a more consistent build system.

The problem is that this approach is slow.

Following the 80/20 rule, the OBS engineers (thanks Martin, Jan Simon) identified the applications that represented the greatest load on the build system:

• bash
• bzip2
• dpkg
• gzip
• m4
• make
• perl

And of course:

• gcc

So the intention was to accelerate these binaries in the emulator by using host machine code, not emulated code.

The emulator uses binfmt_misc to call qemu for foriegn binaries when it hits them so simply replacing the binaries with host binaries should work.

Well; almost.

The host binaries need their shared libraries; and they live in the same place as the emulated shared libraries... which may be used by non-replaced code. Hmmm.

(Before going any further, lets establish a naming convention: target is the architecture of the chroot and eventual target; host is the architecture of the underlying CPU. Typically for us target is armel, host is x86.)

The approach to solving this problem is to minimise changes from a pure arm build environment. This is achieved by installing both arm and x86 packages; to avoid collisions the x86 packages are installed into /lib-x86 (in the chroot) and of course, shared libraries will fall into /lib-x86/[usr/]lib and not overwrite the target .so files.
The final step is to move selected binaries in /bin and /usr/bin out of the way and replace them with symlinks to the i586 binaries in /lib-x86/[usr/]bin

Most x86 packages are unmodified; any paths they use will relate to the main / directory and the config/data files installed by the target package will be used. (This works in general although the eagle-eyed will spot issues with things like XS modules for perl - not a major problem for a simple build environment though).

Some packages are modified to minimise build times - eg to avoid 64 bit builds; others are modified (eg dpkg) to set the target architecture as a default.

Obviously the host binaries need to link to shared libraries which are not in the normal LDPATH. We don't want to have to rely on environment variables etc so 2 steps are used:

• modify ldlinux.so to look in /lib-x86/lib etc
• modify the rpath value in the ELF header of each replaced binary

This is done by creating a static build of patchelf and modifying the binaries as they are installed.

Finally a modified package has all postinst scripts removed and replaced with something like:

prefix=/lib-x86
$prefix/usr/bin/patchelf --set-rpath $prefix/lib $prefix/usr/bin/make
mv /usr/bin/make /usr/bin/make-orig-x86lib
ln -s $prefix/usr/bin/make /usr/bin/make

Note that some libraries have an rpath and need to be treated with patchelf too.

One final problem arises with fakeroot: it modifies the LD_LIBRARY_PATH to look for libfakeroot-XXX.so This is handled by editing the fakeroot script to append the /lib-x86/ paths in the same way the multi-arch paths are added.

All in all this is actually a very clean solution:

• the modified binary uses the exact same source as the one it replace
• the installation principles mimic biarch (and will, we hope use biarch when it's ready)
  
• only the specific binary executable is replaced - data and scripts come from the target package
• it can easily be switched on or off for a specific binary
• enabling it on the OBS is done in the enclosing project, not the package

Next steps, making it work on the OBS and then ... building a cross-gcc.