Sunday 23 May 2010

I like crashes like that...

So I'm ssh'ed into my kerberos kdc server and at the same time I'm playing with nfsv4 and I clearly push the boundary somewhere and my NFS server has a kernel BUG and goes wibbly... ctrl-alt-sysreq S-U-B wibbly...

Now the kerberos kdc is a VM on the nfs server; which also holds the home directory for my desktop. So my desktop hangs hard.... toast.

I wander over to the nfs server and shut it down as best I can and reboot it.

It comes back... and as it does my desktop wanders into life, my ssh session to the VM is still active; nothing happened.

I like crashes like that :)

Saturday 22 May 2010

Kerberos and ssh

So you're setting up Kerberos and you've got a ticket with kinit

the target has a host/@REALM

but ssh fails with:
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context

Well the first thing to realise is that you're probably not using gssapi-keyex, you're probably using gssapi-with-mic (see http://www.ietf.org/rfc/rfc4462.txt if you want to know more).

Running:
/usr/bin/sshd -D -ddd -e
may give
debug2: input_userauth_request: try method gssapi-with-mic
debug3: mm_request_send entering: type 38
debug3: mm_request_receive_expect entering: type 39
debug3: mm_request_receive entering
debug3: monitor_read: checking request 38
debug1: Unspecified GSS failure.  Minor code may provide more information
Key table entry not found

The problem I has was a typical one in kerberos setups... DNS and name resolution has to work.
In this case a simple entry in the hosts file with a non-fqdn and there are very few clues.

So maybe you'll google some of that and it'll help... :)

Sunday 16 May 2010

OBS and Fremantle ... huge success :) and HELP!!!!

As you may know I recently volunteered to setup a community OBS in my "spare time" (hah, right!). This is a progress report because I've reached a significant milestone... the following 193 applications have been built from 'Extras' most on both ARM and X86:

accdisplay, acl, actman, adblock-plus-1.0, adflashblock-css, alienblaster, anarchism, attitude, attr, audiofile, avahi, battery-eye, belltower, blubbels, bluetooth-dun, bluezwitch, brainparty-data, burgerspace, catorise, clutter, clutter-gtk, copernicium, crimson, currencyconverter, datetoday-home-widget, dbus-python, de-lite, dialcentral, doom-wad-shareware, easy-deb-chroot, easypark, ejpi, fapn, feedingit, feedparser, flac, flatzebra, fm-boost, fmradio, freecell4maemo, fschedule, fuse, gboggle, glibmm2.4, glogarchive, gnome-python, gonvert, goocanvas, gpe-icons, gpodder-icon-theme, greekiradio, gst-plugins-good0.10-flv, gst-plugins-good0.10-mkv, gst-plugins-good0.10-ogg, gst0.10-python, gstreamer0.10-ffmpeg, gtk+extra2, gweled, headphoned, healthcheck, hex-a-hop, host, irssi, jamaendo, json-glib, kernel-power-settings, libart-lgpl, libcap, libdaemon, libevent, libffi, libgnomecanvas, libgnomeprint, libgtkhtml2, libliqbase, libmad, libmicrofeed, libmodplug, libraw-lite, librsvg, libsidplay, libsigc++-2.0, libsoup2.26-1, libxinerama, liqflow, lirc, lybniz, lzma, lzo2, madbomber, maemo-optify, mafw-lastfm, mancala, mastory, masudoku, mauku, mclock, meanwhile, mediabox, microfeed, microfeed-providers-unstable, microfeed-utils, milkyway-wallpaper, mirror, mplayer, mspede, mutagen, mygpoclient, n900-fmrx-enabler, n900fly, nako, ncalc, nmap, notify-python, omweather, openvpn, openvpn-applet, panucci, pedometerhomewidget, pexpect, polipo, poppler, proximityd, pycairo, pycurl, pygame, pygobject, pygtk, pygtkeditor, pyinotify, pymaemo-optify, pypianobar, pyrecipe, python-central, python-dbus, python-evolution, python-facebook, python-gdata, python-gtkhtml2, python-imaging, python-location, python-numeric, python-support, python-twitter, python-xml, python-ystockquote, qtsixa, quickbrownfox, quicknote, recaller, reflect, rfk, rmz-helsinki-wallpaper, rmz-senaatintori-wallpaper, rootsh, rsync, sdl-net1.2, sdl-ttf2.0, sdlgfx, sdlpango, sense, sgt-puzzles, shortcutd, sip4, sleeper, solarwolf, sopwith, sqlite, ssh-status, stockthis, storageusage, supertux-stable, systeminfowidget, tar, tennix, thai-ttf, tickstill, tor, touchsearch, tsocks, uqm-3do-data, uqm-data, uremote, ussd-common, ussd-widget, vim, vor, vultures, wget, witter, worldtv99, xkblayouts-rx51-ru, zoutube

There are a lot that don't build but I think we're at the point where I need help from some experts and it's time to open the beta service up to early adopters. You can see what I've done to setup Fremantle on the OBS.

As I said HELP
I now need to fix build issues like:
installing hildon-update-category-database
/usr/bin/hildon-update-category-database: I don't have write permission on /usr/share/mime.
Try rerunning me as root.
dpkg: error processing hildon-update-category-database (--install):
 subprocess post-installation script returned error exit status 1
Errors were encountered while processing:
 hildon-update-category-database
exit ...
which blocks a large number of applications

and package specific ones like this in libogg:
automake: configure.in: installing `./mkinstalldirs'
Makefile.am:3: require version 1.6, but have 1.4-p6
include/ogg/Makefile.am:6: invalid variable `nodist_ogginclude_HEADERS'
autoreconf: automake failed with exit status: 1
make: *** [configure-stamp] Error 1
dpkg-buildpackage: failure: debian/rules build gave error exit status 2

I can fix almost all of them myself but that's not the point - we need a team to make this work. Please get in touch and help.

Community Building for Maemo and MeeGo - What does the OBS *Do*?

In a closed environment you use what's in the SDK and you get your own little space; talking to others is discouraged and sharing and re-use outside the blessed area is frowned upon. This dictatorial style has the advantage of making life easier for the vendor. In an open world we have more interactions... and as students of networks know: increased connectivity brings increased complexity as well as increased benefits.  So this is an initial proposal for the organisation of OBS build projects and packages to support a QA process into an app-store / Extras or garage-like environment.  I'll introduce some basic OBS concepts and then describe how this might work. I would like this to raise awareness of some potential complexities that we may face and get some thoughts on how to deal with them. ...... Oh, and this is about both Maemo and MeeGo.

First an OBS Intro
  1. It's a build system (fully local) and a build service (like the autobuilder).
  2. You put source into it and say "use this repository" and it spits binary packages out.
  3. It builds a minimal and "clean" SDK using the repository you told it to use
  4. It has packages - a package corresponds to a tarball and a spec/dsc
  5. It has projects - a project is like a directory with packages
  6. Packages are 'published' into a corresponding repository (which can be shared with others)
  7. The published repositories can also be used by devices to download binary packages.

There is a lot more to it; scheduling, dependency based building, multiple architecture support, home projects, package signing...

The 'trick' then, is to set up a structure of projects that we can use to support the publishing and sharing activities that we need.

So What are our Goals?
We want:
  • to allow freedom for developers to develop;
  • to provide a great build service and SDK
  • to work with the core distro as much as possible
  • to provide excellent quality assured applications for our "app store";
  • and for MeeGo, to extend the core distro with a community supported area for libraries and packages that are depended upon by more than one application.

First thing we get: Individual Homes and PPAs
You'll start by uploading and building a package in your "home" directory (which can have a structure underneath it for multiple projects). This will allow you to build against any of the main distros; any group/community projects or even any other community member home projects. Oh, and they can use your code as a baseline too. Once built your code is automatically published to a repo on the
community downloads server.

This is a lot like the Ubuntu PPA solution.

At that point you can stop if you want. You have a complete set of repositories (1/subproject). No painful QA processes. No 'fragmentation'. But equally your repo(s) will needed to be manually added to a device in order for it to appear in any apt-get/zypper/yum etc.

A huge benefit here is that to get at a development version of an application you use a specific repository, not a mishmash of randomly unstable packages like Extras-devel.

For the pros: Community QA'ed Repository - Extras
If you'd like your application to appear in the Extras repo which is pre-installed on devices then you can submit to the QA process.

First you register for the package name, then submit your code to the distro promoter; there it is QA checked, copied to Extras:Testing, built, QA'ed again and becomes available in the Testing repository.

<here be dragons> The community testers then approve your code. </dragons>

(I know that's a contentious point - and I don't have an answer. I do however have confidence that we can solve the problem through collaboration and co-operation.)

Once approved the code is copied to the Extras:Stable repository for publication.

Conclusion

Much of this text is online at:
http://wiki.maemo.org/OpenSuse_Build_Service
(I really hate it when things get locked up in blog posts and forums)

How can you help?

Some tasks needed to get the OBS running:
  • OBS Installation - a detailed guide on how the system was setup.
  • Application QA Process on the OBS - a proposal
  • Setting up Fremantle - WIP - post to follow
  • Fixing the Fremantle SDK - mmmm
  • Setting up cross-building - started, not imported yet
  • An OBS Autobuilder queue - not started
  • Reporting - not started

Also see:

Friday 7 May 2010

It was the Dawn of the 3rd Age...

...and not only is the Shadow War over but it looks like we will have a community OBS thanks to the team formerly known as maemo.org infrastructure :)

I'm really excited as I see this as a way for the Maemo and Moblin development communities to transition smoothly into a MeeGo future; we should eventually be able to offer an easy way to build applications for Fremantle, Harmattan, MeeGo and future device OSes too.


So what's happened?

A while ago the RWG proposal was begun and we've been pushing it ever since. Tero, Mike, Neils and others have been working on the CAT proposal as well as the RWG. We got together on irc and had a chat about the scope and how to proceed.

Last week the TSG agreed we need something (kinda!) - so we decided that we'd go ahead and do it.

We now (thanks to Tero & Nokia?) have a test machine behind a firewall and I've setup an OBS (more than you ever wanted to know... but you said to work in the open, right?) to prototype how we'd deploy it for community use. This has a link to the MeeGo OBS and successfully builds MeeGo packages.

Where are we going?

I don't know yet; but here's what I'm proposing (my ideas tempered with some input from others):

Initially we should offer a MeeGo OBS and a Maemo OBS. There are corporate issues around this and whilst my personal aim is a single build system able to build debs and rpms against any M[aemG]*o distro, that isn't the first step.

The MeeGo OBS will probably offer home directories and a way to build against the MeeGo weekly snapshots. We may be able to offer a 'live' feed if some OBS bugs are resolved and we have the build resources.

The Maemo OBS will provide an alternative to the autobuilder for Fremantle and, eventually, for Harmattan. This makes sense as our longer term future with MeeGo is clearly the OBS.

One thing I dread is having to go to each device vendor's build system to make build my app for their community app store; so maybe off into the future we'll even provide a neutral place where we can support multiple vendors: a single build system that can build community MeeGo apps for devices from any vendor that participate.

Another area of interest to me is something I call "Surrounds" (yes, Quim, I know... it needs work). This isn't my idea... lots of software products and distros have it. It's that chaotic area around a product where 'outsiders' or community people contribute their "unsupported" bits and pieces. Actually this deserves a post on its own. Anyhow, I think the OBS will make managing this a lot easier for the community.

There is still a lot to do...

We've asked a few community devs to join (and please come and talk to us if you feel you can help - bear in mind that we need documentation and build system process experience more than development skills).

A big task is the setup of an OBS build system for Fremantle. I managed to get this 80% done a while ago so that work needs digging up and finishing off. Once that's done we will need a lot of help validating all the Extras apps.

Yet more work is required on my cross-compilation system used by the OBS. That was deployed for Mer and we need to redeploy it for Fremantle too.

There is work on reporting systems and QA automation systems going on that we can re-use which will help in transforming the autobuilder promotion type processes and figuring out what is and isn't needed in the OBS world.

Speaking of which ... do we take this opportunity to tweak the Extras promotion policy?
I think we need something like the Devel->Testing->Stable process for Maemo apps - but maybe we should be working with the CAT guys to figure this one out?

Finally please don't ask me when you'll get to use it, we don't have a schedule yet :(

However, do feel free to ask me what we're working on or what we've done or what you can help with.