MidnightBSD Developer Journal

I created a new MidnightBSD group on LinkedIn  for developers and users.

Over the weekend, I made a number of changes to rc.d.  In particular, networking and file system mounting scripts were altered.  Please report problems if they are encountered.

On the upside, more of the ZFS stuff is in place.  

Yesterday, I added bwi(4) to MidnightBSD.  It's still quite experimental, but it's working with my LINKSYS pccard.  In fact, there are problems using it with ndis in CURRENT, but it works with bwi(4).  

In case you're curious, I use WPA and have wpa_supplicant kicking in on boot.  So far, I've only tested it with very light usage (ssh).  It's probably not as stable as in DragonFly or OpenBSD.   

The Huawei E220 modem has been added to usbdevs and ubsa(4).  There are still some configuration headaches. 

See http://iswandi.blogs.yarsi.ac.id/2008/09/18/huawei-e220-under-freebsd/

A security issue was recently corrected in the kernel on 0.1.1, 0.2 and current (0.3) MidnightBSD.  Be sure to update your system.  For 0.2.1, this is 0.2.1-RELEASE-p4.

Users of current should also consider updating their src tree if they have not recently.  Most recently, env was fixed.  

I updated the time zone files today (2008i) to account for things like the added leap second, and various international changes that have occurred since March 2008.   

There was a bug in imake on current that ctriv@ corrected today.  This in combination with one of the xorg config files ports caused some problems in current.  Users of current should update their ports tree and rebuild these ports.

As of this weekend, MidnightBSD i386 and AMD64 are building with GENERIC kernel.  Several kernel modules still need work as well as some bugs ironed out.  Users who attempted to build MidnightBSD earlier this weekend may have it a snap while rebooting.  If you get an error about a missing libutil.so.7, please copy the file from /usr/lib or /usr/obj ... to /lib in single user mode.  You can use the mount, ls etc commands from /rescue to do so.  

I noticed an odd bug with mports on CURRENT, but it might be a localized problem.  Packages are not picked up even though their built and placed in /usr/mports/Packages/<archhere>/All

Assume many ports are broken on CURRENT until further notice.  We'll probably do a magus run in the next few weeks to determine the exact problems.   

I've been running CURRENT on my desktop for a few days.  So far, the results are promising.  There are many things to do.  I've had luck on amd64, but i386 is still broken. (progress is being made)  I have not tested sparc64 due to the length of time required to build it.  I will begin testing it after we get intel based stuff working properly.  

You can follow the progress on http://tinderbox.midnightbsd.org/  That process does a more exhaustive LINT kernel test which is not going well on all three architectures.  GENERIC is working on amd64.   

I built a live cd from the amd64 build on my desktop and tested it on two intel macs.  The first was an iMac (early 2008) which booted, and dhclient was started manually with success.  I was able to ssh to other systems, etc.  The wireless card is not supported (broadcom), but the msk(4) works fine.  The SATA and IDE controllers were detected properly, but I have not yet done a hard drive install to verify it's working well.  The second system was a Mac Pro (2 Ghz, 5GB ram, Pioneer SATA optical drive).  This system did not boot completely.  The video worked unlike 0.2.1 release including display on both LCDs (dual head)  It froze around GPT_LABEL i think.  I suspect it's having trouble with the sata optical drive.  The SMP kernel seemed ok as all the cores "launched".  

We have a new mirror for MidnightBSD.  Currently, the mirror is rsyncing so not all files are up yet.  We've posted it on the download page, in mports sites.mk (so it's a backup for files), and in the metalink file for 0.2.1-release.

Special thanks to Secution 

I finally broke down and setup a torrent tracker for MidnightBSD.  We're still working out some bugs with it and could use some help testing.  You can get the torrents at http://www.midnightbsd.org/torrents/

 Once we get this working, we'll post torrents for all architectures for the latest release. 

 For those german speakers, we had a brief mention here.  A good part of that section is about PC-BSD, but still interesting.

Today, I've been working on syscons, twa, twe, and tx.  I've had some interesting suggestions brought to me regarding sub notebooks.  

First, I'd like to start off with some good news.  Our last few magus runs have shown a noticeable improvement in mports.  The tree again appears to be stable with only 5 ports failing.  Remember that you need to update your OS if you haven't already to take advantage of recent mports.  It is a good idea to update your system anyway as we've had several security updates since 0.2.1 was released.  I'd rather avoid doing another 0.2.x release, but it might be something for consideration for i386 at least.  

The crazy merge we're doing is moving along.  We missed the November completion date, but I feel that we've made progress.  Each architecture has a unique problem right now and a few common ones.  amd64 will probably be working first.  The world can be built on amd64 and sparc64 now.  i386 is only failing with pcc (which can be commented out).  At this point, we're working on updating device drivers in the kernel.  I'm in the middle of working on the ATA code.  Sadly, some areas of the kernel had not yet been touched and I've brought many of them in line with FreeBSD 7.  Other parts of the kernel had many local changes which require case by case decisions. I'm still not happy with this approach, and would rather have had more assistance maintaining.  

Since we're injecting a large amount of new code into the project, there are bound to be many bugs at first.  The new code has some known issues and the old code did as well.  I will need useful bug reports from users.  Please check tinderbox.midnightbsd.org before reporting compile errors with the code as I may already know about it.  

The next question is what do we do now.  This injection of code is not unlike the bsd 4.4 lite / UCB lawsuit situation from the early 90s for the other BSDs.  The reasons are different, but it's a bunch of code to get cleaned up and put in.  FreeBSD and NetBSD diverged greatly after that time period because they had very distinct goals as well as coders adding, tweaking and fixing here and there.  I started this project to work on a desktop BSD.  In the mean time, many of the other BSDs have gotten better on the desktop.  OpenBSD, NetBSD, and DragonFly have decent wifi support.  FreeBSD has been improving it for 7.x and 8.x.  Many of them have loosely associated live cd/dvd projects now.  

In order to make a list of areas we need to address, let's first consider the environment.  I use MidnightBSD on some desktops, a laptop, many mports nodes, and several servers.   For my own purposes, I often try to maintain some of the server elements of the system.  It does aid me in finding bugs, but it also distracts me from the desktop environment.  Over the holidays, my mother told me she got a new monitor.  I asked her if she wanted to use the old monitor on her 700mhz celeron which has MidnightBSD on it.  I had set it up as a backup computer for her some time ago.  She was concerned about using something besides windows.  I wanted to argue in favor of using MidnightBSD for her, but I realized that we don't have an intuitive system just yet for someone like her.  When I first envisioned MidnightBSD, it was to make it for everyone.  That task has not been completed (nor started in a satisfactory manor).  So the obvious step is to pop a CD into a system, and try to set it up for a typical home computer user.  There are many challenges to that task.  These areas need to be met with our project first.

As we've discussed before, the installer is a priority and next on my list after the merge is complete.  Combined with the installer, we'll have a beginnings of a decent live dvd.  This solves the first two complaints I often hear if we do it right.

So what's next?  What packages do we install?  How do we partition the system?  What features should be enabled by default?  What should be a kernel module and what should be part of the kernel?  What do we do about binary blobs?

These questions should all be answered by now, and many of them were.  Things are different now.  We need to evaluate some of these decisions and write up a clear policy on our website. 

Two different sites are now offering MidnightBSD installation sets.  The links have been added to the download page of our website.  A portion of the sales for OnDisk.com will be given to the project.  

I also want to update everyone on MidnightBSD progress.  A few months ago, we had a discussion with developers and a few users about the future of the project and issues with MidnightBSD.  The most common complaint about MidnightBSD is hardware support and the installer.  There have also been feature requests for journaling file systems, ZFS, jemalloc, and .98 openssl.  

When MidnightBSD was forked, it was done off 6.1 beta.  We spent time fixing bugs that were corrected in the release version and adding hardware support.  However, we haven't made significant progress with the base system.  This defect has held us back.  In part, I feel the steep learning curve to work with the src tree has discouraged new developers.  Our mports system is working out well and we've succeeded in dramatically improving the experience and packages generated from ports.  A good share of the credit should go to ctriv@ for digging in to the mess and designing something new that continues to get better.  However, a ports system is not enough to keep us viable.  

A decision was made to bring in select parts of FreeBSD 7 that we feel are useful or parts that we've not had time to maintain.  It was a difficult decision because it meant we haven't  yet separated ourselves entirely from their project.  Other projects such as DragonFly occasionally bring in code from FreeBSD such as the ata code (nata in DF) or their latest approach at writing a wrapper for the network stack in FreeBSD Current.  Haiku uses FreeBSD drivers, jemalloc and other parts and has successfully wrote a wrapper for many parts of the system they use.  Still, this was something much larger.  We haven't blindly imported the entire FreeBSD 7 environment as we have accomplished improving parts of the system, but it was still a large piece of code.  

The work on this started in July and began to appear in CVS sometime later.  The last two months have been a constant stream of merges, fixes, imports and more testing.  At the same time, ctriv@ has been refactoring parts of the mports system to make it more modular and consistent.  

The result will be a new system with many improvements and a few regressions.  The changes will be present in 0.3-RELEASE.  We do not have a very specific time table for this release aside from a target date of October-November of next year.  

We have an increasing user base, but need more developers of all skill levels. 

mports

As for mports, you can take advantage of the new changes by updating your system to the latest stable version RELENG_0_2 and updating your mports tree.  A file was added to /usr/share/mk (src/share/mk) required by the mports system.  Without this file, you will have make errors on some ports.  This change has occurred in the last two weeks.

GUI

Many users have expressed an interest in our choice of desktop environments.  GNUstep and many ports were updated recently.  They work on i386 and often sparc64.  The amd64 version of gnustep is not working and we're investigating that issue.  Etoile is out of date and must be updated for the new version of GNUstep.  

We also plan to ship KDE and Gnome in the next release.  Until Etoile matures to a point it's usable, it makes sense to let users pick alternatives.  Many linux distros have chosen to do a similar approach.  Fedora has flavors.  Other projects, like Ubuntu, have variations for use scenarios like education, or for desktop choice like Kunbuntu (KDE).  

As the GNUstep project is aggressively moving forward, we don't feel it's a good time to count on it for our installer just yet.  The amd64 issue is an obvious road block.  Our plan is to write the installer in GTK for now and evaluate GNUstep later.  Why GTK? It's not huge like QT4, and I have experience with it.  We plan to include GNUstep on live DVDs in the future so users can experience it.  I'll follow up in a later post.

 There's been a flurry of activity lately with mports.  We've added a number of new ports including qt4.  

Results from our last magus run:

• untested: 187 
• pass: 1656 
• fail: 29 
• warn: 408 
• internal: 1 
• skip: 23
•  
•  

Magus is running again.  It's the first run since 0.2.1-RELEASE.  We've already found 20 ports that need some TLC and the run is still going.   

Various security issues have been reported in the last two months.  It is important that users update their OS if they use IPV6 or ftpd.  Updates have been committed to our src repository.  At this time, we don't have a binary update system, so there is no method to update in that fashion.  

All patches have been applied to RELENG_0_2 which is recommended for 0.2-RELEASE or 0.2.1-RELEASE users.  0.1 users can get some of the updates on RELENG_0_1 (cvs branch), but it's recommended to update to 0.2.x as soon as possible. 

We now have xfce 4.4.2 in mports.  You may install xfce by using the new meta port mports/x11-wm/xfce4  

 We do not have a package at this time.  

xfce

Today, I added an xfce port to mports.  It is only version 3, but the missing gdk-pixbuf dependancy was finally fixed.  In case anyone is curious, I had to use our port of libtool to get it to build properly with shared libraries.  xfce 4 is very large so I don't know about supporting it right away.  This was done after a user request for xfce. I don't care for it.

flock

linux-flock was updated to 1.2.5 today.  This includes several security fixes and feature enhancements. For instance, Digg was added in 1.2.  It also includes a working copy of flash player so you can enjoy youtube without much hassle.

twitter

I added a new port, deskutils/twitter which is a shell script that allows you to update your twitter account from the command line.  usage is like: 

# twitter username password "my entry here"

midnightbsd-backgrounds

Another port of interest is the x11-themes/midnightbsd-backgrounds that installs desktop wallpaper with the MidnightBSD logo.  There are several variations created by myself and "smultron".  We'll be adding more to that port at a later time.  It currently installs in a way that KDE can pick up.  We'd like to get gnome aware at a later time.  

bind 9.5.0-p2

I've updated our bind 9.5 port to p2.

x11/themes/midnightbsd-backgrounds was added today. It includes several MidnightBSD inspired desktop backgrounds in PNG and JPEG format.

 I've changed the frequency that ds9's cvs mirror updates.  It no longer rsync's every 30 minutes, but rather every 6 hours.  The previous cycle was giving people trouble over slow links.

I've created a new CVS activity blog.  It will display the commits to MidnightBSD in real time.   Since CIA.vc is having reliability problems, this is a way for developers to check on recent activity while avoiding the large emails on the midnightbsd-cvs@midnightbsd.org list.  However, the formatting sucks and I don't have messages included yet.  I'll try to tackle that after I get some sleep.

The upside to this approach is that we've also got RSS and ATOM feeds of the cvs history automatically.