MidnightBSD Developer Journal

The 1.0 release is finally available. Still buidling packages for i386 and plan to do an amd64 package build later in the week.  

The single largest issue with the release process has been the web server performance.  The CPU is overloaded and has been at solid 100% for several days.  The server has a core i7 7700 in it.  I'm trying to figure out what to buy as an upgrade so that we don't continue to have this issue going forward. As it's actually blocked in multiple processes, a 6 or 8 core chip might be an improvement for the workload.

The 1.0 ISOs are on the FTP server. The release notes are nearly written. Waitnig on some updated packages and we'll be able to call this thing.

One of our servers was setup in MidnightBSD 0.7 and had an older ZFS configuration. Over time, the drivers were replaced with 4k advanced format disks. Since ZFS won't let you change the ashift setting on an existing pool, the drivers were not running at optimal performance.  MidnightBSD 1.0 current warns about this sutation. 

How to fix it? 

In this case, there were 2 drives in a mirror and a cache disk as well as another backup pool.  First, we performed a backup to the other pool just to be safe using zfs send -R mypool@snapshot | zfs receive -F backup/mypool, then we removed one of the disks from the pool using zpool detach ada4p1.  We then blew away the partition and created a new gpt partion that was 4k aligned.  gpart add -t mnbsd-zfs -a 4k ada4 

Next, we created a new pool called tank  with the one drive.  We then ran zfs send and receive to copy data from mypool to tank. 

Once this data is copied, we went into single user mode (shutdown now) and exported both pools. We then imported the new pool tank.  zfs export tank; zfs export mypool; zfs import tank.  This allowed our mount points to kick in.  

Next, we repreated the process of repartitioning the second disk, ada3.  We then add it by doing zfs attach tank ada4p1 ada3p1 to include the second disk in the mirror.  Finally we did zpool add cache tank ada1 to re-add our ssd cache drive. 

The drive will need to rebuild which can take some time. 

This approach prevents any data loss.

We're currently working on a large merge of FreeBSD 10-stable (from late may) into MidnightBSD.  A large portion of this work is done, but there are still a few loose ends. 

• Symbol adjustments might be made to libc

• Several utilities are not connected to the build yet. 

• There are problems building some of the release targets such as the uefi memstick

• A few third party apps were etiher newer or older and not updated.  For example, our svn is still 1.8.x. 

• There were bugs in the boot loader code due to a bad merge. Most if not all of this was fixed. 

• Some architectures may still be using freebsd partition types, some utilities might not be switched over yet.  

We also removed the sensors framework during this migration.  There have been some locking problems in 0.8.x and it requires a rework.  

Also, we now have bhyve.  The good news is that it's possible to run FreeBSD on MidnightBSD in bhyve. The trick is to use a freebsd userboot file.  You can compile one from FreeBSD 10.x on midnightbsd or use the new port.  Another option that may work is using the grub port.

Initial testing of MidnightBSD on a Ryzen 7 1700 CPU with an Asus PRIME x370-pro motherboard indicate that there are some issues that prevent the system from booting.

There are two problematic areas:

1. The system timer detection code is not finding a timer that will work .
2. The Intel NIC included on this motherboard is not detecting properly and causing a panic.

You can check out MidnightBSD with git svn.

For developers
git svn clone svn+ssh://laffer1@stargazer.midnightbsd.org/home/svn/repos/src/ -T trunk -b stable -t releases
(replace with your username of course)

For folks looking to fetch the code, you can just use the http protocol instead. Use the URL:
http://svn.midnightbsd.org/svn/src/

MidnightBSD 0.9 current now includes the doas(1) utility, an alternative to sudo. Learn more about this utility http://www.tedunangst.com/flak/post/doas-mastery

I am experimenting with MidnightBSD on AWS. It's possible to take the memstick image of 0.8.5 and modify it to enable the network interface, sshd and a test user to ssh in and then create a AWS AMI. So far, I've gotten it to boot to a login prompt.
The EC2 steps are as follows:
ec2-import-volume ~/memstick -f RAW --region us-east-1 -z us-east-1a -s 10 -d "MidnightBSD 0.8.5" -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" -o "${AWS_ACCESS_KEY}" -w "${AWS_SECRET_KEY}" -b "mnbsd-vm"
find out if it's done converting
ec2-describe-conversion-tasks -O ${AWS_ACCESS_KEY} -W ${AWS_SECRET_KEY} --region us-east-1
create a snapshot (get volume id for last arg from previous command)
ec2-create-snapshot -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" --region us-east-1 -d "MidnightBSD 0.8.5" vol-0d149b0dfc22962b1
check on snap
ec2-describe-snapshots -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" --region us-east-1
register ami (snap argument shown from last command)
ec2-register -n "MidnightBSD 0.8.5" -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" --region us-east-1 -a x86_64 -d "MidnightBSD 0.8.5 AMD64 Test" --root-device-name /dev/sda1 --virtualization-type hvm -s snap-0a7bb785269bfb08e

MidnightBSD 0.8.5 RELEASE ISOs now available on our primary ftp server for 32bit and 64bit processors. (i386 and amd64) It includes all the fixes for mport package manager, and security updates to openssh, openssl, sqlite, etc.

I'm happy to announce the availability of #MidnightBSD 0.8 release! It is now available on our primary FTP server and uploading to our mirrors. You can download it from our website.

You may also read about the changes in the release notes. http://www.midnightbsd.org/notes/

MidnightBSD 0.8 amd64 packages have been updated.

Note that xorg is included!

There are 2,884 packages available (more than i386 now) including part of qt5 and lumina desktop environment.

Also, if you have issues installing packages, make sure you have the latest libmport and mport tools from SVN (0.8 stable).

The 0.8 release has been delayed due to bugs found with the package manager. As this is a critical part of the system, the decision was made to delay it.

We currently have developers debugging the problem.

Fix four security issues with MidnightBSD.

The implementation of TIOCGSERIAL ioctl(2) does not clear the output
struct before sending to userland in the linux emulation layer.

The compat 43 stat(2) system call exposes kernel stack to userland.

libarchive - CVE-2015-2304 and CVE-2013-0211 fix issues with
cpio directory traversal and an integer signedness error in the archive
write zip data routine.

We’ve had 1370 downloads from our CDN in the last month for 0.7 release.?

Website traffic from April 20 to May 20:

1840 sessions
1504 users
3558 page views
31% traffic from Russia
17.66% from US
5% from Germany

Browser stats:
48% Chrome
27% Firefox
6% Safari
5% Opera

Fix two security vulnerabilities.

We've identified a bug related to package management in MidnightBSD .0.4-RELEASE.

The hash check that is part of libmport is improperly working.  This means you can't install packages with the mport command.

To work around this issue, please checkout the 0.4-RELEASE source from CVS using the directions on the site and then rebuild and install libmport.

cd /usr/src/lib/libmport

make

make install

In addition to our mirrors, we have some other options for downloading 0.4 release popping up. 

as a stability release and meant to work on desktop related functionality. 

There have been two snapshots release in the last few months, i386 and amd64.  The former appears to be bug free and was created this month.  You can find it in the snapshots directory under i386 and 0.4-130610-SNAP.  The amd64 snap has a few bugs, but can be installed.

Both of these snapshots are for 0.4-CURRENT.  Recently, we created a branch for 0.4 and there are a few large big fixes and one security update since the snapshots were released. It is strongly recommended to rebuild from the 0.4 branch after installing a snapshot.

There are currently no packages for i386 available.  The index does not work with the newer mport tool in RELENG_0_4.  As the ports tree is in the middle of a major update, it's not stable enough to release packages yet.  I'm working on this problem.

Most notebly QT4 is broken right now. X.org ports, dbus, gcc and many other ports have been updated in the last month. There have been many architecture changes to the mports/Mk extensions as well. We now support some FreeBSD ports USES statements (pathfix, charset, ncurses, pkgconfig) which makes migrating ports from FreeBSD easier.

Magus has been running lately and churning out test builds of packages. The results for the last 3 runs were quite bad. 

We have a new snapshot uploading to the FTP server. It's the first snap in a year.  This snapshot is a little buggy, but does allow you to install MidnightBSD.

Please note there are many changes from 0.3-RELEASE:

1. Uses new midnightbsd partition types: mnbsd-ufs, mnbsd-boot, etc on GPT

2. ZFS is much newer than 0.3.  If you upgrade your pools, you can't use them with 0.3 anymore.

3. KMS with Intel Ivy Bridge graphics

4. Installer is completely different

5. hastd

6. updated mksh, BIND, tcsh, file, diff, binutils, mDNSResponder, libffi, openpam, openresolv, tnftp, tzcode, tzdata, wpa, xz, compiler_rt, sqlite3, ncurses, netcat, pf, traceroute, perl, openssh, openssl, less

7. updated from FreeBSD: make ipfw & ash, forth menus for the loader, bsdinstall, bsd sort, new USB stack, new cam based ATA, geom

8. llvm + clang

mport is the default package manager!

Major hardware support updates.. several wifi adapters, etc.