The Design and Implementation of laffer1

Downstreams don’t have it easy. They need to be informed about new upstream releases but there’s no uniform way to do that. Debian has uscan and the DEHS (Debian External Health Status), rsc is monitoring me using Fedora infrastructure, etc. but other projects, such as the AOSP (Android Open Source Project) seemingly don’t have such a resource set up. Recently, the desire for an mksh-announce@ mailing list was stated.

Mailing list spam is a thing (sorry about that), and I currently do not have anything set up that would allow me to make it possible for only me to post to a list. But we do have RSS feeds; e.g. for my APT repo I use a script creating one easily from a plaintext file, and the MirBSD wlog infrastructure has a (complex) setup, creating RSS and HTML, paginated and permalinks, off a data file.

Enter the MirBSD “announce” (valid RSS 2.0) feed, which will provide information relevant for downstreams (especially subproject releases), and the occasional MirBSD snapshot, ISO or release. I chose the former (easy script from plaintext) for this and will occasionally prune older entries.

http://www.mirbsd.org/announce.rss

In the hope of being able to help, I wish my downstreams a blessed time, with most calendaries just having entered a new year.

I’ve been using “a Unicode (and ASCII) field separator” for my SSV flavour of CSV. I thought I should be using the FS control character (considering “FS”, according to much documentation, is a field separator).

Turns out most Unicode control characters have shitty official names and/or acronyms/abbreviations… such as…

• PLD: partial line forward (not: partial line down)
• SPA: start of guarded area (not: start of protected area)
• VTS: line tabulation set (not: vertical tabulation set)
• DC1: device control one (not: XON)
• RI: reverse line feed (not: reverse index)
• NP: form feed (probably for “new page”)
• NL: line feed (not newline, but we weren’t expecting that either, as an ASCII newline is CR+LF plus Unicode C1 has NEL (next line)…)
• Adding insult to injury, U+0080, U+0081, U+0084 and U+0099 do not even have a name (but Unicode “name aliases” which include an acronym (which (of course) WTF knows about) and at least a longer name).

… and so forth. There’s separators, too!

• FS: [U+001C] [?] INFORMATION SEPARATOR FOUR [file separator]
• GS: [U+001D] [?] INFORMATION SEPARATOR THREE [group separator]
• RS: [U+001E] [?] INFORMATION SEPARATOR TWO [record separator]
• US: [U+001F] [?] INFORMATION SEPARATOR ONE [unit separator]

And guess what… ASCII and Unicode FS is file separator (US is field separator). Oops. Sorry.

So… I guess when I use SSV next I’ll update (change in an incompatible way) the spec. Again, sorry about that.

It’s only in another 48 minutes but enjoy the Solstice! Blessed be!

I’ve created an SF2 format SoundFont (compressing to SF3 is not worth it really) for use on RAM-constrained devices, such as the Raspberry Pi. It’s comprised of:

• the piano from Fluid (R3) Mono 2.315 (which is very slim, one twenty-fifth the size of a wonderful new piano in MS_General)
• monoified (left channel, panned to centre) Choir Aahs (to save another 2½ MB) from MuseScore_General 0.2 (expressive and regular, for SND support)
• the harpsichord from MuseScore_General 0.2

The result, a whopping 7.3 MiB, is enough for accompanied voice, therefore called “SATBkc”?—?SATB, Klavier (Pianoforte), Cembalo (Harpsichord). It’s published under the same MIT licence as its two constituent soundfonts.

Download the soundfont (not going to package it, it has limited use) as well as a test score (in v3 format, it tests Single Note Dynamics too) if desired. Discussion on the MuseScore forum for soundfonts, please. Also remember that the waveforms generated from the soundfonts are, most likely, derivative works, requiring reproduction of legal notices.

Combined with TimGM6mb, this gives you full GM but better sounds for some instruments in just 13 MiB HDD and RAM (both are uncompressed SF2). “Choir Aahs” still could be better, but they are not “Choir Aarghs” any more at least ? TimGM6mb is GPLv2-only though so not universally usable (YMMV).

OK, toned down on the rant, I already did enough in the commit messages…

My webpages are valid XHTML/1.1. But what does that mean?

I write them conforming to XHTML/1.1, with spaces before the “/>” sequence. This allows me use of tools such as xmlstarlet to operate on them as XML files, and even validation against the DTD, offline. That “extra” space allows HTML browsers to process them as HTML. I’m now, as per some part of the spec, supposed to serve them over HTTP as application/xhtml+xml content type?—?two questions: why does the XHTML spec say anything about HTTP, and, why doesn’t another spec agree with it?

Turns out, much later, it has a reason?—?the XHTML/1.1 spec is mostly just a diff against XHTML/1.0 Strict, which is just a diff against HTML 4.01 Strict. The HTML5 spec (both concurrents, W3C and WHATWG) is however standalone and merges the XHTML parts. It, now, in contrast to the three older specs I mentioned above, has a side note, in a tag-specific chapter (with nothing mentioned in the XML part), explaining a parsing difference (basically, in XML mode, it doesn’t skip a leading newline immediately after an opening pre tag). Fucktards.

I’ll just serve my XHTML/1.1 files only as text/html now, even if I get an Accepts for XHTML+XML from the request. (The HTML5 spec, at least one, now forbids me to use XML namespaces, both for things like embedded SVG (I am supposed to just use an <svg> tag) and custom ones, e.g. to embed DC in SVG… but we all know just how binding this is for browsers, and that browsers will handle all kinds of things under the sun, and then some, so I’m ignoring this, ’sides, I even don’t write XHTML5 at all…)

Anyway the too-large space around section and subsection headers in our HTML manpages is now “fixed”, for some very low value thereof (but with HTML and CSS the expectation is extremely low anyway…).

I’ve concocted a workaround for the issue that MuseScore cannot reproduce the soundfont copyright in exported files yet, by placing it and (also necessary, not present in every export format) score metadata in the “associated documentation files”, which fulfills the licence. For now, Free Music repository directory listings show a hint requesting the user acknowledge them; I also plan a fancy thingy in ECMAscript to offer downloads and play the sheet music in the browser, if modern enough (lynx, of course, I will handle properly, you know me).

Enjoy!

With a mixed bag of changes, I’ve released mksh(1) R59 yesterday. Some of those changes are breaking to the shell language:

• When printf(1) was compiled as builtin, and a matching external utility (i.e. $(which printf)) didn’t exist, and builtin printf was not used to specifically invoke the built-in utility, it could not be found. This is critical but only for a very small area: mostly when mksh (or more specifically lksh), with printf as builtin, is used as /bin/sh and the udev SYSVinit script uses printf while insisting on setting PATH to just /bin while printf(1) sits in /usr/bin. If this affects you, you want this fix.
• OS/2 only: the test(1) builtin already sometimes automatically added the suffixes .ksh, .exe, .sh, .cmd, .com, .bat to a file argument if one without these suffic?s was not found. This was extended to cover more cases to improve the user experience. (Thanks to KO Myung-Hun for this!)
• The output from some builtins is now formatted differently. This mostly affects how alias names, and in some cases their definitions, are printed (by alias, command, whence, etc.) and the output from the bind builtin was also made safe for re-entry into the shell. These are desirable from a security PoV but change formats.
• In the manpage, some documentation was wrong: the example command given for how tab completion escapes, and the right-hand side of string comparisons only globs in [[, not in [ and test.
• The shell argv[0] (after removing a leading dash to indicate a login shell and using the basename(1) of the rest) is now checked whether it begins with an ‘r’, and if yes, restricted mode is enabled.
• In [[ x = $y ]] we now parse the right operand $y as full extglob.
• Since we already have breaking changes, the former global builtin introduced in R40b and deprecated, in favour of typeset -g in R55, was removed.
• ^[Q (Esc+Q) was added as new editing command, quoting (for use as shell parameter, i.e. with \'…\' or $\'…\' like typeset does) the area between the mark and the cursor.
• The manual page, besides featuring properly spaced “em” dashes, was completely overhauled in documenting reserved words and built-in utilities and now also documents built-in aliases and even those aliases and functions dot.mkshrc offers, more or less verbosely, and indicating, with every entry, which is which, including specialness and keeping assignments, deferring (with flags, like cat, or always, i.e. rename and the optional printf), being a declaration utility (where ‘b’ in export a=b is not IFS-splitted) or declaration utility forwarder (like command export a=b also skips the field splitting) and requirements (such as job control, or the presence of select(2) etc.)
• The testsuite works again with OS/2 and pre-glibc_2.30-5 GNU/Hurd.

Now some of these changes are desirable and indicate you ought to upgrade. If you can’t (due to the breaking changes), talk with me, and I may release an R58b with only some of the changes. But please do consider whether R59 might work just as well. TIA!

Continuing with the idea of “let’s get releases out”, hopefully with no regressions introduced, and all updated to the latest UCS, find infos for jupp & mksh updated on their respective pages.

There are still some known unfixed issues, but time will see to them. It’s best to occasionally get the more stable codebas?s out, so users can test (and break ?) them.

rs(1) is a classical BSD tool I noticed was missing under Debian. So I made the MirBSD one portable, some long time ago, and, because grml’s mikap wanted it as well, uploaded it to Debian. Turns out this invites actual users to report bugs ?

So here we are, rebased to include latest OpenBSD changes, bugfixed, made portable, and even with a convenience strtonum implementation:

• SHA256 (rs-20200313.tar.gz) = 919215dc9fe85a27a30bf63d56406cfb503f9fc9820323c4bd3bfe75a6a3bc3f
• RMD160 (rs-20200313.tar.gz) = a8dfa5bb7ef63c66e011ec81bf20e089fdd827f5
• TIGER (rs-20200313.tar.gz) = 42135e4d75e7865b817f1b4027d383416d326c305e6553ce
• 1362219422 12571 /MirOS/dist/mir/rs/rs-20200313.tar.gz
• MD5 (rs-20200313.tar.gz) = cc6a310b7f3bae98ea6296fbee0f85b4

If you really need build instructions, look at the Debian package.

Development on other fronts is also continuing. See you in IRC only, I guess… (what with the current situations, the last newspost also had conference presence). Due to the sheer amount of changes, a release of mksh is somewhat imminent, if only to get my users to find regressions caused by me attempting bugfixing ?

Another release of one of MirBSD’s subprojects. Now, both the 8x16 VGA (cp437-encoded) and the full Basic Multilingual Plane 8x16/16x16 proper font are also available, on all possible platforms, as “doubled”, that is, 16x32 and 18x36/36x36, version suited for e.g. hiDPI displays. (This was mostly done with simple pixel doubling for each axis, with only few glyphs fixed up afterwards to achieve a slighly improved, but still FixedMisc bitmap font, look. Thanks to apotheon for the suggestion (even if it ended up being a tad too large in his eyes) and to cnuke@ for testing and to both plus Sarah for feedback.)

The APT repository was, of course, updated with xfonts-base/consolefonts-base and console-setup to match. It also, in mirabilos-support, ships an updated version of the Linux text/framebuffer console keymap.

Download and check:

• SHA256 (FixedMisc-20200214.tgz) = 92cd16d302741be9314014960f2c57866b7e31f720b47df8efebfec7c6c35319
• RMD160 (FixedMisc-20200214.tgz) = 9bbf24131664d201411294b633e265fc3d940fb1
• TIGER (FixedMisc-20200214.tgz) = 92099b2a989d7a66b22aacf93836345581f8ba27aab0cab5
• 758244556 5955999 /MirOS/dist/mir/Foundry/FixedMisc-20200214.tgz
• MD5 (FixedMisc-20200214.tgz) = 546f492a4b0459cbf2689306560070a2

Mind this is slightly larger (6/46 MB download/decompressed) than the previous releases (1¼/10½ MB) because it now ships the fonts not only in regular and doubled versions but also the HW-only versions expanded and the full font (normal and doubled) for GRUB and the cp437 font in PSF and PSFU format (version 1 for 8x16, version 2 for 16x32). Enjoy!

I also wanted to give you a new release of the another MirBSD subproject, jupp, but I haven’t managed to finish my work on it in time. After that will, most likely, lead me to more mksh bugfixing, followed by the long-expected next regular release (it’s already cooking in Debian unstable).

And then, I hope I’ll manage to get a bit of time to get back to the BSD base and manage a rollup rolling release snapshot for those updating from binary, not from source themselves. (Rumours about being discontinued are just that, rumours; they originate (hah!) from Wikipedia, whose page about MirBSD has, incidentally, never been fully right.)

See you in IRC or around on conferences!

I’ve managed to miss FOSDEM this year, unfortunately, because I’ve got a beginning sinusitis (this time before the conference) staying home cautiously. But fear not, I’m working on porting the MirKeyboardLayout™ to Windows® 95, and, during that, I noticed that I need another glyph in the documentation comments. Cue FixedMisc.

As usual, download FixedMisc and check the integrity before installing:

• SHA256 (FixedMisc-20200202.tgz) = 91396414e169b37bc906746ae34188ad360be271865ac44271d9b7d9746c97f1
• RMD160 (FixedMisc-20200202.tgz) = 8f672de47df8bc67df52f5f48ac49105953d19e9
• TIGER (FixedMisc-20200202.tgz) = d875a4835c053e21914ead312a6ec9afc40b347ee1d04fa5
• 3480714773 1275833 /MirOS/dist/mir/Foundry/FixedMisc-20200202.tgz
• MD5 (FixedMisc-20200202.tgz) = eb494f7f71b2c610346d58e3ac6c46ce

I’ll update my APT repository later. The separate Powerline font has been merged considering we don’t even ship glyphs for “Cirth” in CSUR and it’s being considered for inclusion into the SMP anyway.

Update: The APT repository is updated, and the MirKeyboardLayout for Windows® 95/98/9x (self-extracting LHarc archive) is also done, as far as I can make it anyway: the 102nd key (“<>|”) operates as “…€„™”, as in the NT/2k/XP/… layout, but it produces wrong results (at least on 950 B) if Shift and/or AltGr are pressed, and I couldn’t test AltGr-Tab and AltGr-Shift-Tab ‘“”’ because my window manager caught them before they could be passed into the VM… and since it uses cp1252, I used the florin ‘ƒ’ for AltGr-- instead of U+2010, randomly. The full source is also available. Test results, fixes and improvements welcome. Next: xkb

Update: I’ll be doing a script for customisation of the xmodmap and Linux layout (unswap unshifted Esc and `, move Mode_switch to Alt_R/AltGr keeping Alt on Alt_L and Meta on Win_L, and a tristate one: CapsLock as …€„™ and the <>| (102??) key as Compose, vs. the 102?? key as …€„™ and CapsLock being either Compose or Ctrl) soon. Stay tuned!

Today I’ve released another new CVS snapshot of the FixedMisc [MirOS] font; as usual, the tarball contains the font in BDF form, with no conflict with the system Fixed [Misc] font; sources for use (compilation, editing) with bdfctool(1) are in CVS.

New: a Powerline variant of the halfwidth font, and massively more alternative UCS mapping for the cp437 font.

• SHA256 (FixedMisc-20190911.tgz) = 1aa35a3128b3e5ca452467fca8150ad394054f60f847eca7296480bd23039dd7
• RMD160 (FixedMisc-20190911.tgz) = fc2a61166ea4c955d5c34e03f5da0c00df132a00
• TIGER (FixedMisc-20190911.tgz) = f3b087c819c8fdc2c319feca5d11f1ad25f89d7ce17e2907
• 830148610 1378344 /MirOS/dist/mir/Foundry/FixedMisc-20190911.tgz
• MD5 (FixedMisc-20190911.tgz) = 87ef903a45e5a6e1c9dfa86b172b24d3

My “WTF” APT repository contains the updated consolefonts-base and xfonts-base packages, as usual.

Today, I realised that, to use a laptop hard disc outside of a laptop, no matter whether via converters or in a regular (n?n-laptop PC), most likely…

	hdparm --security-unlock password /dev/hda 

… is needed. (No, I don’t currently know how to do this in MirBSD.)

Update:

	 SG_IO: bad/missing sense data, sb[]:  70 00	05 00 00 00 00 0a 04 51 40 01 21 04 00 00 00	00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 

… maybe not ?
Send help.

MirBSD has just recently become 17 years old, and I wrote (in German, sorry ?) a reminiscing piece about that and thanking everyone involved.

Today my calendar(1) reminded me of the first steps towards “MirLinux”, a.k.a. “MirOS Linux”, 16 years ago and given this pops up regularily, especially due to Wikipedia spreading it, I feel I have to clarify: cnuke (the original Jupp) really likes the BSD userspace but wants to play Quake ? with accelerated 3D, so the idea was to maybe build everything for Linux, add a glibc and other dependent libraries (and we’d use different paths, so linking is unaffected and we’d have nicer linking semantics than those GNU people), and maybe things would just work.

It was a woozy idea right from the start, and there might have been beer involved, and nobody ever got around to actually doing so, and it clearly was never a/the project goal. Yes, we probably could have done it, back then, up to 90% satisfaction, and with some more binaries thrown in from GNU/Linux (e.g. for the packet filter, as?—?sadly…???pf(4) for Linux has never materialised) it could have become usable, and there was ecce!GNU/Linux precedent, but BSD’s the focus. Perhaps if a certain few people had been less Verpeiler… oh well?—?no big loss.

I did turn out fixing stuff in GNU/Linux and porting stuff over in the end, but we never merged them, which perhaps turned out, looking back, to be a good thing.

Tomorrow 16 years ago, plip(4) support was added… I need to dig out the cable and run some interoperability tests some time to see if it’s still working, with both Crynwr and Linux on the remote end, and FreeBSD (if they still have it).

(read more…)

So… the Debian package of gitlab is too buggy to be used (was built against ruby-asciidoc version X.Y while sid carries X.(Y+1) now, which causes it to bug around, of course, as proper for an immature language like that. So, someone decided to switch to the GitLab CE *.deb format packages (not Debian packages?—?not Free; just Open Core but Debian itself uses those for its “Salsa” instance as well (which is, incidentally, why I refuse use of that whenever possible) and, for that, removed the Debian packages. The gitlab binary package helpfully offered to not delete the repositories, but gitlab-common’s postrm not only removed the user account (a big no-no!) but used the option to delete its home directory… which is where the git repositories and project icons and the likes are stored under. (Note that undeleting from ext3/4 is hard, unlike ext2, and if fsck and/or a journal replay is run, chances get worse… the ext4undelete tool “helpfully” requires an fsck run… ’nuff said… if you ever accidentally delete something, immediately unplug power and destroy VMs hard, then snapshot the filesystem so multiple rescue approaches aren’t made impossible.)

Anyway, it’s apparently running GitLab CE now, which means that all the remotes have changed. I used this…

	sudo find / -xdev -name config | grep \'/\.git/config$\' >~/xgc	sudo fgrep -li gitlab@edugit.org $(<~/xgc) >~/xgc2	<~/xgc2 sudo xargs perl -pi -e \'s/gitlab\@edugit.org/git\@edugit.org/gi\' 

… for fixing up mine (inspect the temporary files and check find(1) and mount(8) for -xdev to get the right files found).

Also, ~/.gitconfig insteadOf / pushInsteadOf need fixing. Let me plug an undercover avertisement for my .gitconfig here, which contains examples for insteadOf as well as commands to download GitLab merge and GitHub pull requests.

After having fixed those up, go to the web UI and click on “Create empty repository”, then push all remote branches recorded in your hopefully up-to-date clone and (all) tags to the instance:

	remote=origin	git branch -r | sed -n "/^  $remote\\//s///p" | \	    while read branchname rest; do		test x"$branchname" = x"HEAD" && continue		echo "pushing $remote/$branchname"		git push "$remote" "$remote/$branchname:refs/heads/$branchname"	done	git push "$remote" --tags 

Adjust the remote variable if necessary. Run this in all clones you have access to; not using force pushes makes only those pushes which actually add commits succeed. All repositories hosted on the edugit instance are affected and need(ed) restoring, which, thankfully, appears to make everything else, like stored merge requests, work again (although the project and group logos are gone, which need re-uploading). That being said, unapplied merge requests are stored in special refs which are not normally cloned… so they’re gone now.

There’s a new MirCPIO (paxmirabilis; tar, ar) release. Difference is, some operating systems don’t yet support passing nil as second argument to realpath(3) which incidentally included (note: past tense) a certain BSD whose installer segfaulted in tar(1)…

Debian GNU/Hurd was, btw, not affected.

In other news, it’s way too hot and other IRL things take up tuits.

And in completely (I’m sure) unrelated news, my waypoint statistics are not getting updated for now, and acronym submissions pile up in the queue. (The broken iOS Äpp link has been forwarded to the author. Techniker ist informiert. YMMV)