{"version":"https://jsonfeed.org/version/1","title":"BSD Now","home_page_url":"https://www.bsdnow.tv","feed_url":"https://www.bsdnow.tv/json","description":"Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros.\r\n\r\nThe show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day. ","_fireside":{"subtitle":"A weekly podcast and the place to B...SD","pubdate":"2024-11-14T08:00:00.000-05:00","explicit":false,"copyright":"CC Attribution + Noncommercial + ShareAlike (BY-NC-SA) by JT Pennington","owner":"JT Pennington","image":"https://media24.fireside.fm/file/fireside-images-2024/podcasts/images/c/c91b88f1-e824-4815-bcb8-5227818d6010/cover.jpg?v=4"},"items":[{"id":"137023c9-3a8f-495e-8b66-8db48e5b1ee7","title":"585: Infrastructure Administration Workstation","url":"https://www.bsdnow.tv/585","content_text":"From Proxmox to FreeBSD - Story of a Migration, FreeBSD At 30: The History And Future Of The Most Popular BSD-Based OS, Using a dedicated administration workstation for my infrastructure, LibreSSL 4.0.0 Released, Plasma6 and FreeBSD 14, Replace gnu diff, diff3, and sdiff with BSD versions, and more\n\nNOTES\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nFrom Proxmox to FreeBSD - Story of a Migration\n\n\n\nFreeBSD At 30: The History And Future Of The Most Popular BSD-Based OS\n\n\n\nNews Roundup\n\nUsing a dedicated administration workstation for my infrastructure\n\n\n\nLibreSSL 4.0.0 Released\n\n\n\nPlasma6 and FreeBSD 14\n\n\n\ngit: world - Replace gnu diff, diff3, and sdiff with BSD versions\n\n\n\n\n\nBeastie Bits\n\n- How to Upgrade FreeBSD KDE 5 to KDE 6\n\n\n***\n\n\nTarsnap\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\nJoin us and other BSD Fans in our BSD Now Telegram channel\n\n\n","content_html":"
From Proxmox to FreeBSD - Story of a Migration, FreeBSD At 30: The History And Future Of The Most Popular BSD-Based OS, Using a dedicated administration workstation for my infrastructure, LibreSSL 4.0.0 Released, Plasma6 and FreeBSD 14, Replace gnu diff, diff3, and sdiff with BSD versions, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFrom Proxmox to FreeBSD - Story of a Migration
\n\nFreeBSD At 30: The History And Future Of The Most Popular BSD-Based OS
\n\nUsing a dedicated administration workstation for my infrastructure
\n\ngit: world - Replace gnu diff, diff3, and sdiff with BSD versions
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
New CIS® FreeBSD 14 Benchmark: Secure Your Systems with Expert-Guided Best Practices, Accelerating ZFS with Copy Offloading: BRT, The uncertain possible futures of Unix graphical desktops, Jailfox - Firefox in a Freebsd Jail, Make Your Own Read-Only Device With NetBSD, ex/vi/nvi editor: .exrc advanced,
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nNew CIS® FreeBSD 14 Benchmark: Secure Your Systems with Expert-Guided Best Practices
\n\nAccelerating ZFS with Copy Offloading: BRT
\n\nThe uncertain possible futures of Unix graphical desktops
\n\nJailfox - Firefox in a Freebsd Jail
\n\nMake Your Own Read-Only Device With NetBSD
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Run Linux Containers on FreeBSD 14 with Podman, Open Source FreeBSD NAS: Maintenance Best Practices, Self-hosting Bitwarden / VaultWarden on FreeBSD, I most definitely should (self-host)!, My 71 TiB ZFS NAS After 10 Years and Zero Drive Failures, Make Your Own CDN With OpenBSD Base and Just 2 Packages, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nOpen Source FreeBSD NAS: Maintenance Best Practices
\n\nSelf-hosting Bitwarden / VaultWarden on FreeBSD
\n\nI most definitely should (self-host)!
\n\nMy 71 TiB ZFS NAS After 10 Years and Zero Drive Failures
\n\nMake Your Own CDN With OpenBSD Base and Just 2 Packages
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nMessage from JT... the problem is spam, sometimes real messages get lost in flood of spam, if we don't cover your email within a few weeks, please email back in.
And now... for some laughs, I shall share with you all, some of the delightful spam we have gotten for your entertainment.
\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Why laptop support, why now: FreeBSD’s strategic move toward broader adoption, ZBM 101: Introduction to ZFSBootMenu, How I batch apply and save one-liners, Moving an Entire FreeBSD Installation to a New Host or VM in a Few Easy Steps, How to install "standard" TTF Microsoft fonts, We need more zero config tools, Reasons I still love the fish shell, You Have Installed OpenBSD. Now For The Daily Tasks, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nWhy laptop support, why now: FreeBSD’s strategic move toward broader adoption
\n\nZBM 101: Introduction to ZFSBootMenu
\n\nHow I batch apply and save one-liners
\n\nMoving an Entire FreeBSD Installation to a New Host or VM in a Few Easy Steps
\n\nHow to install "standard" TTF Microsoft fonts
\n\nWe need more zero config tools
\n\nReasons I still love the fish shell
\n\nYou Have Installed OpenBSD. Now For The Daily Tasks.
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Debunking Common Myths About FreeBSD - Part 2, FreeBSD 13.4-RELEASE Announcement, OpenBSD -current has moved to version 7.6, acpidumping,Install snac2 on FreeBSD – An ActivityPub Instance for the Fediverse, Managing dotfiles with chezmoi, Podman testing on FreeBSD, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nDebunking Common Myths About FreeBSD - Part 2
\n\nFreeBSD 13.4-RELEASE Announcement
\nFreeBSD 14.0 end-of-life - You should have upgraded to 14.1 by now
OpenBSD -current has moved to version 7.6
\n\nInstall snac2 on FreeBSD – An ActivityPub Instance for the Fediverse
\n\nInstalling Uptime-Kuma on a FreeBSD Jail
\n\nManaging dotfiles with chezmoi
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Jason is still on location at EuroBSDcon getting interviews with those in the BSD Community.
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Jason is on location at EuroBSDcon getting interviews with those in the BSD Community.
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Jason is on location at EuroBSDcon getting interviews with those in the BSD Community.
","summary":"Jason is on location at EuroBSDcon getting interviews with those in the BSD Community.","date_published":"2024-10-03T08:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/22c6b8d0-ef8b-4925-b6a7-ea8a666dec26.mp3","mime_type":"audio/mpeg","size_in_bytes":54336384,"duration_in_seconds":3396}]},{"id":"9ccb83c4-7aca-44f6-85bd-8e3e3487f781","title":"578: KVM, but Smol","url":"https://www.bsdnow.tv/578","content_text":"Limiting Process Priority in a FreeBSD Jail, Why You Should Use FreeBSD, The web fun fact that domains can end in dots and canonicalization failures, Replacing postfix with dma + auth, modern unix tool list, Smol KVM, The Computers of Voyager\n\nNOTES\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nFreeBSD Tips and Tricks: Limiting Process Priority in a FreeBSD Jail\n\n\n\nWhy You Should Use FreeBSD\n\n\n\nNews Roundup\n\nThe web fun fact that domains can end in dots and canonicalization failures\n\n\n\nReplacing postfix with dma + auth\n\n\n\nmodern unix tool list\n\n\n\nSmol KVM\n\n\n\nThe Computers of Voyager\n\n\n\nBeastie Bits\n\n\nNo unmodified files remain from original import of OpenBSD\nThe BSDCan 2024 Playlist is now complete\nUDP parallel input committed to -current\nYour browser is your Computer\nFor the member-berries\n\n\n\n\nTarsnap\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\nFeedback/Questions\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\nJoin us and other BSD Fans in our BSD Now Telegram channel\n\n\n","content_html":"Limiting Process Priority in a FreeBSD Jail, Why You Should Use FreeBSD, The web fun fact that domains can end in dots and canonicalization failures, Replacing postfix with dma + auth, modern unix tool list, Smol KVM, The Computers of Voyager
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD Tips and Tricks: Limiting Process Priority in a FreeBSD Jail
\n\nThe web fun fact that domains can end in dots and canonicalization failures
\n\nReplacing postfix with dma + auth
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
New Host Introduction 🤭, From Bridging to Routing With FreeBSD, Sovereign Tech Fund to Invest €686,400 in FreeBSD Infrastructure Modernization, The Dying Computer Museum, In practice, abstractions hide their underlying details, LZ4 Compression Algorithm Gets Multi-Threaded Update, Using Windows or Linux on FreeBSD's vm-bhyve, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\n[New Host Introduction]
\n\nEvolving the BSD Cafe Network Setup: From Bridging to Routing With FreeBSD
\n\nSovereign Tech Fund to Invest €686,400 in FreeBSD Infrastructure Modernization
\n\nIn practice, abstractions hide their underlying details
\n\nLZ4 Compression Algorithm Gets Multi-Threaded Update
\n\nUsing Windows or Linux on FreeBSD's vm-bhyve
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nhttps://github.com/BSDNow/bsdnow.tv/blob/master/episodes/577/feedback/Derek%20-%20Thanks.md
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
From Cloud Chaos to FreeBSD Efficiency, August 2024 Foundation Update, Email encryption at rest on OpenBSD using dovecot and GPG, Workarounds are often forever (unless you work to make them otherwise), Remote Desktop using RDP and VNC, Iconography of the X Window System: The Boot Stipple, Plan 9 is a Uniquely Complete Operating System, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFrom Cloud Chaos to FreeBSD Efficiency
\n\nEmails encryption at rest on OpenBSD using dovecot and GPG
\n\nWorkarounds are often forever (unless you work to make them otherwise)
\n\nRemote Desktop using RDP and VNC
\n\nIconography of the X Window System: The Boot Stipple
\n\nPlan 9 is a Uniquely Complete Operating System
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
X Window System At 40, Lessons from Ancient File Systems, HardenedBSD July 2024 Status Report, FreeBSD's 'root on ZFS' is appealing, I Miss BSD/Linux, Simple automated deployments using git
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nLessons from Ancient File Systems
\n\nHardenedBSD July 2024 Status Report
\n\nFreeBSD's 'root on ZFS' default appeals to me for an odd reason
\n\nSimple automated deployments using git push
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Antithesis: Pioneering Deterministic Hypervisors with FreeBSD and Bhyve, Our slowly growing Unix monoculture, The six dumbest ideas in computer security (2005), Video Edition notes on OpenBSD, Full-featured email server running OpenBSD, ever heard of teaching a case study of Initial Unix?, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nAntithesis: Pioneering Deterministic Hypervisors with FreeBSD and Bhyve
\n\nOur slowly growing Unix monoculture
\n\nThe six dumbest ideas in computer security (2005) + HN Thread
\n\nVideo Edition notes on OpenBSD
\n\nFull-featured email server running OpenBSD
\n\nAnyone ever heard of teaching a case study of Initial Unix?
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
What Would It Take to Recreate Bell Labs?, Human Scale Software vs Open Source, How to run Visual Studio (VS) Code Remote over SSH on FreeBSD 13 and 14, Why are some emails from Charlie Root and others are from root?, Backward compatibility has real costs even for settings, Kyua graduates, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nWhat Would It Take to Recreate Bell Labs?
\n\nHuman Scale Software vs Open Source
\n\nHow to run Visual Studio (VS) Code Remote over SSH on FreeBSD 13 and 14
\n\nWhy are some emails from Charlie Root and others are from root?
\n\nBackward compatibility, even for settings, has real costs
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\n573 - Vedran - linuxulator
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
OpenBSD Workstation for the People, Bridging Networks Across VPS With Wireguard and VXLAN on FreeBSD, Updating FreeBSD the Manual Way, Part of (computer) security is convincing people that it works, Where’s my backup?, Vi and Vim: A Brief Overview, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nOpenBSD Workstation for the People
\n\nBridging Networks Across VPS With Wireguard and VXLAN on FreeBSD
\n\nUpdating FreeBSD the Manual Way
\n\nPart of (computer) security is convincing people that it works
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Navigating FreeBSD’s New Quarterly and Biennial Release Schedule, EuroBSDCon 2024 Schedule, From Cloud Chaos to FreeBSD Efficiency, Local-to-anchors tables in PF rules, CloudBSD, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nNavigating FreeBSD’s New Quarterly and Biennial Release Schedule
\n\nhttps://mccd.space/posts/netbsd-review/
\n\nFrom Cloud Chaos to FreeBSD Efficiency
\n\nEnable local-to-anchors tables in PF rules
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nRick - Feedback about Docs Bugs
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
The FreeBSD-native-ish home lab and network, FreeBSD 14.1: What’s new, and how did we get here?, The Old Computer Challenge v4 (Olympics edition), MicroMac, a Macintosh for under £5, Adding a USB Port to the ThinkPad X1 Nano (the Hard Way), Reasons to use your shell's job control, RIP dhclient(8)
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nThe FreeBSD-native-ish home lab and network
\n\nFreeBSD 14.1: What’s new, and how did we get here?
\n\nThe Old Computer Challenge v4 (Olympics edition)
\n\nMicroMac, a Macintosh for under £5
\n\nAdding a USB Port to the ThinkPad X1 Nano (the Hard Way)
\n\nReasons to use your shell's job control
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Special Guest: Brad Alexander.
","summary":"The FreeBSD-native-ish home lab and network, FreeBSD 14.1: What’s new, and how did we get here?, The Old Computer Challenge v4 (Olympics edition), MicroMac, a Macintosh for under £5, Adding a USB Port to the ThinkPad X1 Nano (the Hard Way), Reasons to use your shell's job control, RIP dhclient(8)","date_published":"2024-08-01T08:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/4654582b-5a43-46d1-ac04-0be04cf72754.mp3","mime_type":"audio/mpeg","size_in_bytes":56421504,"duration_in_seconds":3526}]},{"id":"766ceaa1-9d99-40fc-8a8c-b640d050e19e","title":"569: The ZFS Pi","url":"https://www.bsdnow.tv/569","content_text":"Enhancing FreeBSD Stability With ZFS Pool Checkpoints, Plaintext is not a great format for (system) logs, Initial playlist of 28 BSDCan Videos released, Installing FreeBSD 14 on Raspberry Pi 4B with ZFS root, A practical guide to VPNs, IPv6, routing domains and IPSEC, How to mount ISO or file disk images on OpenBSD, and\nmore\n\nNOTES\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nEnhancing FreeBSD Stability With ZFS Pool Checkpoints\n\n\n\nPlaintext is not a great format for (system) logs\n\n\n\nNews Roundup\n\nInitial playlist of 28 BSDCan Videos released\n\n\n\nInstalling FreeBSD 14 on Raspberry Pi 4B with ZFS root\n\n\nThe following components make up my setup:\n\n\nRaspberry Pi 4B, 8 GB RAM\nOfficial Raspberry Pi 4 Power Supply\nGeekworm Raspberry Pi 4 11mm Embedded Heatsink (P165-B)\nGeekworm for Raspberry Pi 4, X862 V2.0 M.2 NGFF SATA SSD Storage Expansion Board with USB 3.1 Connector Support Key-B 2280 SSD\nWD Blue SA510 SATA SSD 2 TB M.2 2280\n4K 60Hz Micro HDMI to HDMI Adapter (to connect to a monitor, can also run headless with just power and network cable connected)\n\n\n\n\n\nA practical guide to VPNs, IPv6, routing domains and IPSEC\n\n\n\nHow to mount ISO or file disk images on OpenBSD\n\n\n\nBeastie Bits\n\n\nDeadBSD Series - There have been a few FreeBSD derived OS’s over the years, some stay, many others fade away. In this series, DeadBSD’s, we will be revisiting those long gone BSD’s and see what we missed out on.\nFury\nCultBSD\n\n\n\n\nTarsnap\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\nFeedback/Questions\n\n569 - RobN - A Thanks\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\nJoin us and other BSD Fans in our BSD Now Telegram channel\n\n\n","content_html":"Enhancing FreeBSD Stability With ZFS Pool Checkpoints, Plaintext is not a great format for (system) logs, Initial playlist of 28 BSDCan Videos released, Installing FreeBSD 14 on Raspberry Pi 4B with ZFS root, A practical guide to VPNs, IPv6, routing domains and IPSEC, How to mount ISO or file disk images on OpenBSD, and
\nmore
NOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nEnhancing FreeBSD Stability With ZFS Pool Checkpoints
\n\nPlaintext is not a great format for (system) logs
\n\nInitial playlist of 28 BSDCan Videos released
\n\nInstalling FreeBSD 14 on Raspberry Pi 4B with ZFS root
\n\nA practical guide to VPNs, IPv6, routing domains and IPSEC
\n\nHow to mount ISO or file disk images on OpenBSD
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\n569 - RobN - A Thanks
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
regreSSHion vulnerability, Improving and debugging FreeBSDs Intel wifi support, FreeBSD adds an implementation of the 9P filesystem, FreeBSD Zero to Desktop Speedrun Challenge, Why and how to run your own FreeBSD package cache, Game of Trees Hub, Why Does FreeBSD Default to Csh/Tcsh, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nregreSSHion: RCE in OpenSSH's server, on glibc-based Linux systems and OpenBSD 9.8
\n\nImproving and debugging FreeBSDs Intel wifi support
\n\nFreeBSD adds an implementation of the 9P filesystem
\n\nFreeBSD Zero to Desktop Speedrun Challenge
\n\nWhy and how to run your own FreeBSD package cache
\n\nGame of Trees Hub: A Git Repository Hosting Service Based on OpenBSD
\n\nWhy Does FreeBSD Default to Csh/Tcsh? Exploring Its Advantages
\n\nAI-assisted computer interfaces of the future
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
SSH as a sudo replacement, Core.13 is Now In Office, Running GoToSocial on NetBSD, A DMD package for OpenIndiana, Adding more swap space to Omnios, OpenBSD adds initial support for Qualcomm Snapdragon Elite X after 1 day, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nAdding more swap space to Omnios
\n\nOpenBSD added initial support for Qualcomm Snapdragon Elite X after 1 day
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
A Journey Through 31 Years of Open Source Excellence, Proxmox vs FreeBSD: Which Virtualization Host Performs Better?, Upstreaming FreeBSD Code to the Linux Vector Packet Processor Project, FreeBSD Tips and Tricks: Creating Snapshots With UFS, My Concern With Rust, or a Case for the BSD's, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nCelebrating FreeBSD Day: A Journey Through 31 Years of Open Source Excellence
\n\nProxmox vs FreeBSD: Which Virtualization Host Performs Better?
\n\nUpstreaming FreeBSD Code to the Linux Vector Packet Processor Project
\n\nFreeBSD Tips and Tricks: Creating Snapshots With UFS
\n\nMy Concern With Rust, or a Case for the BSD's
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
NetBSD 10 on a Pinebook Pro, OpenBSD extreme privacy setup, Version 256 of systemd boasts '42% less Unix philosophy', Posix.1 2024 is out, Blocking Access From or to Specific Countries Using FreeBSD and Pf, and more.
\nDate: 2024.06.17
NOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nVersion 256 of systemd boasts '42% less Unix philosophy'
\n\nBlocking Access From or to Specific Countries Using FreeBSD and Pf
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Results from the 2024 FreeBSD Community Survey Report, What is Computer Science? ~1967, Computation Poems, Old Info, but still good -- HOWTO: Set up and configure security/sshguard-pf, observium-freebsd-install, FreeBSD Tips and Tricks: Native Read-Only Root File System, OpenSSH introduces options to penalize undesirable behavior, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nResults from the 2024 FreeBSD Community Survey Report
\n\nWhat is Computer Science? ~1967
\n\nOld Info, but still good -- HOWTO: Set up and configure security/sshguard-pf
\n\nFreeBSD Tips and Tricks: Native Read-Only Root File System
\n\nOpenSSH introduces options to penalize undesirable behavior
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
FreeBSD 14.1-RELEASE Announcement, Automatic dark mode with OpenBSD and dwm, dhcp6leased(8) imported to -current, DHCPv6-PD - First steps by florian@, Replacing my OPNsense gateway hardware by a Protectli appliance, How to alter file owernship and permissions with a feedback information, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD 14.1-RELEASE Announcement
\n\nAutomatic dark mode with OpenBSD and dwm
\n\ndhcp6leased(8) imported to -current
\n\nDHCPv6-PD - First steps by florian@
\n\nReplacing my OPNsense gateway hardware by a Protectli appliance
\n\nHow to alter file owernship and permissions with a feedback information
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
My personal BSDCan Devsummit and Schedule, Syncthing, Paperless-ngx, neovim, Things we always remind ourselves while coding, and more.
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD Devsummit 2024 Schedule
\n\n\n\nA list of things I was drawn deeper into, got excited about, and wanted to tell you more about.
\n\nThings we always remind ourselves while coding
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Why FreeBSD Continues to Innovate and Thrive, Why BSD, A BSD person tries Alpine Linux, This message does not exist, Demise of Nagle's algorithm, How Jerry Pournelle Got Kicked Off the ARPANET, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nWhy FreeBSD Continues to Innovate and Thrive
\n\nA BSD person tries Alpine Linux
\n\nDemise of Nagle's algorithm (RFC 896 - Congestion Control) predicted via sysctl
\n\nHow Jerry Pournelle Got Kicked Off the ARPANET
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
FreeBSD Status Report First Quarter 2024, Why not BSD, LibreSSL version 3.9.2 released, Running NetBSD on OmniOS using bhyve, X.Org on NetBSD, Unix version control lore: what, ident, How I search in 2024, sshd split into multiple binaries, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD Status Report First Quarter 2024
\n\nWhy not BSD + Sequel next week
\n\nLibreSSL version 3.9.2 released
\n\nRunning NetBSD on OmniOS using bhyve
\n\nX.Org on NetBSD - the state of things
\n\nUnix version control lore: what, ident
\n\nsshd(8) split into multiple binaries
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
An RNG that runs in your brain, Going Stateless, SmolBSD, The Wi-Fi only works when it's raining, Wayland, where are we in 2024?, Omnios pxe booting, OpenBSD scripts to convert wg-quick VPN files, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nAn RNG that runs in your brain
\n\nThe Wi-Fi only works when it's raining
\n\nWayland, where are we in 2024? Any good for being the default?
\n\nOpenBSD scripts to convert wg-quick VPN files
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
NetBSD 9.4, FreeBSD SSDF Attestation to Support Cybersecurity Compliance, The Lost Worlds of Telnet, alter file ownership and permissions with a feedback information, parallel raw IP input, OpenBSD routers on AliExpress mini PCs, FreeBSD for Devs. Plus a special interview with the organizers of BSDCAN 2024.
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD Foundation Delivers V1 of FreeBSD SSDF Attestation to Support Cybersecurity Compliance
\n\nHow to alter file ownership and permissions with a feedback information
\n\nComing soon to a -current system near you: parallel raw IP input
\n\nOpenBSD routers on AliExpress mini PCs
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Open Source Software: The $9 Trillion Resource Companies Take for Granted, Tinkering with Manjaro and NetBSD on the Pinebook Pro: a crumbs-in-the-forest tutorial & review, OpenSMTPD 7.5.0p0 Released, OpenBSD 7.5 locks down with improved disk encryption support and syscall limitations, Book 8088, Custom Prometheus dashboards using Console templates, FreeBSD Foundation March 2024 Partnerships Update, Ray tracing made possible on 42-year-old ZX Spectrum: 'reasonably fast, if you consider 17 hours per frame to be reasonably fast', and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nOpen Source Software: The $9 Trillion Resource Companies Take for Granted
\n\nTinkering with Manjaro and NetBSD on the Pinebook Pro: a crumbs-in-the-forest tutorial & review
\n\nOpenBSD 7.5 locks down with improved disk encryption support and syscall limitations
\n\nCustom Prometheus dashboards using Console templates
\n\nFreeBSD Foundation March 2024 Partnerships Update
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
OpenBSD is a Cozy Operating System, Lichee Console 4A - RISC-V mini laptop, Lessons learned with XZ vulnerability, Techies vs spies: the xz backdoor debate, Not Not Porting 9front to Power64, One less Un*xy option for 32-bit PowerPC, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nOpenBSD is a Cozy Operating System
\n\nLichee Console 4A - RISC-V mini laptop
\n\nLessons learned with XZ vulnerability
\n\nTechies vs spies: the xz backdoor debate
\n\nNot Not Porting 9front to Power64
\n\nOne less Un*xy option for 32-bit PowerPC
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Kubernetes and back - Why I don't run distributed systems, NetApp’s strategic contributions to FreeBSD: a deep dive into upstreaming efforts, Make your own E-Mail server - Part 2 - Adding Webmail and More with Nextcloud, Poudriere on Apple Silicon, One less Un*xy option for 32-bit PowerPC, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nKubernetes and back - Why I don't run distributed systems
\n\nNetApp’s strategic contributions to FreeBSD: a deep dive into upstreaming efforts
\n\nMake your own E-Mail server - Part 2 - Adding Webmail and More with Nextcloud
\n\nOne less Un*xy option for 32-bit PowerPC
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
The XZ Backdoor, NetBSD 10.0, iX announces that they will put out a release of TrueNAS 13.3, State of the Terminal, LibreSSL 3.8.4 and 3.9.1 released and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nPeople have no doubt heard of this by now, but are not aware of the BSD side of
\nthings since its mostly been Linux getting all the news. It'd be nice if we
\ncould give a summary of the issue and then address how it does/doesn't affect
\nthe BSDs.
\nThe XZ Backdoor
\n
iX announces that they will put out a release of TrueNAS 13.3
\n\n\n\nLibreSSL 3.8.4 and 3.9.1 released
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nDerek via feedback has asked for some discussion around this NetBSD security advisory
\n-- Advisory Link
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Using Git offline, Make your own E-mail server, quiz: a tool for rapid OpenZFS development, Configuring openzfs for nvme databases, Mirroring OmniOS: The Complete Guide part 1, Installing OpenBSD 7.4 on a VisionFive 2 rev, and more...
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nMake your own E-Mail server - FreeBSD, OpenSMTPD, Rspamd and Dovecot included - Part 1
\n\nquiz: a tool for rapid OpenZFS development
\n\nConfiguring openzfs for nvme databases
\n\nMirroring OmniOS: The Complete Guide; Part One
\n\nInstalling OpenBSD 7.4 on a VisionFive 2 rev 1.2a
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Setup diskless booting of Raspberry Pi, TrueNAS abandons FreeBSD, SPARCbook 3000ST: The coolest 90s laptop, Sparkbook Teardown, SSH over HTTPS, Keycloak Identity and Access Management on FreeBSD, Ford Aerospace and BSD Unix, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nSPARCbook 3000ST: The coolest 90s laptop
\nSparkbook Teardown
\nAuthor Comment
Keycloak Identity and Access Management on FreeBSD
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
This week on the show, The story of SSH getting port 22, GGC using Clang, AuxRunner, Stabweek, Using a Kensington SlimBladePro on OpenBSD, and more...
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nThe story of getting SSH port 22
\n\nCan GCC use Clang as its assembler?
\n\nAUXrunner: a macOS QEMU-based app for running A/UX
\n\nUsing the Kensington SlimBlade Pro TrackBall with OpenBSD
\n\nRunning 9front on an emulated SGI Indy via MAME
\n\nHuffman Codes – How Do They Work?
\nNetBSD 10.0_RC5
\nNew code for SIGILL faults help identify misbranches
\nNew Illumos telegram channel
\nThe Jan Feb issues of the FreeBSD Journal is here
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
This week on the show, you're not too late to develop the future, netmap on czgbe, OpenZFS 2.2.3, SSH Brute Forcing, some unknown OpenBSD Features, Release notes for the latest Omni OS, and more...
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nWhen the Power Macintosh ran NetWare (featuring Wormhole and Cyberpunk)
\n\nA recent abrupt change in Internet SSH brute force attacks against us
\n\nSome OpenBSD features that aren't widely known
\n\nRelease Notes for OmniOS v11 r151048
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
FreeBSD Foundation Statement on the European Union Cyber Resiliency Act, DragonFly BSD on a Thinkpad T480s, How FreeBSD
\n Employs Ampere Arm64 Servers in the Data Center, FreeBSD Yubikey authentication, that time I almost added Tetris to htop, and more
NOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD Foundation Statement on the European Union Cyber Resiliency Act
\n\nDragonFly BSD on a Thinkpad T480s
\n\nAmpere in the Wild: How FreeBSD Employs Ampere Arm64 Servers in the Data Center
\n\nFreeBSD Yubikey authentication
\n\nThat time I almost added Tetris to htop
\n\nMail Software Projects for You
\nAt long last: the MWL Title Index
\nFreeBSD on a RPi
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
FreeBSD Status Report Q4 2023, In Memorium of the NTP inventor, Migrate a FreeBSD bhyve virtual machine to OmniOS, AI-free blog, Hard disk LEDs and Noisy Machines, SSH based comment system, NetBSD 10 RC.4 is available, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nFreeBSD Status Report Fourth Quarter 2023
\n\nIn Memoriam : Inventor of NTP protocol that keeps time on billions of devices dies at age 85
\n\nMigrate a FreeBSD bhyve virtual machine to OmniOS
\n\nHard disk LEDs and Noisy Machines
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Overcoming imposter syndrome in IT, A Practical Guide to GNU sed With Examples, Early computer art by Barbara Nessim, Don't prefill config files, Trapping Spambots Based on Target Domain Only, You cannot cURL under pressure, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nOvercoming imposter syndrome in IT
\n\nA Practical Guide to GNU sed With Examples
\n\nEarly computer art by Barbara Nessim (1984)
\n\nA Simpler Life: Trapping Spambots Based on Target Domain Only
\n\nYou cannot cURL under pressure
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Debunking Common Myths About FreeBSD, Please, don’t force me to log in, Exploring FreeBSD service(8) basics, Failed Product Designs: A Laptop with Seven Screens, What’s a Permissive License – and Why Should I Care?, Beginning of the year Laugh
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nDebunking Common Myths About FreeBSD
\n\nPlease, don’t force me to log in
\n\nExploring FreeBSD service(8) basics
\n\nFailed Product Designs: A Laptop with Seven Screens
\nThe Expanscape Aurora 7
“What’s a Permissive License – and Why Should I Care?”
\n\nNetBSD 10: Thirty Years, Still Going Strong!
\nDracula theme using bash shell
\npinsyscalls(2) working in anger
\nFirst bits of a Haiku compatibility layer for NetBSD
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
ZFS High Availability with Asynchronous Replication and zrep, Stop
\nBlogging and start documenting, 2023 in Review: Infrastructure, NovaCustom NV41
\nlaptop review, OpenBSD Video Audio Screen Recording, HDMI Audio sound patches
\ninto GhostBSD source code, DSA removal from OpenSSH, NetBSD/evbppc 10.99.10 on
\nthe Nintendo Wii, NetBSD/amd64 current performance patch
NOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nZFS High Availability with
\nAsynchronous Replication and zrep
Stop Blogging and start documenting
\n\n2023 in Review: Infrastructure
\n\nOpenBSD Video Audio Screen Recording
\n\nHDMI Audio sound patches into GhostBSD source code /usr/ghost14/ghostbsd-src SOLVED Jan20 2024
\n\nNetBSD/evbppc 10.99.10 on the Nintendo Wii
\n\nNetBSD/amd64 current performance patch
\n\nNovember/December 2023 FreeBSD Journal Issue
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
GPL 3: The Controversial Licensing Model and Potential Solutions,
\nThe Geeks way of checking what the outside weather is like, Alpine on a
\nFreeBSD Jail, DragonFly BSD on a Thinkpad T480s, Dealing with USB Storage
\ndevices on OmniOS, Creating a Time Capsule instance using Samba, FreeBSD, and
\nZFS
NOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nGPL 3: The Controversial Licensing Model and Potential Solutions
\n\nThe Geeks way of checking what the outside wheather is like
\n\nDragonFly BSD on a Thinkpad T480s
\n\nDealing with USB Storage devices on OmniOS
\n\nCreating a Time Capsule instance using Samba, FreeBSD, and ZFS
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
OpenZFS Storage Best Practices and Use Cases Part 3: Databases and VMs, 2023 in Review: Continuous Integration and Workflow Improvement, Running OpenBSD on OmniOS using bhyve, FreeBSD jailed ZFS datasets – how do I find the .zfs/snapshot directory?, OpenBSD workstation hardening, KDE Plasma now linked to packages build on -current, MidnightBSD 3.1.3 release
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nOpenZFS Storage Best Practices and Use Cases Part 3: Databases and VMs
\n\n2023 in Review: Continuous Integration and Workflow Improvement
\n\nRunning OpenBSD on OmniOS using bhyve
\n\nFreeBSD jailed ZFS datasets – how do I find the .zfs/snapshot directory?
\n\nKDE Plasma now linked to packages build on -current
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
8 Open Source Trends to Keep an Eye Out for in 2024, System Design
\nfor Advanced Beginners, 2024 plans and 2023 retrospective, Upgrading from NetBSD 5.1 to 10*RC1, FreeBSD has a new C compiler: Oracle Developer Studio 12.6, Ctrl+Alt Museum
NOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\n8 Open Source Trends to Keep an Eye Out for in 2024
\n\nSystem Design for Advanced Beginners
\n\n2024 plans and 2023 retrospective
\n\nUpgrading from NetBSD 5.1 to 10_RC1
\n\nFreeBSD has a new C compiler: Oracle Developer Studio 12.6
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Security, Performance, and Interoperability; Introducing FreeBSD 14, HardenedBSD November 2023 Status Report, How to create a FreeBSD Jail hosting a remote desktop, A sneak Peak, Programming FreeBSD Reading Process Information, Why Unix kernels have grown caches for directory entries 'name caches', Always learning, Always Teaching
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Terrapin Attack, SSH Hardening with ssh-audit, MidnightBSD 3.1.2, syscall(2) removed from -current, 2024 FreeBSD Community Survey is Here
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
In this special holiday episode, we, the BSDNow hosts, get together to answer questions that listeners have sent us over time. We give you updates on our gear, books we read, favorite places, and a whole lot more. Enjoy!
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nDAK and the Golden Age of Gadget Catalogs, FreeBSD 13.2 upgrade to 14.0, Running OpenBSD on Raspberry Pi Zero 2 W, Netgate Releases pfSense CE Software Version 2.7.1, SSH agent forwarding and tmux done right, Some explanations about OpenBSD memory usage, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nOpenZFS Storage Best Practices and Use Cases pt 2, MNT Reform – almost a year on, Why do I know shell, and how can you, Authenticate the SSH servers you are connecting to, dsynth in DragonFly, Navigating around in shell, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nOpenZFS Storage Best Practices and Use Cases, EuroBSDcon trip report, Disks from the Perspective of a File System, Creating Jails using flavours in pot, OpenIKED 7.3 released, OpenSMTPD 7.4.0p1 Released, FreeBSD can now boot in 25 milliseconds, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nFreeBSD 14 has been released, Reading your RSS feed on FreeBSD, Manipulate PDF files easily with pdftk, clang(1)/llvm updated to version 16 in OpenBSD, NetBSD Security Advisory: multiple vulnerabilities in ftpd(8), and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [Quick update](https://www.daemonology.net/blog/2023-11-21-late-breaking-FreeBSD-14-breakage.html)\n• [Vermaden’s FreeBSD 14 valuable news] (https://vermaden.wordpress.com/2023/11/17/valuable-freebsd-14-0-release-updates)\n
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nMigrating from an Old Linux Server to a New FreeBSD Machine, The Internet Was Designed With a Narrow Waist, The Worst New Guys In History, FreeBSD Jails vs. Docker: A Comparison, Oracle Developer Studio 12.6 on Illumos
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nFreeBSD on the RISC-V Architecture, A bit of XENIX history, pkgbase: Official packages, recover lost text by coredumping firefox, FuguIta 7.4 has been released, LibreSSL 3.8.2 Released, OpenSMTPD 7.4.0p0 Released
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\n218 dollars to open source, EuroBSDCon 2023 Trip Report, FreeBSD vs Linux (Debian), Introduction to sysclean8, Run your own Syncthing discovery server on OpenBSD, FreeBSD years: 2000-2005, Using OpenBSD relayd(8) as an Application Layer Gateway, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nOpenBSD 7.4, Making Software Last Forever, DragonFlyBSD Per-process capability-based restrictions, HardenedBSD September 2023 Status Report, NetBSD as a Kubernetes Pod, Firefox hardening with Arkenfox, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nImplementing a system call for OpenBSD, Self-Hosted Email services on OpenBSD, First 5 Minutes on a New FreeBSD Server, OLD COMPUTER RESCUE - X201, sec(4) for Route Based IPSec VPNs, send syslog messages using command-line utilities, Keeping email sorted (the hard way), and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nAdopting FreeBSD as Your Open Source Operating System, How Hard is it to Adapt a Memory Allocator to CHERI, Running Stable Diffusion on FreeBSD, Self-hosting Pixelfed on OpenBSD, Time Capsule instance using Samba, FreeBSD, and ZFS, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [OpenZFS on Twitter](https://x.com/openzfs/status/1704212154558324827?s=12&t=-_bfM_adaiX8Ri_3lN9OYw)\n• [EuroBSDcon 2023, Portugal](https://m.youtube.com/playlist?list=PLskKNopggjc7s6nAMxKF0tAO77ZIowZdx&cbrd=1)\n• [The lost history if Emoticons](https://x.com/rainmaker1973/status/1704006098909352016?s=12&t=-_bfM_adaiX8Ri_3lN9OYw)\n• [Solving the same problem](https://blog.fredrb.com/2023/09/08/same-problem-multiple-times/)\n• [http://vihart.com/fifty-fizzbuzzes/](50 Fizz buzzes)\n
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nIf you can use Open Source you can build hardware, Good performance is not just big O, Proof You Should Not Run MWL Code, How to add pledge to a program in OpenBSD, 3D printing on OpenBSD, Getting the right type of certificate, Jenny’s Daily Drivers, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nUnlocking Infrastructure Sovereignty, first meeting of the FreeBSD Enterprise Working Group, HardenedBSD August 2023 Status Report, GhostBSD August 2023 donation report, MidnightBSD 3.1 Released, OpenBSD Webzine ISSUE #14, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [HardenedBSD 14-STABLE Now Available](https://hardenedbsd.org/article/shawn-webb/2023-09-11/hardenedbsd-14-stable-now-available)\n
\n\n• [Late on the announcement but... GhostBSD 23.06.01 ISO is now available](http://ghostbsd.org/23.06.01_iso_is_now_available)\n
\n\n• [ZFS for Dummies](https://ikrima.dev/dev-notes/homelab/zfs-for-dummies/)\n• [The Switch runs FreeBSD](https://www.reddit.com/r/NintendoSwitch/comments/5xbe5a/the_switch_runs_freebsd_making_it_nintendos_first/)\n• [KDE on OpenBSD](https://marc.info/?l=openbsd-ports&m=169391479324962)\n• [(Kubernetes v1.28.0) for illumos, FreeBSD and OpenBSD](https://medium.com/@norlin.t/by-the-way-planternetes-kubernetes-v1-28-0-for-illumos-freebsd-and-openbsd-5d57026d6a25)\n• [Video: C Programming on System 6 - VCF Midwest, Wi-Fi DA](https://jcs.org/2023/09/20/vcfmw)\n
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nWhy DNS is still hard to learn, Unix support 50 years ago, ZFS Replication tools, Between ISA and PCI, PCs had EISA and VLB, Old Computer Challenge v3, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [Installing and Using Research Unix Version 7 on the OpenSIMH PDP-11 Emulator](https://decuser.github.io/unix/research-unix/v7/videos/2023/07/14/installing-and-using-research-unix-v7-in-open-simh-video.html)\n• [Cheat Sheets](https://github.com/cheat/cheatsheets/tree/master)\n• [Introducing BSD Cafe](https://www.reddit.com/r/BSD/comments/15rt7em/introducing_the_bsdcafe/)\n• [Keystroke timing obfuscation added to ssh(1)](http://undeadly.org/cgi?action=article;sid=20230829051257)\n
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nDo one thing and do it well, Turning a 15 years old laptop into a children proof retrogaming station, Old Computer Challenge v3: day 1, It Takes 6 Days to Change 1 Line of Code, Rejected GitHub Profile Achievements, that old netbsd server, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
\n\nOn the Loss and Preservation of Knowledge, Unix Recovery Legend, Useful Unix commands for data science, Tarsnap outage post-mortem, OpenBSD 7.3 on a twenty year old IBM ThinkPad R31, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
The Elements Of Style: UNIX As Literature, The shell and its crappy handling of whitespace, Theo de Raadt on Zenbleed, OPNsense 23.7 released, illumos gets a new C compiler, fixing Thinkpad X1 WIFI on FreeBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
)
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nTop Ten Reasons to Upgrade to FreeBSD 13.2, History never repeats but sometimes it rhymes, Wayland on OpenBSD, OpenBGPD 8.1 released, Shoot yourself in the foot, Zenbleed: aka: The new fun for a while, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD Status Report Q2 2023, Klara Systems Recommended Summer Reads 2023, install Kanboard on OpenBSD howto, A bit of Unix history on 'su -', hints for splitting commits, Live from OpenBSD in Amsterdam, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
In Memoriam: Hans Petter William Sirevåg Selasky
\n\n4 Months of BSD, Self Hosted Calendar and address Book, Ban scanners IPs from OpenSMTP logs, Self-hosted git page, Bastille template example, Restrict nginx Access by Geographical Location on FreeBSD, and more.
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\n3 Advantages to Running FreeBSD as Your Server OS, FreeBSD 14 Release Schedule, Stream your OpenBSD desktop audio, DOD KSOS Secure UNIX Operating System Manual, How to limit bandwidth usage with SCP transfers, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
A Guide to Problem-Solving for Software Developers with Examples, making 20% time work, Long Live Netbooks, OpenBSD Router on Sg105w, Set Up a Simple and Actually Working Wireguard Server, Unix Edition Zero, how to be a -10x engineer, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Linux and FreeBSD Firewalls Comparison Part 2, 27 Years with the Perfect OS, Top 20 OpenSSH Server Best Security Practices, Huge pfsync rewrite, OpenSMTPD 7.3.0p1 release, Running OpenBSD 7.3 on your laptop is really hard (not), and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Linux and FreeBSD Firewalls Part 1, Why Netflix Chose NGINX as the Heart of Its CDN, Protect your web servers against PHP shells and malwares, Installing and running Gitlab howto, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [World built in 36 hours on a Pentium 4!](https://www.reddit.com/r/freebsd/comments/13undl9/world_built_in_36_hours_on_a_pentium_4/)\n• [Fart init](https://x61.sh/log/2023/05/23052023153621-fart-init.html](https://x61.sh/log/2023/05/23052023153621-fart-init.html)\n• [Organized Freebies](https://mwl.io/archives/22832)\n• [OpenSMTPD 7.3.0p0 released](http://undeadly.org/cgi?action=article;sid=20230617111340)\n• [shutdown/reboot now require membership of group _shutdown](http://undeadly.org/cgi?action=article;sid=20230620064255)\n• [Where does my computer get the time from?](https://dotat.at/@/2023-05-26-whence-time.html)\n
\n\nFreeBSD or Linux – A Choice Without OS Wars, The Computer Scientist Who Can’t Stop Telling Stories, ChatGPT was asked to write a pf.conf to spec, GhostBSD 23.06.1 is now available, OPNsense 23.1.9 released, Running VSCode in Chromium on OpenBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
OpenZFS, Your Data and the Challenge of Ransomware, I Didn’t Learn Unix By Reading All The Manpages, I try to answer "how to become a systems engineer", Writing shell scripts in Nushell, Sudo and signal propagation, infecting SSH Public Keys with backdoors, OpenBSD Thinkpad, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nWe have a new show host, Understanding ZFS vdev Types, Don't abuse su for dropping user privileges, Dynamic Tracing on OpenBSD 7.3, new Libressl, Manual Jails on FreeBSD 12, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nRecorded at BSDCan 2023
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\nAs a final thing Allan would like to make an announcement:
\n\nSun Ray laptops, MIPS and getting root on them, OpenZFS for HPC Clusters, Self-Hosted Bookmarks using DAV and httpd on OpenBSD, Terraform + Proxmox + OpenBSD = <3, WOL Plex Server, Against innovation, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
AsiaBSDCon 2023 Trip Report, Converting My X201 ThinkPad into a Slabtop, Stream your OpenBSD desktop audio to other devices, The Gnome and Its "Secret Place", ttyload, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [OpenIndiana with a Sun Microsystems 22" LCD monitor. Running on a 1.8GHz quad core AMD Phenom 9100e processor, 4Gb RAM, nVidia GEForce GT630.](https://www.reddit.com/r/unix/comments/13otjnt/openindiana_with_a_sun_microsystems_22_lcd/)\n• [cron(8) now supports random ranges with steps](https://www.undeadly.org/cgi?action=article;sid=20230507122935&utm_source=bsdweekly)\n• [BSDCan 2024 Reorganization](https://mwl.io/archives/22799)\n• [Depenguin me](https://depenguin.me/)\n
\n\nLeveraging OpenZFS to Build Your Own Storage Appliance, Install OpenBSD as a VM, Set up your own CalDAV and CardDAV servers on OpenBSD, display basic computer information using DMI table decoder, Gpart CheatSheet, Rob Pike on the Origin of Unix Dot File Names, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nFreeBSD Foundation Welcomes New Team Members, OpenZFS the Ideal Storage Solution for University Environments, SCaLE20X Conference Report, 916 days of Emacs, XTerm: It's Better Than You Thought, NetBSD Annual General Meeting 2023, and more
\n\nNOTES**
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Author Michael W. Lucas joins us in this interview to talk about his latest book projects. Find out what he’s up to regarding mail servers, conferences, his views on ChatGPT, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
OpenBSD Mastery Filesystems
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Special Guest: Michael W Lucas.
","summary":"Author Michael W. Lucas joins us in this interview to talk about his latest book projects. Find out what he’s up to regarding mail servers, conferences, his views on ChatGPT, and more.","date_published":"2023-05-18T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/188e3b3f-dc07-43ba-aa49-de8223858ead.mp3","mime_type":"audio/mpeg","size_in_bytes":56347776,"duration_in_seconds":3521}]},{"id":"a130428b-d80d-45a3-a07b-e7b6ce4b3565","title":"506: A greener BSD","url":"https://www.bsdnow.tv/506","content_text":"Comparing Modern Open-Source Storage Solutions, FreeBSD Q1 Status Report, Hello Systems 0.8.1 Release, OpenBSD: Managing an inverter/converter with NUT, Tips for Running a Greener FreeBSD, BSDCAN Registration open\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nComparing Modern Open-Source Storage Solutions OpenZFS vs. The Rest\n\n\n\nFreeBSD Q1 Status Report\n\n\n\nNews Roundup\n\nHello Systems 0.8.1 Release\n\n\n\nOpenBSD: Managing an inverter/converter with NUT\n\n\n\nCelebrating Earth Day: Tips for Running a Greener FreeBSD\n\n\n\nBSDCAN Registration\n\n\n\nBeastie Bits\n\n• [SimCity 2000 running on OpenBSD 7.3 via DOSBox 0.74-3](https://www.reddit.com/r/openbsd_gaming/comments/12k9zt2/simcity_2000_running_on_openbsd_73_via_dosbox_0743/)\n• [OpenBSD Webzine #13](https://webzine.puffy.cafe/issue-13.html)\n• [AWS Gazo bot](https://github.com/csaltos/aws-gazo-bot)\n\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n\n\n","content_html":"Comparing Modern Open-Source Storage Solutions, FreeBSD Q1 Status Report, Hello Systems 0.8.1 Release, OpenBSD: Managing an inverter/converter with NUT, Tips for Running a Greener FreeBSD, BSDCAN Registration open
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [SimCity 2000 running on OpenBSD 7.3 via DOSBox 0.74-3](https://www.reddit.com/r/openbsd_gaming/comments/12k9zt2/simcity_2000_running_on_openbsd_73_via_dosbox_0743/)\n• [OpenBSD Webzine #13](https://webzine.puffy.cafe/issue-13.html)\n• [AWS Gazo bot](https://github.com/csaltos/aws-gazo-bot)\n
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nOpenBSD 7.3 released, Accelerating Datacenter Energy Efficiency by Leveraging FreeBSD as Your Server OS, install Cinnamon as a Desktop environment, xmonad FreeBSD set up from scratch, Burgr books in your terminal, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD 13.2 Release, Using DTrace to find block sizes of ZFS, NFS, and iSCSI, Midnight BSD 3.0.1, Closing a stale SSH connection, How to automatically add identity to the SSH authentication agent, Pros and Cons of FreeBSD for virtual Servers, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
ZFS Optimization Success Stories, Linux Namespaces Are a Poor Man's Plan 9 Namespaces, better support for SSH host certificates, Fast Unix Commands, Fascination with AWK, and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\n5 Key reasons for a OpenZFS Performance Audit, The Ping from Hell, OpenBGPD 7.9 released, Setting the clock ahead to see what breaks, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Nextcloud + OpenBSD = <3, Understanding the Origins of DTrace, Bastille Templates for FreeBSD Jails, Initial support for guided disk encryption in the OpenBSD installer, Dynamic host configuration please, OpenBSD Storage Management tutorial at BSDCan 2023, Jan/Feb 2023 Column Out in the FreeBSD Journal, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nWireguard VPN Server with Unbound on OpenBSD, Auditing for OpenZFS Storage Performance, OpenBSD 7.2 on a Thinkpad X201, Practical Guides to fzf, Replacing postfix with dma, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
We’re interviewing Dan Langille about his new server project. He’ll talk to us about the things he’s building, some of which are a bit out of the ordinary. We’re also talking about BSDCan 2023 and what to expect after returning to an in-presence conference format. Enjoy!
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Special Guest: Dan Langille.
","summary":"We’re interviewing Dan Langille about his new server project. He’ll talk to us about the things he’s building, some of which are a bit out of the ordinary. We’re also talking about BSDCan 2023 and what to expect after returning to an in-presence conference format. Enjoy!","date_published":"2023-03-23T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/b57b3e71-4395-4296-98ea-9eea94bffd1a.mp3","mime_type":"audio/mpeg","size_in_bytes":38735616,"duration_in_seconds":2420}]},{"id":"34def0f7-bb67-4f62-a94c-6ff7ac8576f9","title":"498: Dropping Privileges","url":"https://www.bsdnow.tv/498","content_text":"OpenZFS auditing for storage Performance, Privilege drop; privilege separation; and restricted-service operating mode in OpenBSD, OPNsense 23.1.1 release, Cloning a System with Ansible, FOSDEM 2023, BSDCan 2023 Travel Grants\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nOpenZFS auditing for storage Performance\n\n\n\nPrivilege drop, privilege separation, and restricted-service operating mode in OpenBSD\n\n\n\nNews Roundup\n\nOPNsense 23.1.1 released\n\n\n\nCloning a System with Ansible\n\n\n\nFOSDEM 2023\n\n\n\nBSDCan 2023 Travel Grant Application Now Open\n\n\n\nThe Undeadly Bits\n\nGame of Trees milestone\nGame of Trees Daemon - video and slides (May make the older game of trees obsolete)\namd64 execute-only committed to -current\nUsing /bin/eject with USB flash drives\nTunneling vxlan(4) over WireGuard wg(4)\nConsole screendumps\nExecute-only status report\nOpenBSD in Canada\nPrivilege drop, privilege separation, and restricted-service operating mode in OpenBSD\nTheo de Raadt on pinsyscall(2)\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nKevin - PLUG\nLuna - FOSDEM\n***\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n\n","content_html":"OpenZFS auditing for storage Performance, Privilege drop; privilege separation; and restricted-service operating mode in OpenBSD, OPNsense 23.1.1 release, Cloning a System with Ansible, FOSDEM 2023, BSDCan 2023 Travel Grants
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Game of Trees milestone
\nGame of Trees Daemon - video and slides (May make the older game of trees obsolete)
\namd64 execute-only committed to -current
\nUsing /bin/eject with USB flash drives
\nTunneling vxlan(4) over WireGuard wg(4)
\nConsole screendumps
\nExecute-only status report
\nOpenBSD in Canada
\nPrivilege drop, privilege separation, and restricted-service operating mode in OpenBSD
\nTheo de Raadt on pinsyscall(2)
How to Catch a Bitcoin Miner, A Call For More Collaboration, zstd updates, hating hackathons, How to monitor multiple log files at once, KeePassXC, sshd random relinking at boot, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [Slides](https://fosdem.org/2023/schedule/event/bsd_driver_harmony/attachments/slides/5976/export/events/attachments/bsd_driver_harmony/slides/5976/BSD_Driver_Harmony_FOSDEM.pdf)\n• Video is embedded on the schedule event page\n
\n\nAutomation and Hacking Your FreeBSD CLI, Run your own instant messaging service on FreeBSD, Watch Netflix on FreeBSD, HardenedBSD January 2023 Status Report, How To Set Up SSH Keys With YubiKey as two-factor authentication, OpenSSH fixes double-free memory bug that’s pokable over the network, A late announcement, but better late than never, Next NYC*BUG and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD Status Report Fourth Quarter 2022, How to limit a jail, the parallel port, Hello System 0.8, Solbournes in space, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nMass extinction of UNIX workstations, Determine Who Can Log In to an SSH Server, Factors When Considering FreeBSD vs. Linux Packages, A Visual Guide to SSH Tunnels, Harvesting the Noise While it’s Fresh, Bastille - The Jail Manager on FreeBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Write Admin tools from Day One, Differentiating between Data Security and Data Integrity, 45 year-old Unix tool is finally getting an upgrade, OpenBSD 7.2 on an ODROID-HC4, Dotfiles Management, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD Journal - November/December 2022 - Observability and Metrics
\nHAMMER2 file system for NetBSD
\nRunning OpenBSD 7.2 on your laptop is really hard (not)
\nMinIO on OpenBSD 7.2: Install
\nWireGuard VPN on OpenBSD
\nA tool for glamorous shell scripts
\nVisualize your git commits with a heat map in the terminal
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nWriting your own operating system, Continuous Integration and Quality Assurance Update, feeling for the NetBSD community, Testing wanted: execute-only on amd64, GCC uses Modula-2 and Rust, do they work on OpenBSD, Unix is dead; long live Unix, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [Kevin - Advent of Computing podcast covers BSD](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/492/feedback/Kevin%20-%20Advent%20of%20Computing%20podcast%20covers%20BSD.md)\n• [ilo - thanks](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/492/feedback/ilo%20-%20thanks.md)\n
\n\nDragonfly BSD 6.4 is out, Running OpenZFS – Choosing Between FreeBSD and Linux, OpenBSD Mastery: Filesystems ebook leaks, catching 71% spam, crazy unix shell prompts, Linux Binary Compatibility: Ubuntu on FreeBSD, Reproducible Builds Summit Venice 2022, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD Foundation’s Software Development review of 2022, what can we learn from Vintage Computing, OpenBSD KDE Status Report 2022, a Decade of HardenedBSD, In Praise of Plan9, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
LibreSSL 3.7.0 Released
\nOPNsense 22.7.10 released
\nBSDCan 2023 call for papers
\nHow to lock OpenSSH authentication agent
\nOnce upon a time long ago, I was sitting alone in the UCLA ARPANET site...
FreeBSD vs. Linux – Networking, HDMI sound output through TV speakers on FreeBSD 13, Getting started with tmux, Samba Active Directory, OpenIKED 7.2 released, FreeBSD Plasma 5 GUI Install, DHCP server howto in German, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Finding a 24 year old bug in ping(8), The Role of Operating Systems in IOT, Authentication gateway with SSH on OpenBSD, FreeBSD 12.4 is out, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Vagrant FreeBSD Boxbuilder
\nLibreSSL 3.7.0 Released
\nOPNsense 22.7.9 released
\nBIOS Memory Map for vmd(8) Rewrite in Progress
This year end episode of BSDNow features a trip report to EuroBSDcon by Mr. BSD.tv, as well as an interview with FreeBSD committer John Baldwin. Happy New Year, 2023!
\n\nNOTES***
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Interview topic
\n\nThis special episode features two interviews we did at EuroBSDcon in Vienna this year. We talk with FreeBSD developers about how they got started, their current projects and more. Also, consider donating to your favorite BSD Foundation to keep the projects going.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [FreeBSD Foundation Donation Link](https://freebsdfoundation.org/donate/)\n• [NetBSD Foundation Donation Link](http://www.netbsd.org/donations/#how-to-donate)\n
\n\nInterview topic
\n\nInterview topic
\n\nTails of the M1 GPU, Getting Home Assistant running in a FreeBSD 13.1 jail, interview with AWK creator Dr. Brian Kernighan, Next steps toward mimmutable, Unix's (technical) history is mostly old now, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nVirtualization showdown, The Birth of Standard Error, why Steam started picking a random font, Maintaining Sufficient Free Space with ZFS, updated Apple M1/M2 bootloader, code, FreeBSD on my workstation, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Research Unix Version 6 in the Open SIMH PDP-11 Emulator, The Hot Tub Time Machine is Your ZFS Turn-Back-Time Method, NFS on NetBSD: server and client side, HardenedBSD October 2022 Status Report, Nushell : Introduction, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Unix Pipe Game
\nSlides - The “other” FreeBSD optimizations used by Netflix to serve video at 800Gb/s from a single server
\nMy FreeBSD Friday Lecture: The Writing Scholar’s Guide to FreeBSD
5 Key Reasons to Consider Open Source Storage, OpenBSD Minimalist Desktop, BSD XFCE, Alpine Linux VM on bhyve - with root on ZFS, FreeBSD Jail Quick Setup with Networking, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
EuroBSDcon videos are now up
\nLibreSSL 3.6.1 released
\nRaspberry Pi 4 with FreeBSD 13-RELEASE: A Perfect Miniature Homelab
FreeBSD Q3 2022 status report, Leveraging MinIO and OpenZFS to avoid vendor lock in, FreeBSD on Firecracker platform, How Much Faster Is Making A Tar Archive Without Gzip, Postgres from packages on OpenBSD, Upgrading an NVMe zpool from 222G to 1TB drives, Don't use Reddit for Linux or BSD related questions, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Hinnerk - vnet jails
\nTom’s response example: https://adventurist.me/posts/00304
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nOpenBSD 7.2 and FuguIta have been released, Learn the Whys and Hows with the FreeBSD Sec Team, how to get notified about FreeBSD updates, using unbound for ad blocking on OpenBSD, further memory protections on OpenBSD current, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [“OpenBSD Mastery: Filesystems” Print/Ebook Bundle Preorder](https://mwl.io/archives/22352)\n• [Klara is hiring a FreeBSD Kernel Developer](https://klarasystems.com/careers/freebsd-kernel-developer/)\n• [FreeBSD 12.4-BETA1 Now Available](https://lists.freebsd.org/archives/freebsd-stable/2022-October/000920.html)\n• [Hunting kernel lock and interrupt latency](https://mail-index.netbsd.org/tech-kern/2022/10/30/msg028499.html)\n• [EuroBSDcon 2022 videos available](https://undeadly.org/cgi?action=article;sid=20221027232308)\n
\n\nEuroBSDcon 2022 as first BSD conference, Red Hat’s OpenShift vs FreeBSD Jails, Running a Docker Host under OpenBSD using vmd(8), history of sending signals to Unix process groups, Toolchains adventures - Q3 2022, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
-current has moved to 7.2
\nSeveral /sbin daemons are now dynamically-linked
\nAnnouncing the pkgsrc 2022Q3 branch
Open Source in Enterprise Environments, Your Comprehensive Guide to rc(8): FreeBSD Services and Automation, How Rob Pike got hired by Dennis Richie, what FreeBSD machines rubenerd uses, new debugbreak command, 7 sudo myths debunked
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Analyzing BSD Kernels for Uninitialized Memory Disclosures Using Binary Ninja, Sharing Dual-Licensed Drivers between Linux and FreeBSD, favorite Things About The OpenBSD Packet Filter Tools, How to trigger services restart after OpenBSD update, Gems from the Man Page Trenches, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
The MIPS ThinkPad
\nNix Gems
\nRunning PalmOS without PalmOS
\n"OpenBSD Mastery: Filesystems" draft done!
In this special episode, we interview Warren Toomey from the Unix Historical Society. We chat about his involvement in preserving old Unix systems and why that is important.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Special Guest: Warren Toomey.
","summary":"In this special episode, we interview Warren Toomey from the Unix Historical Society. We chat about his involvement in preserving old Unix systems and why that is important.","date_published":"2022-10-13T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/64bc3a0c-43cf-4e97-af97-b31d799c1154.mp3","mime_type":"audio/mpeg","size_in_bytes":64196352,"duration_in_seconds":2674}]},{"id":"8308672c-2f88-4a7b-9619-ed61184f731d","title":"475: Prompt Injection Attacks","url":"https://www.bsdnow.tv/475","content_text":"Prompt injection attacks against GPT-3, the History of Package Management on FreeBSD, A fresh look at FreeBSD, File Management Tools for Your Favorite Shell, Quick Guide about Video Playback on FreeBSD, and more. \n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nPrompt injection attacks against GPT-3\n\n\n\nA Quick Look at the History of Package Management on FreeBSD\n\n\n\nNews Roundup\n\nA fresh look at FreeBSD\n\n\n\nFile Management Tools for Your Favorite Shell\n\n\n\nVideo Playback on FreeBSD – Quick Guide\n\n\n\nBeastie Bits\n\nps(1) gains support for tree-like display of processes\n... interesting old-timey UNIXes ...\nA retro style online SSH client to play Nethack\nThe Good, the Bad, and the Ugly: The Unix! Legacy\nGame of Trees 0.75 released\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nKen - HPR\nKevin - FreeBSD and EMACS\nNathan - Handbook contribution Question\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"Prompt injection attacks against GPT-3, the History of Package Management on FreeBSD, A fresh look at FreeBSD, File Management Tools for Your Favorite Shell, Quick Guide about Video Playback on FreeBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
ps(1) gains support for tree-like display of processes
\n... interesting old-timey UNIXes ...
\nA retro style online SSH client to play Nethack
\nThe Good, the Bad, and the Ugly: The Unix! Legacy
\nGame of Trees 0.75 released
Deploying FreeBSD on Oracle Cloud, A Tale of 300,000 Imaginary Friends, EuroBSDcon 2022 recap, OpenBSD Mastery: Filesystems” Status Report, OpenBGPD 7.6 Released, immutable userland mappings, Portable OpenSSH commits now SSH-signed, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nWriting FreeBSD kernel modules in Rust, Details behind the FreeBSD aio LPE, Linux subsystem for FreeBSD, FreeBSD Journal: Science, Systems, and FreeBSD, NetBSD improves Amiga support, OpenBSD on Scaleway Elastic Metal, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD on the Framework Laptop, Win32 is the only stable ABI on Linux, why OpenBSD’s documentation is so good, configure dma for mail delivery in jails on internet hosts, introducing muxfs, RAID1C boot support, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Ten Things To Do After Installing FreeBSD, BSD for Linux users, r2k22 Hackathon Report on rpki-client, Configuring OpenIKED, De-Penguin Me, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nIn this special episode, we are interviewing Mateusz Piotrowski about his various roles in the FreeBSD project, his ports work, and a few other interesting things he’s involved with. Enjoy this interview episode, we’ll be back with a regular episode next week.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Interview
\n\nSpecial Guest: Mateusz Piotrowski.
","summary":"In this special episode, we are interviewing Mateusz Piotrowski about his various roles in the FreeBSD project, his ports work, and a few other interesting things he’s involved with. Enjoy this interview episode, we’ll be back with a regular episode next week. ","date_published":"2022-09-01T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/3f9451dd-059e-44da-9055-d7e119765d55.mp3","mime_type":"audio/mpeg","size_in_bytes":75793536,"duration_in_seconds":3158}]},{"id":"7c5cb6f6-6eba-4430-9347-89e87c1e230b","title":"469: Ctrl-C Reset","url":"https://www.bsdnow.tv/469","content_text":"FreeBSD Q2 2022 Status Report, FreeBSD in Science, fastest yes(1) in the west, Why Programmers Can’t \"Reset\" Programs With Ctrl-C, Run Slack in FreeBSD’s Linuxulator, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nHeadlines\n\nFreeBSD Q2 2022 Status Report\n\n\n\nFreeBSD in Science\n\n\n\nNews Roundup\n\nFastest yes(1) in the west\n\n\n\nCtrl-C: Why Programmers Can’t \"Reset\" Programs With Ctrl-C, but Used to Be Able To, and Why They Should Be Able to Again\n\n\n\nRun Slack in FreeBSD’s Linuxulator\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"FreeBSD Q2 2022 Status Report, FreeBSD in Science, fastest yes(1) in the west, Why Programmers Can’t "Reset" Programs With Ctrl-C, Run Slack in FreeBSD’s Linuxulator, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Advocating for FreeBSD in 2022 and Beyond, NetBSD 9.3 released, OPNsense 22.7 available, CHERI-based computer runs KDE for the first time, Run FreeBSD 13.1-RELEASE for ARM64 in QEMU on Apple Silicon Mac, and more
\n\nNotes
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [In -current, dhclient(8) now just logs warnings and executes ifconfig(8)](http://undeadly.org/cgi?action=article;sid=20220703114819)\n• [Freshly installed #NetBSD 4.0.1 booting on a 80386 DX40 with 8MB of RAM in 2022](https://twitter.com/lefinnois/status/1553246084675375104)\n• [nerdctl](https://twitter.com/woodsb02/status/1554481441060560898?s=28&t=8K7_A1RiWnCDU_Mme4_Yqw)\n• [Even more Randomness](https://undeadly.org/cgi?action=article;sid=20220731110742)\n
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nInstalling BSDs on Cubieboard1, Self-hosting a static site with OpenBSD, httpd, and relayd, NetBSD can also run a Minecraft server, A Little Story About the yes
Unix Command, Shell History: Unix, OpenBGPD 7.5 released, and more
NOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
yes
Unix CommandContributing to Open Source Beyond Software Development, bringing TLS 1.3 to the Internet of Old Things, How efficient can cat(1) be, boost the speed of Unix shell programs, Running FreeBSD VNET Jails on AWS EC2 with Bastille, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [binpa.sh](http://binpa.sh/)\n
\n\nGame of Trees 0.74 released
\nOpenBSD -current has moved to 7.2-beta
\nA Unix Command Line Crash Course
\nBSD.DOG vimrc
\nFreeBSD Speedruns
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nDebugging Lisp in Deep Space, 0 Dependency Websites with OpenBSD & AsciiDoc, Deleting old snapshots on FreeBSD, Full multiprocess support in lldb-server, Basic fix between pf tables and macros, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
From 0 to bhyve on FreeBSD, Analyze OpenBSD’s Kernel with Domain-Specific Knowledge, OpenBSD Webzine: ISSUE #10, HardenedBSD June 2022 Status Report, two new C compilers: chibicc and kefir in OpenBSD, SSD TRIM in NetBSD HEAD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Differences between base and ports LLVM in OpenBSD, Netgraph for FreeBSD’s bhyve Networking, Audio on FreeBSD – Quick Guide, FreeBSD’s Legend starts at 1.0, Hacker News running by FreeBSD, TrueNAS 13, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
The Design and Implementation of the NetBSD rc.d system, selling OpenBSD as a salesperson, Speeding up autoconf with caching, Allowing non-root execution of a jailed application, Configure login(1) and sshd(8) for YubiKey on OpenBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Q1 FreeBSD Quarterly Status Report 2022, Nginx on OpenBSD 7.1, Persistent Memory Allocation, Colorize your BSD shell, cgit With Gitolite and Nginx on FreeBSD 13, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Containerd gains support for launching Linux containers on FreeBSD, OpenBSD 7.1 on PINE64 RockPro64, true minimalistic window manager does not exist, OpenBSD folklore, HardenedBSD May 2022 Status Report, DragonFlyBSD 6.2.2 out, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Evaluating FreeBSD CURRENT for Production Use, Time Machine-like Backups on OpenBSD, FreeBSD on the Graviton 3, Compiling the NetBSD kernel as a benchmark, Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022, Hardware Detection & Diagnostics for New FreeBSD Users, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [NetBSD - Announcing Google Summer of Code 2022 projects](https://blog.netbsd.org/tnf/entry/announcing_google_summer_of_code3)\n• [Welcome FreeBSD Google Summer of Code Participants](https://freebsdfoundation.org/blog/welcome-freebsd-google-summer-of-code-participants/)\n• [Network from Scratch](https://www.networksfromscratch.com)\n
\n\nFundamentals of the FreeBSD Shell, Spammers in the Public Cloud, locking user accounts properly, overgrowth on NetBSD, moreutils, ctwm & spleen, interpreting a traceroute, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
How to properly interpret a traceroute or mtr
\n\nLets talk a bit about some of the events happening this year, BSDCan in virtual this weekend, emfcamp is this weekend too and in person, MCH is this summer and eurobsdcon is in september. How were the postgres conferences benedict?
\n\nJourney to ZFS RAIDZ1 on NetBSD, FreeBSD networking basics: WiFi and Bluetooth, smuggling code into the playstation via NetBSD driver hole, KDE FreeBSD CI, remembering buildtool, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
By the Way... Kubernetes for FreeBSD
\nFreeBSD Games Directory
\nCandlelit Console patch set to the framebuffer console
FreeBSD 13.1 is released, Unix command line conventions over time, Branching for NetBSD 10, Microbhyve, Own your Calendar and Contacts with OpenBSD, the PSARC case for ZFS, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
OpenBSD is the Perfect OS post Nuclear Apocalypse, Multiprocess support for LLDB, porting the new Hare compiler to OpenBSD, Writing my first OpenBSD game using Godot, FreeBSD 13 on Thinkpad T460s, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Open Source Voices interview with Deb Goodkin
\nTachyum Successfully Runs FreeBSD in Prodigy Ecosystem, Expands Open-Source OS Support
\nMidnightBSD Minor Update 2.1.7
\nLibreSSL 3.5.2 Released
\nOpenBGPD 7.3 is out
\nPlaying the game Bottomless on OpenBSD
\nWindows Central: OpenBSD already has a version for Apple Silicon
\nOpenBSD Webzine #9 is out
\nIn the "Everone makes mistakes catagory" : I forgot to enable compression on ZFS
\n"Ken Thompson is a singularity" ~Brian Kernighan
OpenBSD 7.1 is out, Building Your Own FreeBSD-based NAS with ZFS Part 2, Let's try V on OpenBSD, Waiting for Randot, Compiling an OpenBSD kernel 50% faster, A Salute for 10+ years of service, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Building Your Own FreeBSD-based NAS, Writing a device driver for Unix V6, EC2: What Colin Percival’s been up to, Beckhoff releases TwinCAT/BSD Hypervisor, Writing a NetBSD kernel module, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Maxi - question about note taking
\n\nThe unknown hackers, Papers we love to read, Dual Boot Homelab in The Bedroom by the bed testbed, OpenSSH 9.0 released, OS battle: OpenBSD vs. NixOS, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Celebrating 50 years of the Unix Operating System
\nKickstarter Campaign Results
\nFreeBSD Virtualization Series
Full system backups with FFS snapshots, ZFS and dump(8), tuning recordsize in OpenZFS, Optimizing FreeBSD Power Consumption on Modern Intel Laptops, remember to check for ZFS filesystems being mounted, Use tcpdump to save wireless bridge, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [FreeBSD on the Vortex86DX CPU](https://www.cambus.net/freebsd-on-the-vortex86dx-cpu/)\n• [HAMMER2 vs USB stick pulls](https://www.dragonflydigest.com/2022/03/22/26800.html)\n• [New US mirror for DragonFly](https://www.dragonflydigest.com/2022/03/09/26742.html)\n• [HelloSystem 13.1 RC1](https://github.com/helloSystem/ISO/releases/tag/experimental-13.1-RC1)\n• [Video introduction to OpenBSD 7.0](https://www.youtube.com/watch?v=KeUsE-3nSes)\n• [Losses in the community](https://minnie.tuhs.org/pipermail/tuhs/2022-April/025643.html)\n
\n\nThe ideas that made Unix, hints for writing Unix tools, cron best practices, three different sorts of filesystem errors, LibreSSL 3.5.1 released, taskwarrior to manage tasks, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD Status Report 4th Quarter 2021, Reproducible clean $HOME in OpenBSD using impermanence, Making RockPro64 a NetBSD Server, helloSystem 0.7.0 is out, lazy approach to FreeBSD dual-booting, going to jail, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• No Feedback emails this week, so instead we can have “Story Time with Allan” and he can regale us with an entertaining BSD story.\n
\n\nControlling Resource Limits with rctl in FreeBSD, It’s always DNS, Google Summer of Code in BSD Projects, Rsync Technical Notes - Q4 2021, Userland CPU frequency scheduling for OpenBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [Work with FreeBSD in Google Summer of Code](https://freebsdfoundation.org/blog/work-with-freebsd-in-google-summer-of-code/)\n• [The NetBSD Foundation is a mentoring organization at Google Summer of Code 2022](https://blog.netbsd.org/tnf/entry/the_netbsd_foundation_is_a)\n
\n\nEric - periodic notifications
\nKevin - no question
FreeBSD Foundation Proposals, UNIX: On the Path to BSD, Fujitsu ends its mainframe and Unix services, Install burpsuite on FreeBSD using Linuxulator, new OpenBSD Webzine is out, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
No Feedback emails this week, so instead Tom can regale us with an entertaining BSD story.
\n\nRestoring a Tadpole SPARCbook 3, The FreeBSD Boot Process, Debugging an ioctl Problem on OpenBSD, Why my game PC runs FreeBSD and Kubuntu, DNSSEC, Badgers, and Orcs, Oh My, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [LibreSSL 3.5.0 development branch released](https://undeadly.org/cgi?action=article;sid=20220301063844)\n• [OpenSSH updated to 8.9](https://undeadly.org/cgi?action=article;sid=20220301063428)\n• [Recent developments in OpenBSD, 2022-02-21 summary](https://undeadly.org/cgi?action=article;sid=20220221060700)\n
\n\nJonathan - X-Wing and Tie Fighter
\nJoshontech - pool options
Idiot's guide to OpenBSD on the Pinebook Pro, FreeBSD Periodic Scripts, history of service management in Unix, journey from macOS to FreeBSD, Unix processes “infecting” each other, navidrom music server on FreeBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
The History of Berkeley DB, modern inetd in FreeBSD, the Unix argv[0] issue, retrocomputing can be more than games, read section 8 of the Unix users manual, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Certifying an OS Unix compliant, 2021 FreeBSD Foundation Impact Report, Netflix, Disney, and other widevine content on FreeBSD, file hashes updated for NetBSD 8.1, Playing with CD-RWs on FreeBSD, Why "process substitution" is a late feature in Unix shells, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
The Birth of Unix, Help request for three big Lumina items, FreeBSD 13 on Thinkpad T460s, HardenedBSD January 2022 Status Report, OPNsense 22.1 "Observant Owl" released, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Migrating our servers from Linux to FreeBSD, Cluster provisioning with Nomad and Pot on FreeBSD, LibBSDDialog, FreeBSD 13.0 Base Jails with ZFS and VNET, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
GhostBSD 22.01 is available, Packet Scheduling with Dummynet and FreeBSD, Inside zone installation, Why the FreeBSD Desktop and my Linux Rant, How to install Gnome on OpenBSD, The important Unix idea of the "virtual filesystem switch", and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
ACM: It takes a community, Don’t use discord for OSS projects, Unix in a browser tab, OpenIndiana Hipster 2021.10 available, Omni OS CE v11 is out, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
FreeBSD Foundation reviews 2021 activities, DragonflyBSD 6.2.1 is here, Lumina Desktop 1.6.2 available, toolchain adventures, The OpenBSD BASED Challenge Day 7, Bastille Template: AdGuard Home, setting up ZSH on FreeBSD and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• Producers Note: We did get some Christmas AMA questions in after we recorded that episode (since we recorded it early) but don't worry, I’ve made a note of them and we’ll save them for our next AMA episode. \n
\n\n\n\nUsing FreeBSD’s pkg-audit, 20 year old bug that went to Mars, FreeBSD on Slimbook, LLDB FreeBSD kernel core dump support, Steam on OpenBSD, Cool but obscure X11 tools, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\n\nIt's rare that you come across a bug so subtle that it can last for two decades. But, that's exactly what has happened with the Lempel-Ziv-Oberhumer (LZO) algorithm. Initially written in 1994, Markus Oberhumer designed a sophisticated and extremely efficient compression algorithm so elegant and well architected that it outperforms zlib and bzip by four or five times their decompression speed.
\n\nI was impressed to find out that his LZO algorithm has gone to the planet Mars on NASA devices multiple times! Most recently, LZO has touched down on the red planet within the Mars Curiosity Rover, which just celebrated its first martian anniversary on Tuesday.
\n\nIn the past few years, LZO has gained traction in file systems as well. LZO can be used in the Linux kernel within btrfs, squashfs, jffs2, and ubifs. A recent variant of the algorithm, LZ4, is used for compression in ZFS for Solaris, Illumos, and FreeBSD.
\n\nWith its popularity increasing, Lempel-Ziv-Oberhumer has been rewritten by many engineering firms for both closed and open systems. These rewrites, however, have always been based on Oberhumer's core open source implementation. As a result, they all inherited a subtle integer overflow. Even LZ4 has the same exact bug, but changed very slightly.
\n\nBecause the LZO algorithm is considered a library function, each specific implementation must be evaluated for risk, regardless of whether the algorithm used has been patched. Why? We are talking about code that has existed in the wild for two decades. The scope of this algorithm touches everything from embedded microcontrollers on the Mars Rover, mainframe operating systems, modern day desktops, and mobile phones. Engineers that have used LZO must evaluate the use case to identify whether or not the implementation is vulnerable, and in what format.
\n
• [OpenSSH Agent Restriction](http://undeadly.org/cgi?action=article;sid=20211220061017)\n• [OpenBSD’s Clang upgraded to version 13](http://undeadly.org/cgi?action=article;sid=20211220060327)\n• [Cool, but obscure X11 tools](http://cyber.dabamos.de/unix/x11/)\n
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nUNIX Wars, What every IT person needs to know about OpenBSD Part 3, FreeBSD 12.3 is here, TrueNAS 13 begins, what Unix pre-boot envs looked liked, run Unix on Microcontrollers with PDP-11 emulators and more.
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
\n\n• [BSDCan 2022 is a go.](https://www.bsdcan.org/2022/)\n
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nIn this last episode of 2021, we interview Solene from OpenBSD. She’s blogging about her experiences with OpenBSD on dataswamp.org, the webzine she created, how she got involved and other topics. Enjoy and best wishes for 2022!
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
https://dataswamp.org/~solene/2021-07-26-old-computer-challenge-after.html
\n\nSpecial Guest: Solène Rapenne.
","summary":"In this last episode of 2021, we interview Solene from OpenBSD. She’s blogging about her experiences with OpenBSD on dataswamp.org, the webzine she created, how she got involved and other topics. Enjoy and best wishes for 2022! ","date_published":"2021-12-30T03:00:00.000-05:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/96e38cf0-0975-4dd8-8eb3-c7626c45369a.mp3","mime_type":"audio/mpeg","size_in_bytes":20684232,"duration_in_seconds":2031}]},{"id":"40afa2c2-e5e7-4c5c-b505-9c02dcc8953a","title":"434: It’s Quiz-mas time","url":"https://www.bsdnow.tv/434","content_text":"In this special xmas episode we let the audience interview us using questions they sent us and we’ll answer now. Tom, Allan, JT, and I are all here, so stay tuned for some interesting answers to your questions.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon\n\nInterview\n\nAllan - allanjude@freebsd.org / Twitter : @allanjude\n\nBenedict - bcr@freebsd.org / Twitter : @bsdbcr\n\nTom - thj@freebsd.org / Twitter : @adventureloop\n\nJT - jt@obs-sec.com / Twitter : @q5sys\n\n\n\nTarsnap\n\n\nThis week’s episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n***\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"In this special xmas episode we let the audience interview us using questions they sent us and we’ll answer now. Tom, Allan, JT, and I are all here, so stay tuned for some interesting answers to your questions.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
GhostBSD 21.11.24 ISO available, why v7 matters so much, OpenBSD on VIA Eden X2 powered HP t510 Thin Client, OctoPkg GUI Package Manager, chdir(2) support in posix_spawn(3), install doas on FreeBSD, Access Modem's Web Interface with OPNsense, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
No feedback for this episode because no one sent any in. :(
\nI guess we’ve answered every BSD and Unix question that everyone has.
HAMBug hybrid meeting, Demystifying OpenZFS 2.0, OpenZFS 3.0 introduced at Dev Summit, HardenedBSD Home Infrastructure Status, Running Awk in parallel, FreeBSD Announces Wayland 1.19.91, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Why use OpenBSD part 2, FreeBSD on the RISC-V Architecture, OpenBSD Webzine Issue 4, Ending up liking GNOME, OPNsense 21.7.5 released, Jenkins with FreeBSD Agents in EC2, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Manipulate a ZFS pool from Rescue System, FreeBSD 3rd Quarter Report, Monitoring FreeBSD jails form the host, OpenBSD on RPI4 with Full Disk Encryption, Onwards with OpenBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [Manage Kubernetes cluster from FreeBSD with kubectl](https://www.youtube.com/watch?v=iUxJIXKtK7c)\n• [amdgpu support in DragonFly](https://www.dragonflydigest.com/2021/11/08/26343.html)\n• [Today is the 50th Anniversary of the 1st Edition of Unix...](https://twitter.com/bsdimp/status/1456019089466421248?s=20)\n
\n\nFreeBSD Foundation October Fundraising Update, Advanced ZFS Snapshots, Full WireGuard setup with OpenBSD, MidnightBSD a Linux Alternative, FreeBSD Audio, Tuning Power Consumption on FreeBSD Laptops, Thoughts on Spelling Fixes, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
OpenBSD Part 1: How it all started, Explaining top(1) on FreeBSD, Measuring power efficiency of a CPU frequency scheduler on OpenBSD, CultBSD, a whole lot of BSD bits, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
• [OpenBSD on the HiFive Unmatched](https://kernelpanic.life/hardware/hifive-unmatched.html)\n• [Advanced Documentation Retrieval on FreeBSD](https://adventurist.me/posts/00306)\n• [OpenBSD Webzine Issue 3 is out](https://webzine.puffy.cafe/issue-3.html)\n• [How to connect and use Bluetooth headphones on FreeBSD](https://forums.freebsd.org/threads/bluetooth-audio-how-to-connect-and-use-bluetooth-headphones-on-freebsd.82671/)\n• [How To: Execute Firefox in a jail using iocage and ssh/jailme](https://forums.freebsd.org/threads/how-to-execute-firefox-in-a-jail-using-iocage-and-ssh-jailme.53362/)\n• [Understanding AWK](https://earthly.dev/blog/awk-examples/)\n• [“Domesticate Your Badgers” Kickstarter Opens](https://mwl.io/archives/13297)\n• [Bootstrap an OPNsense development environment in Vagrant](https://github.com/punktDe/vagrant-opnsense)\n• [VLANs Bridges and LAG Interface best practice questions](https://www.truenas.com/community/threads/vlans-bridges-and-lag-interface-best-practice-questions.93275/)\n• [A Console Desktop](https://pspodcasting.net/dan/blog/2018/console_desktop.html)\n• [CharmBUG Casual BSD Meetup and Games (Online)](https://www.meetup.com/CharmBUG/events/281822524)\n
\n\nBuild Your FreeBSD Developer Workstation, logging is important, how BSD authentication works, pfSense turns 15 years old, OPNsense Business Edition 21.10 released, getting started with pot, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\nIf you like BSDNow, consider supporting us on Patreon
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nA Good Time to Use OpenZFS Slog, OpenBSD 7.0 is out, OpenBSD and Wayland, UVM faults yield significant performance boost, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
If you like BSDNow, consider supporting us on Patreon
\n\nPLAN 9 DESKTOP GUIDE
\nlibvirt and DragonFly
\nEuroBSDCon 2021 videos are available
\nIssue#1 of OpenBSD Webzine
\nThe Beastie has landed.
\nIt’s 1998 and you are Sun Microsystems...
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nThe New Architecture on the Block, OpenBSD on Vortex86DX CPU, lots of new releases, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
J language working on OpenBSD, Comparing FreeBSD GELI and OpenZFS encrypted pools, What is FreeBSD, actually?, OpenBSD's pledge and unveil from Python, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
• [Hibernate time reduced](http://undeadly.org/cgi?action=article;sid=20210831050932)\n• [(open)rsync gains include/exclude support](http://undeadly.org/cgi?action=article;sid=20210830081715)\n• [Producer JT's latest ancient find that he needs help with](https://twitter.com/q5sys/status/1440105555754848257)\n• [Doas comes to MidnightBSD](https://github.com/slicer69/doas)\n• [FreeBSD SSH Hardening](https://gist.github.com/koobs/e01cf8869484a095605404cd0051eb11)\n• [OpenBSD 6.8 and you](https://home.nuug.no/~peter/openbsd_and_you/#1)\n• [By default, scp(1) now uses SFTP protocol](https://undeadly.org/cgi?action=article;sid=20210910074941)\n• [FreeBSD 11.4 end-of-life](https://lists.freebsd.org/pipermail/freebsd-announce/2021-September/002060.html)\n• [sched_ule(4): Improve long-term load balancer](https://cgit.freebsd.org/src/commit/?id=e745d729be60a47b49eb19c02a6864a747fb2744)\n
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nFreeBSD serves Netflix Video at 400Gb/s, Using the RACK TCP stack, an OpenBSD script to update packages fast, Plasma System Monitor and FreeBSD, TrueNAS vs FreeNAS (and why you should upgrade!), auto lock screen on OpenBSD using xidle and xlock, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
We interview Dr. Brian Callahan about his language porting work for OpenBSD, teaching with BSDs and recruiting students into projects, research, and his work at NYC*BUG in this week’s episode of BSDnow.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Special Guest: Brian Callahan.
","summary":"We interview Dr. Brian Callahan about his language porting work for OpenBSD, teaching with BSDs and recruiting students into projects, research, and his work at NYC*BUG in this week’s episode of BSDnow.","date_published":"2021-09-30T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/4ca5efbc-d83b-41a2-981c-42c4dacefb05.mp3","mime_type":"audio/mpeg","size_in_bytes":30162984,"duration_in_seconds":2999}]},{"id":"626e101a-a6c2-43ce-ad87-018474d78971","title":"421: ZFS eats CPU","url":"https://www.bsdnow.tv/421","content_text":"Useless use of GNU, Meet the 2021 FreeBSD GSoC Students, historical note on Unix portability, vm86-based venix emulator, ZFS Mysteriously Eating CPU, traceroute gets speed boost, and more \n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nUseless use of GNU\n\n\n\nMeet the 2021 FreeBSD Google Summer of Code Students\n\n\n\nNews Roundup\n\nLarge Unix programs were historically not all that portable between Unixes\n\n\nReferences this article: I’m not sure that UNIX won\n***\n### A new path: vm86-based venix emulator\n***\n### ZFS Is Mysteriously Eating My CPU\n***\n### traceroute(8) gets speed boost\n***\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nAl - TransAtlantic Cables\nChristopher - NVMe\nJohnnyK - Vivaldi\n***\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"Useless use of GNU, Meet the 2021 FreeBSD GSoC Students, historical note on Unix portability, vm86-based venix emulator, ZFS Mysteriously Eating CPU, traceroute gets speed boost, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Choosing The Right ZFS Pool Layout, changes in OpenBSD that make life better, GhostBSD 21.09.06 ISO's now available, Fair Internet bandwidth management with OpenBSD, NetBSD wifi router project update, NetBSD on the Apple M1, HardenedBSD August Status Report, FreeBSD Journal on Wireless and Desktop, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Reviewing a first OpenBSD port, NetBSD 9.2 on a DEC Alpha CPU in QEMU with X11, FreeBSD Experiment Rethinks the OS Install, GhostBSD switching to FreeBSD rc.d, Irix gets LLVM, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
In this episode, we interview Michael W. Lucas about his latest book projects including Git sync murder, TLS Mastery, getting paid for creative work, writing tools and techniques, and more.
\n\nNOTES
\n\nSpecial Guest: Michael W Lucas.
","summary":"In this episode, we interview Michael W. Lucas about his latest book projects including Git sync murder, TLS Mastery, getting paid for creative work, writing tools and techniques, and more.","date_published":"2021-09-02T03:45:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/5b0aa0e0-4435-47d3-841a-91793cf37806.mp3","mime_type":"audio/mpeg","size_in_bytes":32985120,"duration_in_seconds":3145}]},{"id":"63b2639c-ad67-45db-9581-8053963313c2","title":"417: bhyve private cloud","url":"https://www.bsdnow.tv/417","content_text":"Achieving RPO/RTO Objectives with ZFS pt 1, FreeBSD Foundation Q2 report, OpenBSD full Tor setup, MyBee - bhyve as private cloud, FreeBSD home fileserver expansion, OpenBSD on Framework Laptop, portable GELI, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nAchieving RPO/RTO Objectives with ZFS - Part 1\n\n\n\nFreeBSD Foundation Q2 Report\n\n\n\nOpenBSD full Tor setup\n\n\n\nNews Roundup\n\nMyBee — FreeBSD OS and hypervisor bhyve as private cloud\n\n\n\nExpanding our FreeBSD home file server\n\n\n\nOpenBSD on the Framework Laptop\n\n\n\nPortable GELI\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nChunky_pie - zfs question\nPaul - several questions\nchris - firewall question\n***\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"Achieving RPO/RTO Objectives with ZFS pt 1, FreeBSD Foundation Q2 report, OpenBSD full Tor setup, MyBee - bhyve as private cloud, FreeBSD home fileserver expansion, OpenBSD on Framework Laptop, portable GELI, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
OpenZFS snapshots, OpenSUSE on Bastille, printing with netcat, new opnsense 21.1.8 released, new pfsense plus software available, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
• [MAC Inspired FreeBSD release](https://github.com/mszoek/airyx)\n• [Implement unprivileged chroot](https://cgit.freebsd.org/src/commit/?id=a40cf4175c90142442d0c6515f6c83956336699b)\n• [InitWare: A systemd fork that runs on BSD](https://github.com/InitWare/InitWare)\n• [multics gets a new release](https://multics-wiki.swenson.org/index.php/Main_Page)\n• [Open Source Voices interview with Tom Jones](https://www.opensourcevoices.org/17)\n• [PDP 11/03 Engineering Drawings](https://twitter.com/q5sys/status/1423092689084551171)\n
\n\nWrong Way to Switch Server OS, Net/1 and Net/2 – A Path to Freedom, Permissions Two Mistakes, OpenBSD progress in supporting riscv64 platform, I2P intro, git sync murder is out, GhostBSD init system poll, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
OpenZFS 2.1 is out, FreeBSD TCP Performance System Controls, IPFS OpenBSD, tips for running an online conference, fanless OpenBSD laptop, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Updating GCC GNAT (Ada) in pkgsrc/NetBSD, AdvanceBSD thoughts 2/2, FreeBSD from a NetBSD user’s perspective, FPGA programming and DragonFly, Chimera Linux, EuroBSDcon 2021, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
FreeBSD Performance Observability, Advance!BSD thoughts 1/2, Lumina Desktop Maintainership Change, How to Handle Secrets on the Command Line, Like NetBSD DragonFlyBSD Now Has "COVID", and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Unix System Architecture Evolution, Deep Dive into FreeBSD’s Strengths, how developers chose names, OPNsense 21.1.7 released, Support for chdir(2) in posix_spawn(3), vagrant-freebsd-boxbuilder, OpenBSD’s IATA airport code file, and more
\n\nThis episode of BSDNow is brought to you by Tarsnap
\n\n• Full IEEE article: https://ieeexplore.ieee.org/document/8704965\n
\n\nOpen Source and Blogging Bubbles, Building Customized FreeBSD Images, Updating Minecraft in FreeBSD, Upgrading FreeBSD jails using mkjail, Dragonfly 6.0 Performance benchmark, OpenBSD Consumer Gateway Launch, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
DTrace network probes, next 50 years of shell programming, NetBSD on the Vortex86DX CPU, system CPU time in top, your filesystem as a dungeon, diving into toolchains, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
• [Alfred - Advice](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/409/feedback/Alfred%20-%20Advice)\n• [CY - Portable Patch Util](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/409/feedback/CY%20-%20Portable%20Patch%20Util)\n• [Denis - State of ZFS Ecosystem](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/409/feedback/Denis%20-%20State%20of%20ZFS%20Ecosystem)\n
\n\nReport from virtual FreeBSD DevSummit 2021, another promising release by FreeBSD Based helloSystem, GearBSD, OpenBGPD release, Let’s Encrypt on OpenBSD, FreeBSD 13 on the Panasonic Let’s Note, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
• [Paul - ZFS Questions](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/408/feedback/Paul%20-%20ZFS%20Questions)\n• [Rafael - relic](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/408/feedback/Rafael%20-%20relic)\n• [matthew - sendfile and ktls](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/408/feedback/matthew%20-%20sendfile%20and%20ktls)\n
\n\nConfining the omnipotent root, Jails with ZFS and PF on DigitalOcean, NomadBSD 130R is out, KDE Plasma Wayland on FreeBSD, Firefox under FreeBSD with Privacy, Using NetBSD’s pkgsrc everywhere, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Gemini Capsule in a FreeBSD Jail, FreeBSD Quarterly status report 2021Q1, NetBSD VM on bhyve (on TrueNAS), Interview with Michael Lucas, WireGuard Returns as Experimental Package in pfSense, CGI with Awk on OpenBSD httpd, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nWith the recent release of FreeBSD 13, I wanted to test it out on a spare RaspberryPi 3 that was part of my old Kubernetes cluster.
\n
\nIn particular, FreeBSD Jails have always interested me, although I’ve never used them in practice. Over the years I’ve managed operating system virtualization through Solaris Zones and Docker containers, and Jails seem like and good middle ground between the two - easier to manage than zones and closer to the OS than Docker.
\nI also want to run my own Gemini capsule locally to use some of the features that my other hosted capsules don’t have (like SCGI/CGI) and setting up a capsule in a Jail is a good way to learn both at the same time.
\n\n\nMy new NAS at home is running TrueNAS Core. So far, it has been excellent, however I struggled a bit setting up a NetBSD VM on it. Part of the problem is that a lot of the docs and how-tos I found are stale, and the information in it no longer applies.
\n
\nTrueNAS Core allows running VMs using bhyve, which is FreeBSD’s hypervisor. NetBSD is not an officially supported OS, at least according to the guest OS chooser in the TrueNAS web UI :) But since the release of NetBSD 9 a while ago, things have become far simpler than they used to be – with one caveat (see below).
\n\n\nMichael Lucas is a famous IT book author. Perhaps best know for FreeBSD, OpenBSD, and Unix book series. He worked as a system administrator for many years and has now become a full-time book writer. Lately, I did a quick Q and A with Michael about his journey as a professional book author and his daily workflow for writing books.
\n\n
\n+
\n\npfSense – WireGuard Returns as Experimental Package
\n\n
\n\nCGI with Awk on OpenBSD httpd
\n\n
\n
NetBSD 9.2 released, DragonFly 6.0 is out, Home Network Monitoring using Prometheus, Preventing FreeBSD to kill PostgreSQL, Customizing Emacs for Git Commit Messages, Deleting old FreeBSD boot environments, Always be quitting, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nA good philosophy to live by at work is to “always be quitting”. No, don’t be constantly thinking of leaving your job. But act as if you might leave on short notice. Counterintuitively, this will make you a better engineer and open up growth opportunities.
\n\n
\n
Allan, Benedict and Tom are MIA, so JT fills in with two friends.
\n\nThis episode of BSDNow is brought to you by Tarsnap
\nCoHosts this week:
\n • Ash Gokhale: https://twitter.com/xpi
\n • Jeff Propes : CoHost of The Opinion Dominion
• Interesting dtrace papers I found this week. The first is unfortunately paywalled by an industry journal but hopefully it’ll be publicly available soon. \n ◦ [Using Dtrace for Machine Learning Solutions in Malware Detection](https://ieeexplore.ieee.org/document/9225633)\n ◦ [Process Monitoring on Sequences of System Call Count Vectors](https://arxiv.org/pdf/1707.03821.pdf)\n ◦ Sounds Similar to:\n
\n\n• https://www.nytimes.com/2021/04/27/technology/daniel-kaminsky-dead.html\n• https://www.darkreading.com/vulnerabilities---threats/in-appreciation-dan-kaminsky/d/d-id/1340830\n• https://www.securityweek.com/security-researcher-dan-kaminsky-passes-away\n
\n\nWhy You Should Use BSD Licensing for Your Next Open Source Project or Product, Update on FreeBSD Foundation Investment in Linuxulator, OPNsense 21.1.5 released, FreeBSD meetings on the Desktop, Running FreeBSD jails with containerd 1.5, Markdown, DocBook, and the quest for semantic documentation on NetBSD.org, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThe term “open source” has its origins in the context of software development, designating a specific approach to developing computer programs. Nowadays, however, it stands for a broad set of values – open source means open exchange, transparency, collaborative participation and development for the benefit of the entire community.
\n\n
\n\nUpdate on FreeBSD Foundation Investment in Linuxulator
\n\nDr. Emmett Brown’s similar-sounding Flux Capacitor from the movie Back to the Future bridged the dimension of time, uniting past, present, and future for the McFlys. Similarly, the FreeBSDⓇ Linuxulator project also bridges dimensions – in our case, these are LinuxⓇ and FreeBSD.
\n\n
\n\nNews Roundup
\n\nOPNsense 21.1.5 released
\n\nThis is mainly a security and reliablility update. There are several FreeBSD
\n\n
\nsecurity advisories and updates for third party tools such as curl.\n
\n- OPNsense to rebase on FreeBSD 13\n***\n### FreeBSD meetings on the Desktop\nFreeBSD on the desktop is a whole stack - X11, Qt, KDE Frameworks, KDE Plasma and KDE Gear, and Wayland, and Poppler and GTK - o my!\n***\n### Running FreeBSD jails with containerd 1.5\ncontainerd 1.5.0 was released today and now works on a new operating system: FreeBSD! This new release includes a series of patches (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) which allow containerd to build, enable the native and zfs snapshotters, and use a compatible runtime like runj.\n***\n### Markdown, DocBook, and the quest for semantic documentation on NetBSD.org\nRecently, I’ve been doing a lot of maintenance of the NetBSD website. It contains a boatload of documentation, much of which was originally written in the 2000s. It has some special requirements: it has to work in text-based web browsers like lynx, or maybe even without any working browser installed at all, or just ftp(1) for downloading plain text over HTTP. Naturally, the most important parts are static, suitable for serving from the standard NetBSD http server, which runs from inetd by default.\n***
\n
It's time to say goodbye to the GPL, a new OCI Runtime for FreeBSD Jails, A bit of Xenix history, On Updating QEMU's bsd-user fork, FreeBSD 13 on a 12 year old laptop, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThe trigger for this post is the reinstating of Richard Stallman, a very problematic character, to the board of the Free Software Foundation (FSF). I am appalled by this move, and join others in the call for his removal.
\n\n
\nThis occasion has caused me to reevaluate the position of the FSF in computing. It is the steward of the GNU project (a part of Linux distributions, loosely speaking), and of a family of software licenses centred around the GNU General Public License (GPL). These efforts are unfortunately tainted by Stallman’s behaviour. However, this is not what I actually want to talk about today.
\n\nrunj: a new OCI Runtime for FreeBSD Jails
\n\nToday, I open-sourced runj, a new experimental, proof-of-concept OCI-compatible runtime for FreeBSD jails. For the past 6.5 years I’ve been working on Linux containers, but never really had much experience with FreeBSD jails. runj (pronounced “run jay”) is a vehicle for me to learn more about FreeBSD in general and jails in particular. With my position on the Technical Oversight Board of the Open Containers Initiative, I’m also interested in understanding how the OCI runtime specification can be adapted to other operating systems like FreeBSD.
\n\n
\n\nNews Roundup
\n\nA Bit of Xenix History
\n\nFrom 1986 to 1989, I worked in the Xenix1 group at Microsoft. It was my first job out of school, and I was the most junior person on the team. I was hopelessly naive, inexperienced, generally clueless, and borderline incompetent, but my coworkers were kind, supportive and enormously forgiving – just a lovely bunch of folks.
\n\n
\n\nOn Updating QEMU's bsd-user fork
\n\n
\n
\n\n\nMy old (2009) HP laptop now runs FreeBSD 13.0-RELEASE.
\n\n
\n
Dog's Garage Runs OpenBSD, EuroBSDcon 2021 Call for Papers, FreeBSD’s iostat, The state of toolchains in NetBSD, Bandwidth limiting on OpenBSD 6.8, FreeBSD's ports migration to git and its impact on HardenedBSD, TrueNAS 12.0-U3 has been released, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI was inspired by the April 2017 article in undeadly.org about getting OpenBSD running on a Raspberry Pi 3B+. My goal was to use a Raspberry Pi running OpenBSD to monitor the temperature in my garage from my home. My dog has his own little "apartment" inside the garage, so I want to keep an eye on the temperature. (I don't rely on this device. He sleeps inside the house whenever he wants.)
\n\n
\n
\n\n\nWhile FreeBSD and OpenBSD both switched to using LLVM/Clang as their base system compiler, NetBSD picked a different path and remained with GCC and binutils regardless of the license change to GPLv3. However, it doesn't mean that the NetBSD project endorses this license, and the NetBSD Foundation's has issued a statement about its position on the subject.
\n
\n\n\nI will explain how to limit bandwidth on OpenBSD using its firewall PF (Packet Filter) queuing capability. It is a very powerful feature but it may be hard to understand at first. What is very important to understand is that it's technically not possible to limit the bandwidth of the whole system, because once data is getting on your network interface, it's already there and got by your router, what is possible is to limit the upload rate to cap the download rate.
\n\n
\n\nFreeBSD's ports migration to git and its impact on HardenedBSD
\n\nFreeBSD completed their ports migration from subversion to git. Prior to the official switch, we used the read-only mirror FreeBSD had at GitHub[1]. The new repo is at [2]. A cursory glance at the new repo will show that the commit hashes changed. This presents an issue with HardenedBSD's ports tree in our merge-based workflow.
\n\n
\n
\n\n\niXsystems is excited to announce TrueNAS 12.0-U3 was released today and marks an important milestone in the transition from FreeNAS to TrueNAS. TrueNAS 12.0 is now considered by iXsystems to be a higher quality release than FreeNAS 11.3-U5, our previous benchmark. The new TrueNAS documentation site has also reached a point where it has more content and capabilities than FreeNAS. TrueNAS 12.0 is ready for mission-critical enterprise deployments.
\n\n
\n
FreeBSD 13 is here, multi-factor authentication on OpenBSD, KDE on FreeBSD 2021o2, NetBSD GSoC report, a working D compiler on OpenBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
• OpenZFS 2.0 (almost 2.1) is included in 13.0\n• Removed support for previously-deprecated algorithms in geli(8).\n• The armv8crypto(4) driver now supports AES-GCM which is used by IPsec and kernel TLS.\n
\n\n\n\n\nIn this article I will explain how to add a bit more security to your OpenBSD system by adding a requirement for user logging into the system, locally or by ssh. I will explain how to setup 2 factor authentication (2FA) using TOTP on OpenBSD
\n
\n\n\nGosh, second octant already! Well, let’s take a look at the big things that happened in KDE-on-FreeBSD in these six-and-a-half weeks.
\n
\n\n\nMy code can be found at github.com/teknokatze/src in the gsoc2020 branch, at the time of writing some of it is still missing. The test facilities and logs can be found in github.com/teknokatze/gsoc2020. A diff can be found at github which will later be split into several patches before it is sent to QA for merging.
\n\n
\nThe initial and defined goal of this project was to make system(3) and popen(3) use posix_spawn(3) internally, which had been completed in June. For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determined through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posix_spawn() in this case should be avoided. This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals.
\n
\n\n\nDr. Brian Robert Callahan (bcallah@) blogged about his work in getting D compiler(s) working under OpenBSD.
\n\n\n
\n- Full Post\n***
\n
Comparing sandboxing techniques, Statement on FreeBSD development processes, customizing FreeBSD ports and packages, the quest for a comfortable NetBSD desktop, Nginx as a TCP/UDP relay, HardenedBSD March 2021 Status Report, Detailed Behaviors of Unix Signal, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI had the opportunity to implement a sandbox and I'd like to write about the differences between the various sandboxing techniques available on three different operating systems: FreeBSD, Linux and OpenBSD.
\n\n
\n\nStatement on FreeBSD development processes
\n\nIn light of the recent commentary on FreeBSD's development practices, members of the Core team would like to issue the following statement.
\n\n
\n\nCustomizing FreeBSD Ports and Packages
\n\nA basic intro to building your own packages
\n\n
\n
\n\n\nFVWM substantially allows one to build a fully-fledged lightweight desktop environment from scratch, with an almost unparalleled degree of freedom. Although using FVWM does not require any knowledge of programming languages, it is possible to extend it with M4, C, and Perl preprocessing.
\n\n
\n\nNginx as a TCP/UDP relay
\n\nIn this tutorial I will explain how to use Nginx as a TCP or UDP relay as an alternative to Haproxy or Relayd. This mean nginx will be able to accept requests on a port (TCP/UDP) and relay it to another backend without knowing about the content. It also permits to negociates a TLS session with the client and relay to a non-TLS backend. In this example I will explain how to configure Nginx to accept TLS requests to transmit it to my Gemini server Vger, Gemini protocol has TLS as a requirement.
\n\n
\n\nHardenedBSD March 2021 Status Report
\n\nThis month, I worked on finding and fixing the regression that caused kernel panics on our package builders. I think I found the issue: I made it so that the HARDENEDBSD amd64 kernel just included GENERIC so that we follow FreeBSD's toggling of features. Doing so added QUEUE_MACRO_DEBUG_TRASH to our kernel config. That option is the likely culprit. If the next package build (with the option removed) completes, I will commit the change that removes QUEUE_MACRO_DEBUG_TRASH from the HARDENEDBSD amd64 kernel.
\n\n
\n\nDetailed Behaviors of Unix Signal
\n\nWhen Unix is mentioned in this document it means macOS or Linux as they are the mainly used Unix at this moment. When shell is mentioned it means Bash or Zsh. Most demos are written in C for macOS with Apple libc and Linux with glibc.
\n\n
\n\nTarsnap
\n\n\n
\n- This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nFreeBSD 13.0 Full Desktop Experience, FreeBSD on ARM64 in the Cloud, Plan 9 from Bell Labs in Cyberspace, Inferno is open source as well, NetBSD hits donation milestone, grep returns (standard input) on FreeBSD, Random Programming Challenge, OpenBSD Adds Support for Coordinated Mars Time (MTC) and more
\n\nNOTES
\n\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\n\n\nWith the release of FreeBSD 13.0 on the horizon, I wanted to see how it shapes up on my Lenovo T450 laptop. Previous major releases on this laptop, using it as a workstation, felt very rough around the edges but with 13, it feels like the developers got it right.
\n\n
\n\nFreeBSD on ARM64 in the Cloud
\n\nUntil the end of June, Amazon AWS is offering free ARM64 Graviton instances, learn how to try out FreeBSD to ARMv8 in the cloud
\n\n
\n
\n\n\nThe releases below represent the historical releases of Plan 9. The two versions of 4th Edition represent the initial release and the final version available from Bell Labs as it was updated and patched. All historical releases of Plan 9 have been re-released under the terms of the MIT license.
\n\n\n
\n- Inferno is open source as well\n***\n## News Roundup\n### Hitting donation milestone, financial report for 2020\nWe nearly hit our 2020 donation milestone set after the release of 9.0 of $50,000.\n***
\n
\n\n\nI was dealing with a bizarre error with grep(1) on FreeBSD, and it soon infected my macOS and NetBSD machines too. It was driving me crazy!
\n\n
\n\nRandom Programming Challenge
\n\n
\n\nThis better not be an April Fools Joke… I want to see this actually implemented. I’ll donate $100 to the first BSD that actually implements this for real. Who’s with me?
\n
OpenBSD Adds Support for Coordinated Mars Time (MTC)
\n\n\n\n\nTo make sure that OpenBSD can be used elsewhere than just earth, this diff introduces Coordinated Mars Time (MTC), the Mars equivalent of earth’s Universal Time (UTC).
\n
\nOpenZFS had a good one too
Customizing the FreeBSD Kernel, OpenBSD/loongson on the Lemote Fuloong, how ZFS on Linux brings up pools and filesystems at boot under systemd, LLDB: FreeBSD Legacy Process Plugin Removed, FreshBSD 2021, gmid, Danschmid’s Poudriere Guide in english, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nLearn more about customizing the build of the FreeBSD kernel and its loadable modules
\n\n
\n\nOpenBSD/loongson on the Lemote Fuloong
\n\nIn my article about running OpenBSD/loongson on the Lemote Yeeloong back in 2016, I mentioned looking for a Fuloong. All hope seemed lost until the Summer of 2017, when a fellow OpenBSD developer was contacted by a generous user (Thanks again, Lars!) offering to donate two Lemote Fuloong machines, and I was lucky enough to get one of those units.
\n
\n\n\nOn Solaris and Illumos, how ZFS pools and filesystems were brought up at boot was always a partial mystery to me (and it seemed to involve the kernel knowing a lot about /etc/zfs/zpool.cache). On Linux, additional software RAID arrays are brought up mostly through udev rules, which has its own complications. For a long time I had the general impression that ZFS on Linux also worked through udev rules to recognize vdev components, much like software RAID. However, this turns out to not be the case and the modern ZFS on Linux boot process is quite straightforward on systemd systems.
\n\n
\n\nLLDB: FreeBSD Legacy Process Plugin Removed
\n\nDuring the past month we’ve successfully removed the legacy FreeBSD plugin and continued improving the new one. We have prepared an implementation of hardware breakpoint and watchpoint support for FreeBSD/AArch64, and iterated over all tests that currently fail on that platform. Therefore, we have concluded the second milestone.
\n\n
\n\nFreshBSD 2021
\n\n6 weeks ago I created a branch for a significant rework of FreshBSD. Nearly 300 commits later, and just a week shy of our 15th anniversary, the result is what you’re looking at now. I hope you like it.
\n\n
\n\ngmid is a gemini server for unixes.
\n\n
\n\nDanschmid’s Poudriere Guide now in english
\n\nThe ports system is one of FreeBSD's greatest advantages for users who want flexibility and control over their software. It enables administrators to easily create and manage source-based installations using a system that is robust and predictable.
\n\n
\n\nTarsnap
\n\n\n
\n- This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n
Special Guest: Tom Jones.
","summary":"Customizing the FreeBSD Kernel, OpenBSD/loongson on the Lemote Fuloong, how ZFS on Linux brings up pools and filesystems at boot under systemd, LLDB: FreeBSD Legacy Process Plugin Removed, FreshBSD 2021, gmid, Danschmid’s Poudriere Guide in english, and more","date_published":"2021-04-08T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/c901a741-a25b-4d92-9ce4-03b5f2e18d2f.mp3","mime_type":"audio/mpeg","size_in_bytes":34526808,"duration_in_seconds":3361}]},{"id":"db1ced31-e2bc-41f2-baca-041c750229f4","title":"396: License to thrill","url":"https://www.bsdnow.tv/396","content_text":"FreeBSD Network Troubleshooting, The State of FreeBSD, dhcpleased, bhyve for Calamares Development, EFS automount and ebsnvme-id, Old Usenix pictures, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nFreeBSD Network Troubleshooting\n\n\nFreeBSD has a full set of debugging features, and the network stack is able to report a ton of information. So much that it can be hard to figure out what is relevant and what is not.\n\n\n\nThe State of FreeBSD\n\nLicense to thrill: Ahead of v13.0, the FreeBSD team talks about Linux and the completed toolchain project that changes everything\n\n\n\n\nNews Roundup\n\ndhcpleased(8) - DHCP client daemon\n\n\nWith the following commit, Florian Obser (florian@) imported dhcpleased(8), DHCP daemon to acquire IPv4 address leases from servers, plus dhcpleasectl(8), a utility to control the daemon:\n\n\n\nbhyve for Calamares Development\n\nbhyve (pronounced “bee hive”) is a hypervisor for BSD systems (and Illumos / openSolaris). It is geared towards server workloads, but does support desktop-oriented operation as well. I spent some time wayyyy back in November wrestling with it in order to replace VirtualBox for Calamares testing on FreeBSD. The “golden hint” as far as I’m concerned came from Karen Bruner and now I have a functioning Calamares test-ground that is more useful than before.\n“Calamares is a free and open-source independent and distro-agnostic system installer for Linux distributions.“\n\n\n\nSome new FreeBSD/EC2 features: EFS automount and ebsnvme-id\n\nAs my regular readers will be aware, I've been working on and gradually improving FreeBSD/EC2 for many years. Recently I've added two new features, which are available in the weekly HEAD and 12-STABLE snapshots and will appear in releases starting from 12.2-RELEASE.\n\n\n\nOld Usenix pictures\n\n\n\nBeastie Bits\n\n[https://2021.eurobsdcon.org/](CFP is open until May 26th, 2021)\n\nEuroBSDcon is the European technical conference for users and developers of BSD-based systems. The conference is scheduled to take place September 16-19 2021 in Vienna, Austria or as an all-online event if COVID-19 developments dictate. The tutorials will be held on Thursday and Friday to registered participants and the talks are presented to conference attendees on Saturday and Sunday.\nThe Call for Talk and Presentation proposals period will close on May 26th, 2021. Prospective speakers will be notified of acceptance or otherwise by June 1st, 2021.\n\n\n\n[https://campgnd.com/](CFP is open until 2021-04-15)\n\ncampgndd will be held May 28th, 29th and 30th 2021, from wherever you happen to be.\nWe're looking for submissions on anything you're enthusiastic and excited about. If you enjoy it, the odds are we will too! You don't need to be an expert to propose anything.\nSome example of things we are looking for are:\n Talks\n Walkthroughs\n Music\n\nFrom the Desk of Michael Lucas…\n\nNew Release: Only Footnotes\nI’ve lost count of the number of people who have told me that they purchase my books only for the footnotes. That’s okay. I don’t care why people buy my books, only that they do buy them. Nevertheless, I am a businessman living under capitalism and feel compelled to respond to my market.\nAllow me to present my latest release: Only Footnotes, a handsome hardcover-only compilation of decades of footnotes. From the back cover:\n-----\nOnly Footnotes. Because that’s why you read his books.\nAcademics hate footnotes. Michael W Lucas loves them. What he does with them wouldn’t pass academic muster, but that doesn’t mean the reader should skip them. The footnotes are the best part! Why not read only the footnotes, and skip all that other junk?\nAfter literal minutes of effort, Only Footnotes collects every single footnote from all of Lucas’ books to date.* Recycle those cumbersome treatises stuffed with irrelevant facts! No more flipping through pages and pages of actual technical knowledge looking for the offhand movie reference or half-formed joke. This slender, elegant volume contains everything the man ever passed off as his dubious, malformed “wisdom.”\nSmart books have footnotes. Smarter books are only footnotes.\n*plus additional annotations from the author. Because sometimes even a footnote needs a footnote.\n----\nWith interior illustrations by OpenBSD’s akoshibe, this distinguished tome would make fine inspirational reading for a system administrator, network engineer, or anyone sentenced to a life in information technology. Available at all fine bookstores, and many mediocre ones!\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n\nSpecial Guest: Tom Jones.","content_html":"FreeBSD Network Troubleshooting, The State of FreeBSD, dhcpleased, bhyve for Calamares Development, EFS automount and ebsnvme-id, Old Usenix pictures, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nFreeBSD has a full set of debugging features, and the network stack is able to report a ton of information. So much that it can be hard to figure out what is relevant and what is not.
\n\n
\n\nThe State of FreeBSD
\n\nLicense to thrill: Ahead of v13.0, the FreeBSD team talks about Linux and the completed toolchain project that changes everything
\n
\nWith the following commit, Florian Obser (florian@) imported dhcpleased(8), DHCP daemon to acquire IPv4 address leases from servers, plus dhcpleasectl(8), a utility to control the daemon:
\n\n
\n\nbhyve for Calamares Development
\n\nbhyve (pronounced “bee hive”) is a hypervisor for BSD systems (and Illumos / openSolaris). It is geared towards server workloads, but does support desktop-oriented operation as well. I spent some time wayyyy back in November wrestling with it in order to replace VirtualBox for Calamares testing on FreeBSD. The “golden hint” as far as I’m concerned came from Karen Bruner and now I have a functioning Calamares test-ground that is more useful than before.
\n\n
\n“Calamares is a free and open-source independent and distro-agnostic system installer for Linux distributions.“
\n\nSome new FreeBSD/EC2 features: EFS automount and ebsnvme-id
\n\nAs my regular readers will be aware, I've been working on and gradually improving FreeBSD/EC2 for many years. Recently I've added two new features, which are available in the weekly HEAD and 12-STABLE snapshots and will appear in releases starting from 12.2-RELEASE.
\n\n
\n\nOld Usenix pictures
\n\n
\n\nBeastie Bits
\n\n[https://2021.eurobsdcon.org/](CFP is open until May 26th, 2021)
\n\nEuroBSDcon is the European technical conference for users and developers of BSD-based systems. The conference is scheduled to take place September 16-19 2021 in Vienna, Austria or as an all-online event if COVID-19 developments dictate. The tutorials will be held on Thursday and Friday to registered participants and the talks are presented to conference attendees on Saturday and Sunday.
\n\n
\nThe Call for Talk and Presentation proposals period will close on May 26th, 2021. Prospective speakers will be notified of acceptance or otherwise by June 1st, 2021.
\n\n[https://campgnd.com/](CFP is open until 2021-04-15)
\n\ncampgndd will be held May 28th, 29th and 30th 2021, from wherever you happen to be.
\n\n
\nWe're looking for submissions on anything you're enthusiastic and excited about. If you enjoy it, the odds are we will too! You don't need to be an expert to propose anything.
\nSome example of things we are looking for are:
\n Talks
\n Walkthroughs
\n MusicFrom the Desk of Michael Lucas…
\n\n\n\nNew Release: Only Footnotes\nI’ve lost count of the number of people who have told me that they purchase my books only for the footnotes. That’s okay. I don’t care why people buy my books, only that they do buy them. Nevertheless, I am a businessman living under capitalism and feel compelled to respond to my market.\nAllow me to present my latest release: Only Footnotes, a handsome hardcover-only compilation of decades of footnotes. From the back cover:\n-----\nOnly Footnotes. Because that’s why you read his books.\nAcademics hate footnotes. Michael W Lucas loves them. What he does with them wouldn’t pass academic muster, but that doesn’t mean the reader should skip them. The footnotes are the best part! Why not read only the footnotes, and skip all that other junk?\nAfter literal minutes of effort, Only Footnotes collects every single footnote from all of Lucas’ books to date.* Recycle those cumbersome treatises stuffed with irrelevant facts! No more flipping through pages and pages of actual technical knowledge looking for the offhand movie reference or half-formed joke. This slender, elegant volume contains everything the man ever passed off as his dubious, malformed “wisdom.”\nSmart books have footnotes. Smarter books are only footnotes.\n*plus additional annotations from the author. Because sometimes even a footnote needs a footnote.\n----\nWith interior illustrations by OpenBSD’s akoshibe, this distinguished tome would make fine inspirational reading for a system administrator, network engineer, or anyone sentenced to a life in information technology. Available at all fine bookstores, and many mediocre ones!\n
Tarsnap
\n\n\n
\n- This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***
\n
Special Guest: Tom Jones.
","summary":"Description: FreeBSD Network Troubleshooting, The State of FreeBSD, dhcpleased, bhyve for Calamares Development, EFS automount and ebsnvme-id, Old Usenix pictures, and more.","date_published":"2021-04-01T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/db1ced31-e2bc-41f2-baca-041c750229f4.mp3","mime_type":"audio/mpeg","size_in_bytes":30506976,"duration_in_seconds":3207}]},{"id":"9e4b924f-7f9c-49b4-81b7-b28ade7904b3","title":"395: Tracing ARM’s history","url":"https://www.bsdnow.tv/395","content_text":"Tracing the History of ARM and FreeBSD, Make ‘less’ more friendly, NomadBSD 1.4 Release, Create an Ubuntu Linux jail on FreeBSD 12.2, OPNsense 21.1.2 released, Midnight BSD and BastilleBSD, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nTracing the History of ARM and FreeBSD\n\n\nWhen we think of computers, we generally think of laptops and desktops. Each one of these systems is powered by an Intel or AMD chip based on the x86 architecture. It might feel like you spend all day interacting with these kinds of systems, but you would be wrong.\n\n\n\nUnix Tip: Make ‘less’ more friendly\n\nYou probably know about less: it is a standard tool that allows scrolling up and down in documents that do not fit on a single screen. Less has a very handy feature, which can be turned on by invoking it with the -i flag. This causes less to ignore case when searching. For example, ‘udf’ will find ‘udf’, ‘UDF’, ‘UdF’, and any other combination of upper-case and lower-case. If you’re used to searching in a web browser, this is probably what you want. But less is even more clever than that. If your search pattern contains upper-case letters, the ignore-case feature will be disabled. So if you’re looking for ‘QXml’, you will not be bothered by matches for the lower-case ‘qxml’. (This is equivalent to ignorecase + smartcase in vim.)\n\n\n\n\nNews Roundup\n\nNomadBSD 1.4 Release\n\n\nVersion 1.4 of NomadBSD, a persistent live system for USB flash drives based on FreeBSD and featuring a graphical user interface built around Openbox, has been released: “We are pleased to present the release of NomadBSD 1.4.\n\n\n\nCreate an Ubuntu Linux jail on FreeBSD 12.2\n\n\n\nOPNsense 21.1.2 released\n\nWork has so far been focused on the firmware update process to ensure its safety around edge cases and recovery methods for the worst case. To that end 21.1.3 will likely receive the full revamp including API and GUI changes for a swift transition after thorough testing of the changes now available in the development package of this release.\n\n\n\nMidnight BSD and BastilleBSD\n\nWe recently added a new port, mports/sysutils/bastille that allows you to manage containers. This is a port of a project that originally targetted FreeBSD, but also works on HardenedBSD. \n\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nBrad - monitoring with Grafana\nDennis - a few questions\nPaul - FreeBSD 13\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"Tracing the History of ARM and FreeBSD, Make ‘less’ more friendly, NomadBSD 1.4 Release, Create an Ubuntu Linux jail on FreeBSD 12.2, OPNsense 21.1.2 released, Midnight BSD and BastilleBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nWhen we think of computers, we generally think of laptops and desktops. Each one of these systems is powered by an Intel or AMD chip based on the x86 architecture. It might feel like you spend all day interacting with these kinds of systems, but you would be wrong.
\n\n
\n\nUnix Tip: Make ‘less’ more friendly
\n\nYou probably know about less: it is a standard tool that allows scrolling up and down in documents that do not fit on a single screen. Less has a very handy feature, which can be turned on by invoking it with the -i flag. This causes less to ignore case when searching. For example, ‘udf’ will find ‘udf’, ‘UDF’, ‘UdF’, and any other combination of upper-case and lower-case. If you’re used to searching in a web browser, this is probably what you want. But less is even more clever than that. If your search pattern contains upper-case letters, the ignore-case feature will be disabled. So if you’re looking for ‘QXml’, you will not be bothered by matches for the lower-case ‘qxml’. (This is equivalent to ignorecase + smartcase in vim.)
\n
\n\n\nVersion 1.4 of NomadBSD, a persistent live system for USB flash drives based on FreeBSD and featuring a graphical user interface built around Openbox, has been released: “We are pleased to present the release of NomadBSD 1.4.
\n\n
\n\nCreate an Ubuntu Linux jail on FreeBSD 12.2
\n\n
\n\nOPNsense 21.1.2 released
\n\nWork has so far been focused on the firmware update process to ensure its safety around edge cases and recovery methods for the worst case. To that end 21.1.3 will likely receive the full revamp including API and GUI changes for a swift transition after thorough testing of the changes now available in the development package of this release.
\n\n
\n\nMidnight BSD and BastilleBSD
\n\nWe recently added a new port, mports/sysutils/bastille that allows you to manage containers. This is a port of a project that originally targetted FreeBSD, but also works on HardenedBSD.
\n\n
\n
Onboard Scheduler for the Mars 2020 Rover, Practical Guide to Storage of Large Amounts of Microscopy Data, OpenBSD guest with bhyve - OmniOS, NextCloud on OpenBSD, MySQL Transactions - the physical side, TrueNAS 12.0-U2.1 is released, HardenedBSD 2021 State of the Hardened Union, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nSo you talk to a database, doing transactions. What happens actually, behind the scenes? Let’s have a look.
\n\n
\n\nTrueNAS 12.0-U2.1 is released
\n\n
\n\nHardenedBSD 2021 State of the Hardened Union - NYCBUG - 2021-04-07
\n\n
\n
Michael - remote unlock for encrypted systems
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nLessons learned from a 27 years old UNIX book, Finally dRAID, Setting up a Signal Proxy using FreeBSD, Annotate your PDF files on OpenBSD, Things You Should Do Now, Just: More unixy than Make, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nOne of the Amazon reviewers of "Sun Performance and Tuning: Java and the Internet" gave it 3/5 stars. While still a nice introduction, the book by Adrian Cockcroft has become dated — claimed Roland in 2003, which believe it or not was 18 years ago...
\n\n
\n\ndRAID, Finally!
\n\nAdmins will often use wide RAID stripes to maximize usable storage given a number of spindles. RAID-Z deployments with large stripe widths, ten or larger, are subject to poor resilver performance for a number of reasons. Resilvering a full vdev means reading from every healthy disk and continuously writing to the new spare. This will saturate the replacement disk with writes while scattering seeks over the rest of the vdev. For 14 wide RAID-Z2 vdevs using 12TB spindles, rebuilds can take weeks. Resilver I/O activity is deprioritized when the system has not been idle for a minimum period. Full zpools get fragmented and require additional I/O’s to recalculate data during reslivering. A pool can degenerate into a never ending cycle of rebuilds or loss of the pool Aka: the Death Spiral.
\n\n
\n\nNews Roundup
\n\nSetting up a Signal Proxy using FreeBSD
\n\nWith the events that the private messaging app Signal has been blocked in Iran, Signal has come up with an “proxy” solution akin to Tor’s Bridges, and have given instructions on how to do it.
\n\n
\nFor people who prefer FreeBSD over Linux like myself, we obviously can’t run Docker, which is what Signal’s instructions focus on.
\nFortunately, the Docker image is just a fancy wrapper around nginx, and the configs can be ported to any OS. Here, I’ll show you how to set up a Signal Proxy on FreeBSD.
\n
\n\n\nOn my journey to leave macOS, I regularly look to mimic some of the features I use. Namely, annotating (or signing) PDF files is a really simple task using Preview. I couldn’t do it on OpenBSD using Zathura, Xpdf etc. But there is a software in the ports that can achieve this: Xournal.
\n\n
\nXournal is “an application for notetaking, sketching, keeping a journal using a stylus“. And now that my touchscreen is calibrated, highlighting can even be done with the fingers :)
\n
\n\n\nDescribes things you should do now when building software, because the cost to do them increases over time and eventually becomes prohibitive or impossible.
\n\n
\n\nJust: A command runner. More unixy than Make because it does even less.
\n\nI think it's in the do-one-thing-well spirit of Unix, because it's just a command runner, no build system at all. Just has a bunch of nice features:
\n
\n\n\nJust doesn't replace Make, or any other build system, but it does replace reverse-searching your command history, telling colleagues the weird flags they need to pass to do the thing, and forgetting how to run old projects.
\n
Special Guest: Dan Langille.
","summary":"Lessons learned from a 27 years old UNIX book, Finally dRAID, Setting up a Signal Proxy using FreeBSD, Annotate your PDF files on OpenBSD, Things You Should Do Now, Just: More unixy than Make, and more","date_published":"2021-03-11T03:00:00.000-05:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/edab60b8-425f-45a4-9547-73ca2ca7e341.mp3","mime_type":"audio/mpeg","size_in_bytes":50412600,"duration_in_seconds":3040}]},{"id":"614ca258-a6e1-4c49-ac79-9e37f3e6057c","title":"392: macOS inspired Desktop","url":"https://www.bsdnow.tv/392","content_text":"FreeBSD 13 BETA Benchmarks, FreeBSD Jails Deep Dive by Klara Systems, FreeBSD Foundation looking for a Senior Arm Kernel Engineer & OSS Project Coordinator, macOS-Inspired BSD Desktop OS by helloSystem, A Trip into FreeBSD and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nFreeBSD 13 BETA Benchmarks - Performance Is Much Better\n\n\n\nFreeBSD Jails – Deep Dive into the Beginning of FreeBSD Containers\n\n\nIn recent years, containers and virtualization have become a buzzword in the Linux community, especially with the rise of Docker and Kubernetes. What many people probably don’t realize is that these ideas have been around for a very long time. Today, we will be looking at Jails and how they became part of FreeBSD.\n\n\n\n\n\n\nNews Roundup\n\nFreeBSD Jobs\n\n\nThe FreeBSD Foundation is looking for a Senior Arm Kernel Engineer\nThe FreeBSD Foundation is also looking for an Open Source Project Coordinator.\n***\n### helloSystem Releases New ISOs For This macOS-Inspired BSD Desktop OS\n> The helloSystem motto is being a \"desktop system for creators with focus on simplicity, elegance, and usability. Based on FreeBSD. Less, but better!\" The desktop utilities are written with PyQt5.\n***\n### A Trip into FreeBSD\n> I normally deal with Linux machines. Linux is what I know and it's what I've been using since I was in college. A friend of mine has been coaxing me into trying out FreeBSD, and I decided to try it out and see what it's like. Here's some details about my experience and what I've learned.\n***\n###Tarsnap\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nBeastie Bits\n\n\nTesting Linux Steam Proton on GhostBSD with BSD linuxulator - NO Audio\nNew Build of DragonFlyBSD 5.8\nInstall OpenBSD 6.8 on PINE64 ROCK64 Media Board\nFOSDEM BSD Track Videos are up\n***\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\nSpecial Guest: Dan Langille.","content_html":"FreeBSD 13 BETA Benchmarks, FreeBSD Jails Deep Dive by Klara Systems, FreeBSD Foundation looking for a Senior Arm Kernel Engineer & OSS Project Coordinator, macOS-Inspired BSD Desktop OS by helloSystem, A Trip into FreeBSD and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nIn recent years, containers and virtualization have become a buzzword in the Linux community, especially with the rise of Docker and Kubernetes. What many people probably don’t realize is that these ideas have been around for a very long time. Today, we will be looking at Jails and how they became part of FreeBSD.
\n\n
\n
Special Guest: Dan Langille.
","summary":"FreeBSD 13 BETA Benchmarks, FreeBSD Jails Deep Dive by Klara Systems, FreeBSD Foundation looking for a Senior Arm Kernel Engineer & OSS Project Coordinator, macOS-Inspired BSD Desktop OS by helloSystem, A Trip into FreeBSD and more.","date_published":"2021-03-04T03:00:00.000-05:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/614ca258-a6e1-4c49-ac79-9e37f3e6057c.mp3","mime_type":"audio/mpeg","size_in_bytes":46770312,"duration_in_seconds":2846}]},{"id":"3105d37c-fc28-49e0-983d-1ac767b72f76","title":"391: i386 tear shedding","url":"https://www.bsdnow.tv/391","content_text":"Follow-up about FreeBSD jail advantages, Install Prometheus, Node Exporter and Grafana, Calibrate your touch-screen on OpenBSD, OPNsense 21.1 Marvelous Meerkat Released, NomadBSD 1.4-RC1, Lets all shed a Tear for 386, find mostly doesn't need xargs today on modern Unixes, OpenBSD KDE Status Report, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nFollow-up about FreeBSD jail advantages\n\n\nI’ll admit I ran a lot of justifications together into a single paragraph because I wanted to get to configuring the jails themselves. They’re also, by and large, not specific to FreeBSD’s flavour of containerisation, though I still think it’s easily the most elegant implementation. Sometimes the simplest solution really is the best one.\n\n\n\nHistory of FreeBSD part 4: TCP/IP\n\n\nHow TCP/IP evolved and BSDs special contribution to the history of the Internet\n***\n\n\n\nFreeBSD: Install Prometheus, Node Exporter and Grafana\n\n\nFreeBSD comes out of the box with three great tools for monitoring. If you need more info about how these tools work, please read the official documentation. I’ll explain the installation only and creating a simple dashboard.\n\n\n\n\nNews Roundup\n\nCalibrate your touch-screen on OpenBSD\n\n\nI didn’t expected it but my refurbished T460s came with a touch-screen. It is recognized by default on OpenBSD and not well calibrated as-is. But that’s really simple to solve.\n\n\n\nLets all shed a Tear for 386\n\nFreeBSD is designating i386 as a Tier 2 architecture starting with FreeBSD 13.0. The Project will continue to provide release images, binary updates, and pre-built packages for the 13.x branch. However, i386-specific issues (including SAs) may not be addressed in 13.x. The i386 platform will remain Tier 1 on FreeBSD 11.x and 12.x.\n\n\n\n\nOPNsense 21.1 Marvelous Meerkat Released\n\n\nFor more than 6 years, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.\n\n\n\nNomadBSD 1.4-RC1\n\nWe are pleased to present the first release candidate of NomadBSD 1.4.\n\n\n\n\nfind mostly doesn't need xargs today on modern Unixes\n\n\nI've been using Unix for long enough that 'find | xargs' is a reflex. When I started and for a long time afterward, xargs was your only choice for efficiently executing a command over a bunch of find results.\n\n\n\nOpenBSD KDE Status Report\n\nOpenBSD has managed to drop KDE3 and KDE4 in the 6.8 -> 6.9 release cycle. That makes me very happy because it was a big piece of work and long discussions. This of course brings questions: Kde Plasma 5 package missing.\nAfter half a year of work, I managed to successfully update the Qt5 stack to the last LTS version 5.15.2. On the whole, the most work was updating QtWebengine. What a monster! With my CPU power at home, I can build it 1-2 times a day which makes testing a little bit annoying and time intensive.\n\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nKarl - Firefox webcam audio solution\nMichal - openzfs\nDave - bufferbloat\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"Follow-up about FreeBSD jail advantages, Install Prometheus, Node Exporter and Grafana, Calibrate your touch-screen on OpenBSD, OPNsense 21.1 Marvelous Meerkat Released, NomadBSD 1.4-RC1, Lets all shed a Tear for 386, find mostly doesn't need xargs today on modern Unixes, OpenBSD KDE Status Report, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI’ll admit I ran a lot of justifications together into a single paragraph because I wanted to get to configuring the jails themselves. They’re also, by and large, not specific to FreeBSD’s flavour of containerisation, though I still think it’s easily the most elegant implementation. Sometimes the simplest solution really is the best one.
\n\n
\n\nHistory of FreeBSD part 4: TCP/IP
\n\n\n
\n- How TCP/IP evolved and BSDs special contribution to the history of the Internet\n***
\n
\n\n\nFreeBSD comes out of the box with three great tools for monitoring. If you need more info about how these tools work, please read the official documentation. I’ll explain the installation only and creating a simple dashboard.
\n\n
\n
\n\n\nI didn’t expected it but my refurbished T460s came with a touch-screen. It is recognized by default on OpenBSD and not well calibrated as-is. But that’s really simple to solve.
\n\n
\n\nLets all shed a Tear for 386
\n\nFreeBSD is designating i386 as a Tier 2 architecture starting with FreeBSD 13.0. The Project will continue to provide release images, binary updates, and pre-built packages for the 13.x branch. However, i386-specific issues (including SAs) may not be addressed in 13.x. The i386 platform will remain Tier 1 on FreeBSD 11.x and 12.x.
\n\n
\n
\n\n\nFor more than 6 years, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
\n\n
\n\nNomadBSD 1.4-RC1
\n\nWe are pleased to present the first release candidate of NomadBSD 1.4.
\n\n
\n
\n\n\nI've been using Unix for long enough that 'find | xargs' is a reflex. When I started and for a long time afterward, xargs was your only choice for efficiently executing a command over a bunch of find results.
\n\n
\n\nOpenBSD KDE Status Report
\n\nOpenBSD has managed to drop KDE3 and KDE4 in the 6.8 -> 6.9 release cycle. That makes me very happy because it was a big piece of work and long discussions. This of course brings questions: Kde Plasma 5 package missing.
\n\n
\nAfter half a year of work, I managed to successfully update the Qt5 stack to the last LTS version 5.15.2. On the whole, the most work was updating QtWebengine. What a monster! With my CPU power at home, I can build it 1-2 times a day which makes testing a little bit annoying and time intensive.
\n
Did Linux kill Commercial Unix, three node GlusterFS setup on FreeBSD, OpenBSD on the Lenovo ThinkPad X1 Nano (1st Gen), NetBSD on EdgeRouter Lite, TLS Mastery first draft done
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nSales of commercial Unix have fallen off a cliff. There has to be something behind this dramatic decline. Has Linux killed its ancestor by becoming a perfectly viable replacement, like an operating system version of Invasion of the Body Snatchers?
\n\n
\n\nWireguard: Simple and Secure VPN in FreeBSD
\n\n\n
\n- A great article by Tom Jones about setting up Wireguard on FreeBSD\n***
\n
\n\n\nGlusterFS (GFS) is the open source equivalent to Microsoft's Distributed Filesystem (DFS). It's a service that replicates the contents of a filesystem in real time from one server to another. Clients connect to any server and changes made to a file will replicate automatically. It's similar to something like rsync or syncthing, but much more automatic and transparent. A FreeBSD port has been available since v3.4, and (as of this post) is currently at version 8.0 with 9.0 being released soon.
\n\n
\n\nNews Roundup
\n\nOpenBSD on the Lenovo ThinkPad X1 Nano (1st Gen)
\n\nLenovo has finally made a smaller version of its X1 Carbon, something I’ve been looking forward to for years.
\n\n
\n\nNetBSD on the EdgeRouter Lite
\n\nNetBSD-current now has pre-built octeon bootable images (which will appear in NetBSD 10.0) for the evbmips port, so I decided to finally give it a try. I've been happily running OpenBSD/octeon on my EdgeRouter Lite for a few years now, and have previously published some notes including more detail about the CPU.
\n\n
\n\n“TLS Mastery” first draft done!
\n\n
\n
A week with Plan 9, Exploring Swap on FreeBSD, how to create a FreeBSD pkg mirror using bastille and poudriere, How to set up FreeBSD 12 VNET jail with ZFS, Creating Comfy FreeBSD Jails Using Standard Tools, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI spent the first week of 2021 learning an OS called Plan 9 from Bell Labs. This is a fringe Operating System, long abandoned by it’s original authors. It's also responsible for a great deal of inspiration elsewhere. If you’ve used the Go language, /proc, UTF-8 or Docker, you’ve used Plan 9-designed features. This issue dives into Operating System internals and some moderately hard computer science topics. If that sort of thing isn’t your bag you might want to skip ahead. Normal service will resume shortly.
\n\n
\n\nExploring Swap on FreeBSD
\n\nOn modern Unix-like systems such as FreeBSD, “swapping” refers to the activity of paging out the contents of memory to a disk and then paging it back in on demand. The page-out activity occurs in response to a lack of free memory in the system: the kernel tries to identify pages of memory that probably will not be accessed in the near future, and copies their contents to a disk for safekeeping until they are needed again. When an application attempts to access memory that has been swapped out, it blocks while the kernel fetches that saved memory from the swap disk, and then resumes execution as if nothing had happened.
\n
\n\n\nThis a short how-to for creating a FreeBSD pkg mirror using BastilleBSD and Poudriere.
\n\n
\n\nHow to set up FreeBSD 12 VNET jail with ZFS
\n\nHow do I install, set up and configure a FreeBSD 12 jail with VNET on ZFS? How can I create FreeBSD 12 VNET jail with /etc/jail.conf to run OpenVPN, Apache, Wireguard and other Internet-facing services securely on my BSD box?
\n\n
\nFreeBSD jail is nothing but operating system-level virtualization that allows partitioning a FreeBSD based Unix server. Such systems have their root user and access rights. Jails can use network subsystem virtualization infrastructure or share an existing network. FreeBSD jails are a powerful way to increase security. Usually, you create jail per services such as an Nginx/Apache webserver with PHP/Perl/Python app, WireGuard/OpeNVPN server, MariaDB/PgSQL server, and more. This page shows how to configure a FreeBSD Jail with vnet and ZFZ on FreeBSD 12.x.
\n\nCreating Comfy FreeBSD Jails Using Standard Tools
\n\nDocker has stormed into software development in recent years. While the concepts behind it are powerful and useful, similar tools have been used in systems for decades. FreeBSD’s jails in one of those tools which build upon even older chroot(2) To put it shortly, with these tools, you can make a safe environment separated from the rest of the system.
\n\n
\n
FreeBSD Q4 2020 Status report, a must-have security tool from OpenBSD, Bastille Port Redirection and Persistence, FreeBSD Wall Display Computer, etymology of command-line tools, GhostBSD 21.01.15 Release Notes, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nPf-badhost is a very practical, robust, stable and lightweight security script for network servers.
\n\n
\nIt's compatible with BSD based operating systems such as {Open,Free,Net,Dragonfly}BSD and MacOS. It prevents potentially-bad IP addresses that could possibly attack your servers (and waste your bandwidth and fill your logfiles), by blocking all those IPs contacting your server, and therefore it makes your server network/resources lighter and the logs of important services running on your server become simpler, more readable and efficient.
\n
\n\n\nBastille supports redirecting (rdr) ports from the host system into target containers. This port redirection is commonly used when running Internet services such as web servers, dns servers, email and many others. Any service you want to make public outside of your cluster will likely require port redirection (with some exceptions, see below).
\n\n
\n\nFreeBSD Wall Display Computer
\n\nI've recently added a wall mounted 30" monitor for Grafana in my home. I can highly recommend doing the same, especially in a world where more work from home is becoming the norm.
\n\n
\n\nThe etymology of command-line tools
\n\n
\n\nGhostBSD 21.01.15 Release Notes
\n\nI am happy to announce the availability of the new ISO 21.01.15. This new ISO comes with a clean-up of packages that include removing LibreOffice and Telegram from the default selection. We did this to bring the zfs RW live file systems to run without problem on 4GB of ram machine. We also removed the UFS full disk option from the installer. Users can still use custom partitions to setup UFS partition, but we discourage it. We also fixed the Next button's restriction in the custom partition related to some bug that people reported. We also fix the missing default locale setup and added the default setup for Linux Steam, not to forget this ISO includes kernel, userland and numerous application updates.
\n
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nGNN's tips for surviving Cabin Fever and Coding from Home, Self-host a password manager on OpenBSD, Preliminary OpenBSD Support added to OBS, Dan's CURL tip of the Day, List of some Shell goodies for OpenBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nForgive me if this seems off topic, but I was wondering if you had any advice for the majority of us who are now KFH (koding from home). I don't know how KV works day to day, but it seems pretty clear that the status quo has changed at most workplaces in the last several months, and it's hard to know if there are things we could be doing to stay productive while we're all at home, ordering delivery, and microwaving our mail. Does KV have any good guidance?
\n
\n\n\nI’ve been using Rubywarden to store and access my passwords from OpenBSD workstations and iOS toys. But recent redondant failures from the iOS App and rubywarden not being maintained anymore led to the need for a new solution.
\n
\nI was investing on pass+pgp+git but it was quite complex.
\n\n\nI'm sharing here some practices I'm following and some small tips/tools which facilitate my usage of OpenBSD in my day to day.
\n
\nSome are really specific to my usage, others could be re-used.
• [Traditional text mode games from BSD](https://github.com/msharov/bsd-games)\n• [FreeBSD Easter Eggs](https://twitter.com/freebsdfrau/status/972893680473317377)\n• [A prehistory and history of Unix Slide Deck](https://docs.google.com/presentation/d/1BxnFiP_Hv3HJbbYRfSxpTym7GzqxJPQlTE6Ur5h1Al8/edit#slide=id.g951f86c343_0_95)\n• [How to use Android USB Tethering to get Internet on FreeBSD](https://www.youtube.com/watch?v=cAEmtrEZlV8)\n• [VPN'Othon #2 for CharmBUG](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/387/charmbug_event.md)\n
\n\n• [Kev - Ramdisk](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/387/feedback/kev%20-%20ramdisk.md)\n• [John - new to freebsd](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/387/feedback/John%20-%20new%20to%20freebsd)\n
\n\nRouting and Firewalling VLANS with FreeBSD, FreeBSD 12 VNET jail with ZFS howto, pkgsrc-2020Q4 released, FreeBSD on Raspberry Pi 4 With 4GB of RAM, HardenedBSD December 2020 Status Report, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nIn this article we are going to look at and integrate two network isolation technologies, VLANs and VNET. VLANs are common place, and if you have done some network management or design then you are likely to have interacted with them. The second are FreeBSDs VNET virtual network stacks, a powerful network stack isolation technology that gives FreeBSD jails super powers.
\n
\nEthernet VLAN (standardised by IEEE 802.1Q) are an extension to Ethernet and provide an essential method for scaling network deployments. They are used in all environments to enable reuse of common infrastructure by isolating portions of networks from each other. VLANs allow the reuse of common cables, switches and routers to carry completely different networks. It is common to have data that must be separated from different networks carried on common cables until their VLAN tags are finally stripped at a gateway switch or router.
\n\n\nHow do I install, set up and configure a FreeBSD 12 jail with VNET on ZFS? How can I create FreeBSD 12 VNET jail with /etc/jail.conf to run OpenVPN, Apache, Wireguard and other Internet-facing services securely on my BSD box?
\n
\nFreeBSD jail is nothing but operating system-level virtualization that allows partitioning a FreeBSD based Unix server. Such systems have their root user and access rights. Jails can use network subsystem virtualization infrastructure or share an existing network. FreeBSD jails are a powerful way to increase security. Usually, you create jail per services such as an Nginx/Apache webserver with PHP/Perl/Python app, WireGuard/OpeNVPN server, MariaDB/PgSQL server, and more. This page shows how to configure a FreeBSD Jail with vnet and ZFS on FreeBSD 12.x.
\n\n\nThe pkgsrc developers are proud to announce the 69th quarterly release
\n
\nof pkgsrc, the cross-platform packaging system. pkgsrc is available
\nwith more than 24,000 packages, running on 23 separate platforms; more
\ninformation on pkgsrc itself is available at https://www.pkgsrc.org/
\n\n\nThis is the story of how I managed to get FreeBSD running on a Raspberry Pi 4 with 4GB of RAM, though I think the setup story is pretty similar for those with 2GB and 8GB.1
\n
\n\n\nHappy New Year! On this the last day of 2020, I submit December's status report.
\n
Description: History of FreeBSD: Early Days of FreeBSD, mesh VPN using OpenBSD and WireGuard, FreeBSD Foundation Sponsors LLDB Improvements, Host your Cryptpad web office suite with OpenBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nIn this third part of our series on the history of FreeBSD, we start tracing the early days of FreeBSD and the events that would eventually shape the project and the future of open source software.
\n\n
\n
\n\n\nWireGuard is a new coming to OpenBSD 6.8 and it looks like a simple and efficient way to connect computers.
\n\n
\nI own a few VPS (hello Vultr, hello OpenBSD.amsterdam) that tend to be connected through filtered public services and/or SSH tunnels. And that’s neither efficient nor easy to manage. Here comes the wg(4) era where all those peers will communicate with a bit more privacy and ease of management.
\n
\n\n\nWith FreeBSD Foundation grant, Moritz Systems improved LLDB support for FreeBSD
\n\n
\nThe LLDB project builds on libraries provided by LLVM and Clang to provide a great modern debugger. It uses the Clang ASTs and the expression parser, LLVM JIT, LLVM disassembler, etc so that it provides an experience that “just works”. It is also blazing fast and more permissively licensed than GDB, the GNU Debugger.
\nLLDB is the default debugger in Xcode on macOS and supports debugging C, Objective-C, and C++ on the desktop and iOS devices and the simulator.
\n
\n\n\nIn this article I will explain how to deploy your own Cryptpad instance with OpenBSD. Cryptpad is a web office suite featuring easy real time collaboration on documents. Cryptpad is written in JavaScript and the daemon acts as a web server.
\n
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nAllen K. Briggs Memorial Scholarship, Toward an automated tracking of OpenBSD ports contributions, Trying OpenZFS 2 on FreeBSD 12.2-RELEASE, OpenBSD on TECLAST F7 Plus, Multi-volume support in HAMMER2, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nAllen Briggs was one of the earliest members of the NetBSD community, pursuing his interest in macBSD, and moving to become a NetBSD developer when the two projects merged. Allen was known for his quiet and relaxed manner, and always brought a keen wisdom with him; allied with his acute technical expertise, he was one of the most valued members of the NetBSD community.
\n
\nThe Allen K. Briggs Memorial Scholarship is an endowment to provide scholarships in perpetuity for summer programs at the North Carolina School of Science & Math, which Allen considered to be a place that fundamentally shaped him as a person. We would love to invite Allen's friends and colleagues from the BSD community to donate to this cause so that we can provide more scholarships to students with financial need each year. We are approximately halfway to our goal of $50K with aspirations to exceed that target and fund additional scholarships.
\n\n\nA first step for the CI service would be to create a database of diffs sent to ports. This would allow people to track what has been sent and not yet committed and what the state of the contribution is (build/don’t build, apply/don’t apply).
\n\n
\n
\n\n\nOpenZFS 2 is a huge achievement, and makes me bullish about the long term prospects for the world’s most trustworthy and nicest to use storage system. You can even use try it today on FreeBSD 12.2-RELEASE, though I recommend tracking -CURRENT for these sorts of features.
\n\n
\n
\n\n\nI got myself a TECLAST F7 Plus laptop. It comes preinstalled with Windows 10 but I planned to use it as my daily driver. So I installed OpenBSD 6.8 on it.
\n\n
\n
FreeBSD Remote Process Plugin Final Milestone achieved, Tailscale for OpenBSD, macOS to FreeBSD migration, monitoring of our OpenBSD machines, OPNsense 20.7.6 released, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nMoritz Systems have been contracted by the FreeBSD Foundation to modernize the LLDB debugger’s support for FreeBSD. We are working on a new plugin utilizing the more modern client-server layout that is already used by Darwin, Linux, NetBSD and (unofficially) OpenBSD. The new plugin is going to gradually replace the legacy one.
\n\n
\n\nTailscale on OpenBSD
\n\nI spent some time setting this up today evening and thought I’d post the steps here. Nothing fancy, just putting together various pieces actually.
\n
\nI assume you know what Tailscale is; if not check out their website. Basically it is a mesh network built on top of Wireguard. Using it you can have all your devices both within your LAN(s) and outside be on one overlay network as if they are all on the same LAN and can talk to each other. It’s my new favourite thing!
\n\n\nThis is not a technical documentation for how I migrated from macOS to FreeBSD. This is a high-level for why I migrated from macOS to FreeBSD.
\n\n
\nNot so long ago, I was using macOS as my daily driver. The main reason why I got a macbook was the underlying BSD Unix and the nice graphics it provides. Also, I have an iPhone. But they were also the same reasons for why I left macOS.
\n\nOur monitoring of our OpenBSD machines, such as it is (as of November 2020
\n\nWe have a number of OpenBSD firewalls in service (along with some other OpenBSD servers for things like VPN endpoints), and I was recently asked how we monitor PF and overall network traffic on them. I had to disappoint the person who asked with my answer, because right now we mostly don't (although this is starting to change).
\n\n
\n\nOPNsense 20.7.6 released
\n\nThis update brings the usual mix of reliability fixes, plugin and third party software updates: FreeBSD, HardenedBSD, PHP, OpenSSH, StrongSwan, Suricata and Syslog-ng amongst others.
\n\n
\nPlease note that Let's Encrypt users need to reissue their certificates manually after upgrading to this version to fix the embedded certificate chain issue with the current signing CA switch going on.
\n\nNYC Bug Jan 2021 with Michael W. Lucas
\n\n
\n\nTarsnap
\n\n\n
\n- This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n
We asked for it, you answered our call. This episode features you interviewing us with questions that you sent in. JT, Allan, and Benedict answer everything that you ever wanted to know in this week’s special episode of BSDNow.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
Benedict: You work at a university right? Were you already into tech before you started working there? What do you do there?
\nYes, I do work at the University of Applied Sciences, Darmstadt, Germany. I’m a lab engineer there (without a lab, but with a big data cluster). I teach in the winter semester an undergraduate, elective course called “Unix for Developers”. Yes, I was already in tech by that time. Did some previous work at companies before (selling hardware at the call-in hotline and later in the store) and during my CS studies.
\nAllan: What’s the next big FreeBSD Project you plan on doing?
JT: How did you get involved in BSD? Weren't you a Linux guy?
\n\nAll: Is there any way you can create an entire episode of BSDnow on hardware that runs OpenBSD and FreeBSD? We see you audacity, etc on a mac.
\nBenedict: Not sure about OpenBSD (don’t use it), but FreeBSD should be doable for my part. If we switch from Skype to a different video chat tool, the rest is already there. Production side may be more difficult, but not impossible.
All: if you could finish up one project right now... what would it be?
\nBenedict: Updated ZFS chapter in the FreeBSD handbook.
All: How did all of you guys meet?
\n\nAll: My question is, do you guys use FreeBSD as your main desktop OS? If not, what do you use?
\nBenedict: No, but Mac OS is close enough. Doing a lot of SSHing into FreeBSD from there.
\nAll: Can you all give us the best shot of outside of their windows?
\nJT’s answer: https://photos.smugmug.com/photos/i-2LSbspL/0/69437dbb/5K/i-2LSbspL-5K.jpg
\n Allan: https://photos.app.goo.gl/UnKXnKMt6cn8FDhNA
\n Benedict: No, it’s dark outside anyway. ;-)
All: How old were you when you got your first computer and what was that computer?
\nAllan: 12 or 13, a 486DX2/66hz with an insane 32mb of RAM, 400 and 500 MB SCSI HDDs, 14400 baud model, and a 1.7x CD rom drive
\nBenedict: Around 13 or so. 386DX2, 4 MB RAM, IDE disk drive (no idea how big, but it wasn’t much), 3.5” floppy, DOS, and a lot of games.
\nJT: Technically the first was a Atari 1200XL with a 6502 CPU running at 1.79 MHz 64KB RAM. It had it's own OS and you could load programs off of either cartridges, floppy disks, or cassette tapes. First PC Clone was a Packard Bell with a 386 and 1mb ram which later was upgraded to 4mb and a Dual speed CD-ROM. My dad got me a Compaq 286 laptop... this one (show)... a year or so later because he got tired of fighting me for the computer.
All: Can we have a peek at your bookcase and what books are there?
\nAllan: No picture handy, but my shelf is pretty small, mostly a collection of autographed FreeBSD books. I have D&I with all 3 autographs (took some travel to acquire), and a copy of my first book (FreeBSD Mastery: ZFS) autographed by Jeff Bonwick and Matt Ahrens, the creators of ZFS, plus a bunch of other big names in ZFS like George Wilson.
\nJT’s answer: So... my library is packed away... but here’s about half of it... the rest is still in storage. https://photos.smugmug.com/photos/i-SBG2KDv/0/0b9856b8/4K/i-SBG2KDv-4K.jpg
\nSoftware Collection: https://photos.smugmug.com/photos/i-HfTVPN9/0/ad610dd4/O/i-HfTVPN9.jpg
\nBenedict: A mix of FreeBSD books (by MWL), the graveyard book, 4 hour work week, the once and future king (took me a long time to finish that one), Total Immersion swimming (still learning to swim) and some books in german language, fiction and tech. Groff lives in there while the pandemic lasts.
All: What desktop/Window Manager/shell do each of you primarily use?
\nBenedict: Mainly Mac OS, when on FreeBSD it’s i3. Zsh with zsh-autosuggestions currently.
\nJT: Lumina/zsh
\nAllan: Lumina and tcsh, want to learn zsh but never gotten time to change
All: What spoken languages do you speak?
\n Benedict: German and English (obviously), learning a bit of Spanish via Duolingo at the moment
\n JT: English, Bad English, and some French.
\nAll: Do you have Non-Computer hobbies if so what are those?
\n Benedict: Tai Chi Chuan (Yang Style)
\nJT: I'd say photography, but that's a job for me. I have a lot of varied interests, Krav Maga, working on my VW Corrado, working on the old Victorian house I bought, and camping/backpacking. Ive done the northern half of the AT (Appalachian Trail, I want to finish it up and then do the PCT and CDT. (Pacific Crest Trail and Continental Divide Trail).
All: When COVID passes, when are either of you are coming to BSD pizza night in Portland, OR, USA so I can buy you a beer/wine/whisky or pizza/coffee/tea (or six)
\n\nAll: What was the first car you ever owned?
\n\nAll: Do you own a vehicle and if so what is the make/model?
\n\nAll: Favorite Star franchise? Star Wars, Star Trek, Stargate, Battlestar, etc.
\n\nJT: Will you ever host any more BSDNow episodes?
\n\nAll: Favorite superhero? Marvel and/or DC.
\n\nAll: Favorite game(s) of all time?
\n\nAll: Pants or no pants on virtual meetings/presentations?
\n\nAll: Do you or have you used alternative operating systems that are not "main stream or is considered retro" if so what are those?
\n\nAll: Who has more animals at home?
\n\nAllan: Does Allan have any batteries for his tetris cubes? Can we see that thing light up?
\n\nAllan and Benedict: Are you guys going to go on JT's new show?
\n\nIf you’re wondering what show this is, here are the two shows Im a host of:
\n\nhttps://www.opensourcevoices.org & https://www.theopiniondominion.org
\n\nAllan and Benedict: Have Allan or Benedict lost anything on the way to and from a conference?
\n\nBenedict: Is Benedict going to do his NOEL blocks again?
\n\nBenedict: Does Benedict make his bed every Wednesday morning? It always looks great!
\nNot just Wednesdays, but pretty much every day. Here, watch this: https://www.youtube.com/watch?v=GKZRFDCbGTA Nuff said. ;-)
\nJT: Are you batman because the episodes are always awesome sir so thank you
\nJT’s answer: Can you ever admit to being batman? If I were batman wouldn't I have to deny it?
All: What's your Daily Driver Hardware?
\n\nAll: Who has more servers or VMs at home?
\nBenedict: Allan, easily
\nJT: Allan definitely beats me with VMs, but I think I might give him a run on servers. 4x 4u HP DL580s, one HP DL980, three HP C3000 8 bay bladecenters, three HP C7000 16 bay Bladecenters, 2x Sun 280R, bunch of Dell and IBM 1Us… but all my stuff is old. Allan has all the new and shiny stuff.
\nThe Pile in the Kitchen: https://photos.smugmug.com/photos/i-HBScrpk/0/4b058cc5/X2/i-HBScrpk-X2.jpg
\nThe other pile: https://photos.smugmug.com/photos/i-wNxFszV/0/e7a4b2d6/X2/i-wNxFszV-X2.jpg
All: What book(s) are you currently reading?
\nBenedict: Antifragile by Nassim Taleb
\nJT: Douglas Hofstader - Gödel, Escher, Bach: An Eternal Golden Braid. Douglas Rushkoff - program or be programmed. Also a 4 part book series on the American civil war written in the 1880s, by people in the civil war.
All: Favorite mechanical keyboard switch? Cherry MX, Kalih, Gateron, etc.
\nBenedict: Cherry MX brown currently
\n Allan: Cherry MX Blue (Coolermaster Master Keys Pro-L)
\n JT: I prefer scissor switches, so I use a Logitech K740.
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nThe Origin of the Shell, Return to Plan 9, ArisbluBSD: Why a new BSD?, OPNsense 20.7.5 released, Midnight BSD 2.0 Release Status, HardenedBSD November 2020 Status Report, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nCTSS was developed during 1963 and 64. I was at MIT on the computer center staff at that time. After having written dozens of commands for CTSS, I reached the stage where I felt that commands should be usable as building blocks for writing more commands, just like subroutine libraries. Hence, I wrote "RUNCOM", a sort of shell driving the execution of command scripts, with argument substitution. The tool became instantly most popular, as it became possible to go home in the evening while leaving behind long runcoms executing overnight. It was quite neat for boring and repetitive tasks such as renaming, moving, updating, compiling, etc. whole directories of files for system and application maintenance and monitoring.
\n\n
\n
\n\n\nPlan 9 from Bell Labs has held the same charm after my last visit that took a few days. This time I'll keep this operating system in an emulator where I can explore into it when I am distracted.
\n\n
\n
\n\n\nThis article is to explain some decisions and plans made by the ArisbluBSD team, why we are making our own thing, and what the plan is for the OS. We mainly want to talk about five things: desktop, package management, software availability, custom software, and the future of the OS. We mostly want to explain what the goal of the OS is, and how we plan to expand in the near future. Without further ado, let's explain ArisbluBSD's plan.
\n\n
\n
\n\n\nWe return briefly for a small patch set and plan to pin the 20.1 upgrade path to this particular version to avoid unnecessary stepping stones. We wish you all a healthy Friday. And of course: patch responsibly!
\n\n
\n\nMidnight BSD 2.0 Release Status
\n\nWe identified some issues with the 2.0 ISOs slated for release with the ZFS bootloader not working.
\n\n
\nUntil this issue is resolved, we are unable to build release ISOs. We've left the old ones up as they work fine for anyone using UFS.
\n\nHardenedBSD November 2020 Status Report
\n\nWe're getting close to the end of November. My wife and I have plans this weekend, so I thought I'd take the time to write November's status report today.
\n\n
\n
• [rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.](https://phiresky.github.io/blog/2019/rga--ripgrep-for-zip-targz-docx-odt-epub-jpg/)\n• [exa - A modern replacement for ls](https://the.exa.website/)\n• [The myriad meanings of pwd in Unix systems](https://qmacro.org/2020/11/08/the-meaning-of-pwd-in-unix-systems/)\n
\n\nWe read FreeBSD’s 3rd quarter status report, OpenZFS 2.0, adding check-hash checks in UFS filesystem, OpenSSL 3.0 /dev/crypto issues on FreeBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThe call for submissions for the 4th Quarter is out
\n\n
\n
\n\n\nThis Monday, ZFS on Linux lead developer Brian Behlendorf published the OpenZFS 2.0.0 release to GitHub. Along with quite a lot of new features, the announcement brings an end to the former distinction between "ZFS on Linux" and ZFS elsewhere (for example, on FreeBSD). This move has been a long time coming—the FreeBSD community laid out its side of the roadmap two years ago—but this is the release that makes it official.
\n\n
\n
\n\n\nVarious new check-hash checks have been added to the UFS filesystem
\n\n
\nover various major releases. Superblock check hashes were added for
\nthe 12 release and cylinder-group and inode check hashes will appear
\nin the 13 release.
\n\nOpenSSL 3.0 /dev/crypto issues on FreeBSD
\n\nSo, just learned that the OpenSSL devs decided to break /dev/crypto on FreeBSD.
\n\n
\n
Adventures in Freebernetes, tracing kernel functions, The better way of building FreeBSD networks, New beginnings: CDBUG virtual meetings, LibreSSL update in DragonFly, Signal-cli with scli on FreeBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nPart 2 of experiments in FreeBSD and Kubernetes: Creating your first guest
\n\n
\n
\n\n\nIn my previous post I described how FBT intercepts function calls and vectors them into the DTrace framework. That laid the foundation for what I want to discuss in this post: the implementation of the stack() action and built-in arg variables. These features rely on the precise layout of the stack, the details of which I touched on previously. In this post I hope to illuminate those details a bit more with the help of some visuals, and then guide you through the implementation of these two DTrace features as they relate to the FBT provider.
\n\n
\n
\n\n\nDummynet is the FreeBSD traffic shaper, packet scheduler, and network emulator. Dummynet allows you to emulate a whole set of network environments in a straight-forward way. It has the ability to model delay, packet loss, and can act as a traffic shaper and policer. Dummynet is roughly equivalent to netem in Linux, but we have found that dummynet is easier to integrate and provides much more consistent results.
\n\n
\n
\n\n\nI had overwhelmingly positive responses from the broader *BSD community about restarting CDBUG meetings as virtual, at least for now. Hopefully this works well and even when we're back to in-person meetings we can still find a way to bring in virtual attendees.
\n\n
\n
\n\n\nDragonFly has a new version of libressl, noting cause it has a newer TLS1.3 implementation – something that may be necessary for you.
\n\n
\n
\n\n\nSo couple of days ago I migrated from macOS on Macbook Pro to FreeBSD on ThinkPad T480s.
\n\n
\n
Interview with Michael W. Lucas: SNMP and TLS book, cashflow for creators, book sale and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nSNMP Book
\n\n
\nThe Networknomicon
\nSponsor the TLS Book
\nCashflow for creators
\nBook sale
\n\nTarsnap
\n\n\n
\n- This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n
Special Guest: Michael W Lucas.
","summary":"Interview with Michael W. Lucas: SNMP and TLS book, cashflow for creators, book sale and more. ","date_published":"2020-11-26T06:00:00.000-05:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/5d96e357-c800-4037-bc9d-3251ca0b1cd0.mp3","mime_type":"audio/mpeg","size_in_bytes":55682424,"duration_in_seconds":3380}]},{"id":"610cb191-462b-4968-a1ae-01d1aebf93ba","title":"377: Firewall ban-sharing","url":"https://www.bsdnow.tv/377","content_text":"History of FreeBD: BSDi and USL Lawsuits, Building a Website on Google Compute Engine, Firewall ban-sharing across machines, OpenVPN as default gateway on OpenBSD, Sorting out what the Single Unix Specification is, Switching from Apple to a Thinkpad for development, and more\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nHistory of FreeBSD : Part 2 : BSDi and USL Lawsuits\n\n\nIn this second part of our series on the history of FreeBSD, we continue to trace the pre-history of FreeBSD and the events that would eventually shape the project and the future of open source software. \n\n\n\n\nBuilding a Web Site on Google Compute Engine\n\n\nHere's how I deployed a web site to the Google Cloud Platform. I used FreeBSD for good performance, stability, and minimal complexity. I set up HTTPS with free Let's Encrypt TLS certificates for both RSA and ECC. Then I adjusted the Apache configuration for a good score from the authoritative Qualys server analysis.\n\n\n\n\nNews Roundup\n\nFirewall ban-sharing across machines\n\n\nAs described in My infrastructure as of 2019, my machines are located in three different sites and are loosely coupled. Nonetheless, I wanted to set things up so that if an IP address is acting maliciously toward one machine, all my machines block that IP at once so the meanie won't get to try one machine after another.\n\n\n\nOpenVPN as default gateway on OpenBSD\n\nIf you plan to use an OpenVPN tunnel to reach your default gateway, which would make the tun interface in the egress group, and use tun0 in your pf.conf which is loaded before OpenVPN starts?\nHere are the few tips I use to solve the problems.\n\n\n\nSorting out what the Single Unix Specification is and covers\n\nSorting out what the Single Unix Specification is and covers\nOctober 8, 2020\nI've linked to the Single Unix Specification any number of times, for various versions of it (when I first linked to it, it was at issue 6, in 2006; it's now up to a 2018 edition). But I've never been quite clear what it covered and didn't cover, and how it related to POSIX and similar things. After yesterday's entry got me looking at the SuS site again, I decided to try to sort this out once and for all.\n\n\n\nBye-bye, Apple\n\nThe days of Apple products are behind me. I had been developing on a Macbook for over twelve years, but now, I’ve switched to an ever trending setup: OpenBSD on a Thinkpad.\nThe new platform is a winner. Everything is clean, quick, and configurable. When I ps uaxww, I’m not hogging ‘gigs’ of RAM just to have things up and running. There’s no black magic that derails me at every turn. In short, my sanity has been long restored.\n\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nChris - small projects\nJens - ZFS Question\n\n\nOne pool to rule them all\n\nShroyer - Dotnet on FreeBSD for Jellyfin\n***\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"History of FreeBD: BSDi and USL Lawsuits, Building a Website on Google Compute Engine, Firewall ban-sharing across machines, OpenVPN as default gateway on OpenBSD, Sorting out what the Single Unix Specification is, Switching from Apple to a Thinkpad for development, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nIn this second part of our series on the history of FreeBSD, we continue to trace the pre-history of FreeBSD and the events that would eventually shape the project and the future of open source software.
\n\n
\n
\n\n\nHere's how I deployed a web site to the Google Cloud Platform. I used FreeBSD for good performance, stability, and minimal complexity. I set up HTTPS with free Let's Encrypt TLS certificates for both RSA and ECC. Then I adjusted the Apache configuration for a good score from the authoritative Qualys server analysis.
\n\n
\n
\n\n\nAs described in My infrastructure as of 2019, my machines are located in three different sites and are loosely coupled. Nonetheless, I wanted to set things up so that if an IP address is acting maliciously toward one machine, all my machines block that IP at once so the meanie won't get to try one machine after another.
\n\n
\n\nOpenVPN as default gateway on OpenBSD
\n\nIf you plan to use an OpenVPN tunnel to reach your default gateway, which would make the tun interface in the egress group, and use tun0 in your pf.conf which is loaded before OpenVPN starts?
\n\n
\nHere are the few tips I use to solve the problems.
\n\nSorting out what the Single Unix Specification is and covers
\n\nSorting out what the Single Unix Specification is and covers
\n\n
\nOctober 8, 2020
\nI've linked to the Single Unix Specification any number of times, for various versions of it (when I first linked to it, it was at issue 6, in 2006; it's now up to a 2018 edition). But I've never been quite clear what it covered and didn't cover, and how it related to POSIX and similar things. After yesterday's entry got me looking at the SuS site again, I decided to try to sort this out once and for all.
\n\nBye-bye, Apple
\n\nThe days of Apple products are behind me. I had been developing on a Macbook for over twelve years, but now, I’ve switched to an ever trending setup: OpenBSD on a Thinkpad.
\n
\nThe new platform is a winner. Everything is clean, quick, and configurable. When I ps uaxww, I’m not hogging ‘gigs’ of RAM just to have things up and running. There’s no black magic that derails me at every turn. In short, my sanity has been long restored.
FreeBSD 12.2 is available, ZFS Webinar, Enhancing Syzkaller support for NetBSD, how the OpenBSD -stable packages are built, OPNsense 20.7.4 released, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThe release notes for FreeBSD 12.2-RELEASE contain a summary of the changes made to the FreeBSD base system on the 12-STABLE development line. This document lists applicable security advisories that were issued since the last release, as well as significant changes to the FreeBSD kernel and userland. Some brief remarks on upgrading are also presented.
\n\n
\n\nZFS Webinar: November 18th
\n\nJoin us on November 18th for a live discussion with Allan Jude (VP of Engineering at Klara Inc) in this webinar centred on “best practices of ZFS”
\n\n
\nBuilding Your Storage Array – Everything from picking the best hardware to RAID-Z and using mirrors.
\nKeeping up with Data Growth – Expanding and growing your pool, and of course, shrinking with device evacuation.
\nDatasets and Properties – Controlling settings with properties and many other tricks!
\n
\n\n\nSys2syz would give an extra edge to Syzkaller for NetBSD. It has a potential of efficiently automating the conversion of syscall definitions to syzkaller’s grammar. This can aid in increasing the number of syscalls covered by Syzkaller significantly with the minimum possibility of manual errors. Let’s delve into its internals.
\n\n
\n
\n\n\nIn this long blog post, I will write about the technical details of the OpenBSD stable packages building infrastructure. I have setup the infrastructure with the help of Theo De Raadt who provided me the hardware in summer 2019, since then, OpenBSD users can upgrade their packages using pkg_add -u for critical updates that has been backported by the contributors. Many thanks to them, without their work there would be no packages to build. Thanks to pea@ who is my backup for operating this infrastructure in case something happens to me.
\n\n
\n
\n\n\nThis release finally wraps up the recent Netmap kernel changes and tests.
\n\n
\nThe Realtek vendor driver was updated as well as third party software cURL,
\nlibxml2, OpenSSL, PHP, Suricata, Syslog-ng and Unbound just to name a couple
\nof them.
\n
bhyve - The FreeBSD Hypervisor, udf information leak, being a vim user instead of classic vi, FreeBSD on ESXi ARM Fling: Fixing Virtual Hardware, new FreeBSD Remote Process Plugin in LLDB, OpenBSD Laptop, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nFreeBSD has had varying degrees of support as a hypervisor host throughout its history. For a time during the mid-2000s, VMWare Workstation 3.x could be made to run under FreeBSD’s Linux Emulation, and Qemu was ported in 2004, and later the kQemu accelerator in 2005. Then in 2009 a port for VirtualBox was introduced. All of these solutions suffered from being a solution designed for a different operating system and then ported to FreeBSD, requiring constant maintenance.
\n\n
\n\nZFS and FreeBSD Support
\n\nKlara offers flexible Support Subscriptions for your ZFS and FreeBSD infrastructure. Get a world class team of experts to back you up. Check it out on our website!
\n
\n\n\nFreeBSD UDF driver info leak
\n\n
\nAnalysis done on FreeBSD release 11.0 because that's what I had around.\n
\n- Fix committed to FreeBSD\n***
\n
\n\n\nIn the past I've written entries (such as this one) where I said that I was pretty much a Vi user, not really a Vim user, because I almost entirely stuck to Vi features. In a comment on my entry on not using and exploring Vim features, rjc reinforced this, saying that I seemed to be using vi instead of vim (and that there was nothing wrong with this). For a long time I thought this way myself, but these days this is not true any more. These days I really want Vim, not classical Vi.
\n\n
\n\nFreeBSD on ESXi ARM Fling: Fixing Virtual Hardware
\n\nWith the current state of FreeBSD on ARM in general, a number of hardware drivers are either set to not auto-load on boot, or are entirely missing altogether. This page is to document my findings with various bits of hardware, and if possible, list fixes.
\n\n
\n\nIntroduction of a new FreeBSD Remote Process Plugin in LLDB
\n\nMoritz Systems have been contracted by the FreeBSD Foundation to modernize the LLDB debugger’s support for FreeBSD. We are writing a new plugin utilizing the more modern client-server layout that is already used by Darwin, Linux, NetBSD and (unofficially) OpenBSD. The new plugin is going to gradually replace the legacy one.
\n
\n\n\nHi, I know it’s been a while. I recently had to nuke and re-pave my personal laptop and I thought it would be a nice thing to share with the community how I set up OpenBSD on it so that I have a useful, modern, secure environment for getting work done. I’m not going to say I’m the expert on this or that this is the BEST way to set up OpenBSD, but I thought it would be worthwhile for folks doing Google searches to at least get my opinion on this. So, given that, let’s go…
\n\n
\n
OpenBSD 6.8 has been released, NetBSD 9.1 is out, OpenZFS devsummit report, BastilleBSD’s native container management for FreeBSD, cleaning up old tarsnap backups, Michael W. Lucas’ book sale, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nReleased Oct 18, 2020. (OpenBSD's 25th anniversary)
\n\n
\n\nNetBSD 9.1 Released
\n\nThe NetBSD Project is pleased to announce NetBSD 9.1, the first update of the NetBSD 9 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.
\n\n
\n
\n\n\nAs with most other conferences in the last six months, this year’s OpenZFS Developer’s Summit was a bit different than usual. Held via Zoom to accommodate for 2020’s new normal in terms of social engagements, the conference featured a mix of talks delivered live via webinars, and breakout sessions held as regular meetings. This helped recapture some of the “hallway track” that would be lost in an online conference.
\n\n
\n • After attending the conference, I wrote up some of my notes from each of the talks
\n • Part 2
\n
Klara offers flexible Support Subscriptions for your ZFS and FreeBSD infrastructure, simply sign up for our monthly subscription! What's even better is that for the month of October we are giving away 3 months for free, for every yearly subscription, and one month free when you sign up for a 6-months subscription! Check it out on our website!
\n\n\n\n\nSome time ago, I had the requirement to use FreeBSD in a project, and soon the question came up if Docker and Kubernetes can be used.
\n
\nOn FreeBSD, Docker is not very well supported, and even if you can get it running, Linux is used in a Docker container. My experience with Docker on FreeBSD is awful, and so I started looking for alternatives.
\nA quick search on one of the most significant online search engines led me to Jails and then to BastilleBSD.
\n\n\nI use Tarsnap for my critical data. Case in point, I use it to backup my Bacula database dump. I use Bacula to backup my hosts. The database in question keeps track of what was backed up, from what host, the file size, checksum, where that backup is now, and many other items. Losing this data is annoying but not a disaster. It can be recreated from the backup volumes, but that is time consuming. As it is, the file is dumped daily, and rsynced to multiple locations.
\n
\n\n\nFor those interested in such things, I recently posted my 60,000th tweet. This prodded me to try an experiment I’ve been pondering for a while.
\n\n
\nOver at my ebookstore, two of my books are now on a “Name Your Own Price” sale. You can get git commit murder and PAM Mastery for any price you wish, with a minimum of $1.
\n
We have an interview with Kyle Evans for you this week. We talk about his grep project, lua and flua in base, as well as bectl, being on the core team and a whole lot of other stuff.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nWayland on BSD, My BSD sucks less than yours, Even on SSDs, ongoing activity can slow down ZFS scrubs drastically, OpenBSD on the Desktop, simple shell status bar for OpenBSD and cwm, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nAfter I posted about the new default window manager in NetBSD I got a few questions, including "when is NetBSD switching from X11 to Wayland?", Wayland being X11's "new" rival. In this blog post, hopefully I can explain why we aren't yet!
\n\n
\n\nMy BSD sucks less than yours
\n\nThis paper will look at some of the differences between the FreeBSD and OpenBSD operating systems. It is not intended to be solely technical but will also show the different "visions" and design decisions that rule the way things are implemented. It is expected to be a subjective view from two BSD developers and does not pretend to represent these projects in any way.
\n\nVideo
\n\n\n
\n\n\nBack in the days of our OmniOS fileservers, which used HDs (spinning rust) across iSCSI, we wound up changing kernel tunables to speed up ZFS scrubs and saw a significant improvement. When we migrated to our current Linux fileservers with SSDs, I didn't bother including these tunables (or the Linux equivalent), because I expected that SSDs were fast enough that it didn't matter. Indeed, our SSD pools generally scrub like lightning.
\n\n
\n\nOpenBSD on the Desktop (Part I)
\n\nLet's install OpenBSD on a Lenovo Thinkpad X270. I used this computer for my computer science studies. It has both Arch Linux and Windows 10 installed as dual boot. Now that I'm no longer required to run Windows, I can ditch the dual boot and install an operating system of my choice.
\n\n
\n\nA simple shell status bar for OpenBSD and cwm(1)
\n\nThese days, I try to use simple and stock software as much as possible on my OpenBSD laptop. I’ve been playing with cwm(1) for weeks and I was missing a status bar. After trying things like Tint2, Polybar etc, I discovered @gonzalo’s termbar. Thanks a lot!
\n\n
\nAs I love scripting, I decided to build my own.
\n\nBeastie Bits
\n\nDragonFly v5.8.3 released to address to issues
\n\n
\nOpenSSH 8.4 released
\n\nTarsnap
\n\n\n
\n- This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
\n
New Project: zedfs.com, TrueNAS CORE Ready for Deployment, IPC in FreeBSD 11: Performance Analysis, Unix Wildcards Gone Wild, Unix Wars, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nHave you ever had an idea that keeps coming back to you over and over again? For a week? For a month? I know that feeling. My new project was born from this feeling.
\n\n
\nOn this blog, I mix content a lot. I have written personal posts (not many of them, but still), FreeBSD development posts, development posts, security posts, and ZFS posts. This mixed content can be problematic sometimes. I share a lot of stuff here, and readers don’t know what to expect next. I am just excited by so many things, and I want to share that excitement with you!
\n
\n\n\nTrueNAS 12.0 RC1 was released yesterday and with it, TrueNAS CORE is ready for deployment. The merger of FreeNAS and TrueNAS into a unified software image can now begin its path into mainstream use. TrueNAS CORE is the new FreeNAS and is on schedule.
\n\n
\nThe TrueNAS 12.0 BETA process started in June and has been the most successful BETA release ever with more than 3,000 users and only minor issues. Ars Technica provided a detailed technical walkthrough of the original BETA. There is a long list of features and performance improvements. During the BETA process, TrueNAS 12.0 demonstrated over 1.2 Million IOPS and over 23GB/s on a TrueNAS M60.
\n
\n\n\nInterprocess communication, IPC, is one of the most fundamental functions of a modern operating system, playing an essential role in the fabric of contemporary applications. This report conducts an investigation in FreeBSD of the real world performance considerations behind two of the most common IPC mechanisms; pipes and sockets. A simple benchmark provides a fair sense of effective bandwidth for each, and analysis using DTrace, hardware performance counters and the operating system’s source code is presented. We note that pipes outperform sockets by 63% on average across all configurations, and further that the size of userspace transmission buffers has a profound effect on performance — larger buffers are beneficial up to a point (∼ 32-64 KiB) after which performance collapses as a result of devastating cache exhaustion. A deep scrutiny of the probe effects at play is also presented, justifying the validity of conclusions drawn from these experiments.
\n\n
\n
\n\n\nFirst of all, this article has nothing to do with modern hacking techniques like ASLR bypass, ROP exploits, 0day remote kernel exploits or Chrome's Chain-14-Different-Bugs-To-Get-There... Nope, nothing of the above. This article will cover one interesting old-school Unix hacking technique, that will still work nowadays in 2013.
\n\n
\n
\n\n\nDozens of different operating systems have been developed over the years, but only Unix has grown in so many varieties. There are three main branches. Four factors have facilitated this growth...
\n\n
\n
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nThe world’s first OpenZFS based live image, FreeBSD Subversion to Git Migration video, FreeBSD Instant-workstation 2020, testing the shutdown mechanism, login_ldap added to OpenBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nFuryBSD is a tool to test drive stock FreeBSD desktop images in read write mode to see if it will work for you before installing. In order to provide the most reliable experience possible while preserving the integrity of the system the LiveCD now leverages ZFS, compression, replication, a memory file system, and reroot (pivot root).
\n\n
\n
\n\n\nFreeBSD moving to Git: Why? With luck, I'll be writing a few blogs on FreeBSD's move to git later this year. Today, we'll start with "why"?
\n\n
\nVideo from Warner Losh
\n
\n\n\nA little over a year ago I published an instant-workstation script for FreeBSD. The idea is to have an installed FreeBSD system, then run a shell script that uses only base-system utilities and installs and configures a workstation setup for you.
\n\n
\n
\n\n\nFollowing on from my recent nut setup, this is the second in a series of three posts.
\n\n
\nThe next post will deal with adjusting startup and shutdown times to be sure everything proceeds as required.
\n
\n\n\nWith this commit, Martijn van Duren (martijn@) added login_ldap(8) to -current
\n\n\n
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nHigh Availability Router/Firewall Using OpenBSD, CARP, pfsync, and ifstated, Building the Development Version of Emacs on NetBSD, rc.d belongs in libexec, not etc, FreeBSD 11.3 EOL, OPNsense 20.7.1 Released, MidnightBSD 1.2.7 out, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI have been running OpenBSD on a Soekris net5501 for my router/firewall since early 2012. Because I run a multitude of services on this system (more on that later), the meager 500Mhz AMD Geode + 512MB SDRAM was starting to get a little sluggish while trying to do anything via the terminal. Despite the perceived performance hit during interactive SSH sessions, it still supported a full 100Mbit connection with NAT, so I wasn’t overly eager to change anything. Luckily though, my ISP increased the bandwidth available on my plan tier to 150Mbit+. Unfortunately, the Soekris only contained 4xVIA Rhine Fast Ethernet. So now, I was using a slow system and wasting money by not being able to fully utilize my connection.
\n
\n\n\nI hadn’t really planned on installing a NetBSD VM (after doing all the other two BSDs), but then a NetBSD-related Emacs bug report arrived.
\n
\n\n\nLet’s open with the controversy: the scripts that live under /etc/rc.d/ in FreeBSD, NetBSD, and OpenBSD are in the wrong place. They all should live in /libexec/rc.d/ because they are code, not configuration.
\n
\nThis misplacement is something that has bugged me for ages but I never had the energy to open this can of worms back when I was very involved in NetBSD. I suspect it would have been a draining discussion and a very difficult thing to change.
\n\n\nAs of September 30, 2020, FreeBSD 11.3 will reach end-of-life and will no longer
\n
\nbe supported by the FreeBSD Security Team. Users of FreeBSD 11.3 are strongly
\nencouraged to upgrade to a newer release as soon as possible.
\n\n\nOverall, the jump to HardenedBSD 12.1 is looking promising from our end. From the reported issues we still have more logging quirks to investigate and especially Netmap support (used in IPS and Sensei) is lacking in some areas that were previously working. Patches are being worked on already so we shall get there soon enough. Stay tuned.
\n
\n\n\nMidnightBSD 1.2.7 is available via the FTP/HTTP and mirrors as well as github.
\n
\nIt includes several bug fixes and security updates over the last ISO release and is recommended for new installations.
\nUsers who don't want to updatee the whole OS, should consider at least updating libmport as there are many package management fixes
Modernizing the OpenBSD Console, OS roles have changed, FreeBSD Cluster with Pacemaker and Corosync, Wine in a 32-bit sandbox on 64-bit NetBSD, Find package which provides a file in OpenBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nAt the beginning were text mode consoles. Traditionally, *BSD and Linux on i386 and amd64 used text mode consoles which by default provided 25 rows of 80 columns, the "80x25 mode". This mode uses a 8x16 font stored in the VGA BIOS (which can be slightly different across vendors).
\n\n
\nOpenBSD uses the wscons(4) console framework, inherited from NetBSD
\n
\n\n\nThough I do wonder sometimes, with just a slight tweak to history, how things might have been different. In another dimension somewhere, I’m using the latest BeOS-powered PowerPC laptop, and a shiny new Palm smartphone. Both of these represented the pinnacle of UI design in the 1990s, and still in the 2020s have yet to be surpassed. People call me an Apple fanboy, but I’d drop all of it in a second for that gear.
\n\n
\n
\n\n\nI always missed ‘proper’ cluster software for FreeBSD systems. Recently I got to run several Pacemaker/Corosync based clusters on Linux systems. I thought how to make similar high availability solutions on FreeBSD and I was really shocked when I figured out that both Pacemaker and Corosync tools are available in the FreeBSD Ports and packages as net/pacemaker2 and net/corosync2 respectively.
\n\n
\n
\n\n\n"Mainline pkgsrc" can't do strange multi-arch Wine builds yet, so a 32-bit sandbox seems like a reasonable way to use 32-bit Wine on amd64 without resorting to running real Windows in NVMM. We'll see if this was a viable alternative to re-reviewing the multi-arch support in pkgsrc-wip...
\n\n
\nWe're using sandboxctl, which is a neat tool for quickly shelling into a different NetBSD userspace. Maybe you also don't trust the Windows applications you're running too much - sandboxctl creates a chroot based on a fresh system image, and chroot on NetBSD is fairly bombproof.
\n
\n\n\nThere is one very handy package on OpenBSD named pkglocatedb which provides the command pkglocate.
\n
\nIf you need to find a file or binary/program and you don’t know which package contains it, use pkglocate.
A 35 Year Old Bug in Patch, Sandbox for FreeBSD, Changing from one dataset to another within a jail, You don’t need tmux or screen for ZFS, HardenedBSD August 2020 Status Report and Call for Donations, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nLarry Wall posted patch 1.3 to mod.sources on May 8, 1985. A number of versions followed over the years. It's been a faithful alley for a long, long time. I've never had a problem with patch until I embarked on the 2.11BSD restoration project. In going over the logs very carefully, I've discovered a bug that bites this effort twice. It's quite interesting to use 27 year old patches to find this bug while restoring a 29 year old OS...
\n
\n\n\nA sandbox is a software which artificially limits access to the specific resources on the target according to the assigned policy. The sandbox installs hooks to the kernel syscalls and other sub-systems in order to interrupt the events triggered by the application. From the application point of view, application working as usual, but when it wants to access, for instance, /dev/kmem the sandbox software decides against the assigned sandbox scheme whether to grant or deny access.
\n
\nIn our case, the sandbox is a kernel module which uses MAC (Mandatory Access Control) Framework developed by the TrustedBSD team. All necessary hooks were introduced to the FreeBSD kernel.
\n\n\nZFS has a the ability to share itself within a jail. That gives the jail some autonomy, and I like that.
\n
\nI’ve written briefly about that, specifically for iocage. More recently, I started using a zfs snapshot for caching clearing.
\nThe purpose of this post is to document the existing configuration of the production FreshPorts webserver and outline the plan on how to modify it for more zfs-snapshot-based cache clearing.
\n\n\nBack in January I mentioned how to add redundancy to a ZFS pool by adding a mirrored drive. Someone with a private account on Twitter asked me why FreeBSD—and NetBSD!—doesn’t ship with a tmux or screen equivilent in base in order to daemonise the process and let them run in the background.
\n
\nZFS already does this for its internal commands.
\n\n\nThis last month has largely been a quiet one. I've restarted work on porting five-year-old work from the Code Pointer Integrity (CPI) project into HardenedBSD. Chiefly, I've started forward-porting the libc and rtld bits from the CPI project and now need to look at llvm compiler/linker enhancements. We need to be able to apply SafeStack to shared objects, not just application binaries. This forward-porting work I'm doing is to support that effort.
\n
\nThe infrastructure has settled and is now churning normally and happily. We're still working out bandwidth issues. We hope to have a new fiber line ran by the end of September.
\nAs part of this status report, I'm issuing a formal call for donations. I'm aiming for $4,000.00 USD for a newer self-hosted Gitea server. I hope to purchase the new server before the end of 2020.
\n\n\nUnix and things that run on Unix have been around for a long time now. In particular, GNU Readline was first released in 1989 (as was Bash), which is long enough ago for it (or lookalikes) to become pretty much pervasive, especially in Unix shells. Today it's easy to think of readline support as something that's always been there. But of course this isn't the case. Unix in its modern form dates from V7 in 1979 and 4.2 BSD in 1983, so a lot of Unix was developed before readline and was to some degree shaped by the lack of it.
\n
OpenZFS with ZSTD lands in FreeBSD 13, LibreSSL doc status update, FreeBSD on SPARC64 (is dead), Bringing zpool checkpoints to a FreeBSD bootloader, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nMore than six years ago, LibreSSL was forked from OpenSSL, and almost two years ago, i explained the status of LibreSSL documentation during EuroBSDCon 2018 in Bucuresti. So it seems providing an update might be in order.
\n
\nNote that this is not an update regarding LibreSSL status in general because i'm not the right person to talk about the big picture of working on the LibreSSL code, my work has been quite focussed on documentation. All the same, it is fair to say that even though the number of developers working on it is somewhat limited, the LibreSSL project is quite alive, typically having a release every few months. Progress continues being made with respect to porting and adding new functionality (for example regarding TLSv1.3, CMS, RSA-PSS, RSA-OAEP, GOST, SM3, SM4, XChaCha20 during the last two years), OpenSSL compatibility improvements (including providing additional OpenSSL-1.1 APIs), and lots of bug fixes and code cleanup.
\n\n\n’m coming pretty late to the party, because SPARC64 support in FreeBSD is apparently doomed: After the POWER platform made the switch to a LLVM/Clang-based toolchain, SPARC64 is one of the last ones that still uses the ancient GCC 4.2-based toolchain that the project wants to finally get rid off (it has already happened as I was writing this – looks like the firm plan was not so firm after all, since they killed it off early). And compared to the other platforms it has seen not too much love in recent times… SPARC64 being a great platform, I’d be quite sad to see it go. But before that happens let’s see what the current status is and what would need to be done if it were to survive, shall we?
\n
\n\n\nAlmost two years ago I wrote a blog post about checkpoints in ZFS. I didn’t hide that I was a big fan of them. That said, after those two years, I still feel that there are underappreciated features in the ZFS world, so I decided to do something about that.
\n
\nCurrently, one of the best practices for upgrading your operating system is to use boot environments. They are a great feature for managing multiple kernels and userlands. They are based on juggling which ZFS datasets are mounted. Each dataset has its own version of the system. Unfortunately, boot environments have their limitations. If we, for example, upgrade our ZFS pool, we may not be able to use older versions of the system anymore.
\nThe big advantage of boot environments is that they have very good tools. Two main tools are beadm (which was created by vermaden) and bectl (which currently is in the FreeBSD base system). These tools allow us to create and manage boot environments.
FreeBSD USB Audio, Kyua: An introduction for NetBSD users, Keeping backup ZFS on Linux kernel modules around, CLI Tools 235x Faster than Hadoop, FreeBSD Laptop Battery Life Status Command, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI recently got a Behringer UMC22 sound card for video conferencing and DJing. This page documents what I’ve learned about using this sound card, and USB audio in general, on FreeBSD.
\n\n
\ntl;dr: Everything works as long as the sound card follows the USB audio device class specification.
\n\nKyua: An introduction for NetBSD users
\n\nKyua's current goal is to reimplement only the ATF tools while maintaining backwards compatibility with the tests written with the ATF libraries (i.e. with the NetBSD test suite).
\n\n
\nBecause Kyua is a replacement of some ATF components, the end goal is to integrate Kyua into the NetBSD base system (just as ATF is) and remove the deprecated ATF components. Removing the deprecated components will allow us to make the above-mentioned improvements to Kyua, as well as many others, without having to deal with the obsolete ATF code base. Discussing how and when this transition might happen is out of the scope of this document at the moment.
\n
\n\n\nI'm a long term user of ZFS on Linux and over pretty much all of the time I've used it, I've built it from the latest development version. Generally this means I update my ZoL build at the same time as I update my Fedora kernel, since a ZoL update requires a kernel reboot anyway. This is a little bit daring, of course, although the ZoL development version has generally been quite solid (and this way I get the latest features and improvements long before I otherwise would).
\n\n
\n
\n\n\nAs I was browsing the web and catching up on some sites I visit periodically, I found a cool article from Tom Hayden about using Amazon Elastic Map Reduce (EMR) and mrjob in order to compute some statistics on win/loss ratios for chess games he downloaded from the millionbase archive, and generally have fun with EMR. Since the data volume was only about 1.75GB containing around 2 million chess games, I was skeptical of using Hadoop for the task, but I can understand his goal of learning and having fun with mrjob and EMR. Since the problem is basically just to look at the result lines of each file and aggregate the different results, it seems ideally suited to stream processing with shell commands. I tried this out, and for the same amount of data I was able to use my laptop to get the results in about 12 seconds (processing speed of about 270MB/sec), while the Hadoop processing took about 26 minutes (processing speed of about 1.14MB/sec).
\n
\n\n\nI know how to find out battery life status using Linux operating system. How do I monitor battery status on a laptop running FreeBSD version 9.x/10.x/11.x/12.x?
\n\n
\nYou can use any one of the following commands to get battery status under FreeBSD laptop including remaining battery life and more.
\n
BSD Beer
\nAwk for JSON
\nDrawing Pictures The Unix Way - with pic and troff
\nRefactoring the FreeBSD Kernel with Checked C
FreeBSD Qt WebEngine GPU Acceleration, the grind of FreeBSD’s wireless stack, thoughts on overlooking Illumos's syseventadm, when Unix learned to reboot, New EXT2/3/4 File-System driver in DragonflyBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nFreeBSD has a handful of Qt WebEngine-based browsers. Falkon, and Otter-Browser, and qutebrowser and probably others, too. All of them can run into issues on FreeBSD with GPU-accelerated rendering not working. Let’s look at some of the workarounds.
\n
\n\n\nThe NanoPi NEO2 from FriendlyARM has been serving me well since 2018, being my test machine for OpenBSD/arm64 related things.
\n
\nAs NetBSD/evbarm finally gained support for AArch64 in NetBSD 9.0, released back in February, I decided to give it a try on this device. The board only has 512MB of RAM, and this is where NetBSD really shines. Things have become a lot easier since jmcneill@ now provides bootable ARM images for a variety of devices, including the NanoPi NEO2.
\n\n\nYes, it's been a while since I posted here and yes, it's been a while since I was actively working on FreeBSD's wireless stack. Life's been .. well, life. I started the ath10k port in 2015. I wasn't expecting it to take 5 years, but here we are. My life has changed quite a lot since 2015 and a lot of the things I was doing in 2015 just stopped being fun for a while.
\n
\nBut the stars have aligned and it's fun again, so here I am.
\n\n\nIn a comment on my praise of ZFS on Linux's ZFS event daemon, Joshua M. Clulow noted that Illumos (and thus OmniOS) has an equivalent in syseventadm, which dates back to Solaris. I hadn't previously known about syseventadm, despite having run Solaris fileservers and OmniOS fileservers for the better part of a decade, and that gives me some tangled feelings.
\n
\n\n\nRecently, a friend asked me the history of halt, and when did we have to stop with the sync / sync / sync dance before running halt or reboot. The two are related, it turns out.
\n
\n\n\nWhile DragonFlyBSD has its own, original HAMMER2 file-system, for those needing to access data from EXT2/EXT3/EXT4 file-systems, there is a brand new "ext2fs" driver implementation for this BSD operating system.
\n
\nDragonFlyBSD has long offered an EXT2 file-system driver (that also handles EXT3 and EXT4) while hitting their Git tree this week is a new version. The new sys/vfs/ext2fs driver, which will ultimately replace their existing sys/gnu/vfs/ext2fs driver is based on a port from FreeBSD code. As such, this driver is BSD licensed rather than GPL. But besides the more liberal license to jive with the BSD world, this new driver has various feature/functionality improvements over the prior version. However, there are some known bugs so for the time being both file-system drivers will co-exist.
Casey - openbsd wirewall
\nDaryl - zfs
\nRaymond - hpe microserver
FreeBSD Q2 Quarterly Status report of 2020, Traditional Unix Toolchains, BastilleBSD 0.7 released, Finding meltdown on DragonflyBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThis report will be covering FreeBSD related projects between April and June, and covers a diverse set of topics ranging from kernel updates over userland and ports, as well to third-party work.
\n\n
\nSome highlights picked with the roll of a d100 include, but are not limited to, the ability to forcibly unmounting UFS when the underlying media becomes inaccessible, added preliminary support for Bluetooth Low Energy, a introduction to the FreeBSD Office Hours, and a repository of software collections called potluck to be installed with the pot utility, as well as many many more things.
\nAs a little treat, readers can also get a rare report from the quarterly team.
\nFinally, on behalf of the quarterly team, I would like to extend my deepest appreciation and thank you to salvadore@, who decided to take down his shingle. His contributions not just the quarterly reports themselves, but also the surrounding tooling to many-fold ease the work, are immeasurable.
\n
\n\n\nOlder Unix systems tend to be fairly uniform in how they handle the so-called 'toolchain' for creating binaries. This blog will give a quick overview of the toolchain pipeline for Unix systems that follow the V7 tradition (which evolved along with Unix, a topic for a separate blog maybe).
\n\n
\nUnix is a pipeline based system, either physically or logically. One program takes input, process the data and produces output. The input and output have some interface they obey, usually text-based. The Unix toolchain is no different.
\n
\n\n\nThis release matures the project from 0.6.x -> 0.7.x. Continued testing and bug fixes are proving Bastille capable for a range of use-cases. New (experimental) features are examples of innovation from community contribution and feedback. Thank you.
\n\n
\n
Interview with Warner Losh about Unix history, the 2.11-BSD restoration project, the Unix heritage society, proper booting, and what devmatch is.
\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nSpecial Guest: Warner Losh.
","summary":"Interview with Warner Losh about Unix history, the 2.11-BSD restoration project, the Unix heritage society, proper booting, and what devmatch is.","date_published":"2020-08-06T08:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/5822b2f7-0440-44f4-8f73-70609c960a3d.mp3","mime_type":"audio/mpeg","size_in_bytes":58166072,"duration_in_seconds":3750}]},{"id":"e7930697-b2c2-4603-b015-19d1070a7c69","title":"361: Function-based MicroVM","url":"https://www.bsdnow.tv/361","content_text":"Emulex: The Cheapest 10gbe for Your Homelab, In Search of 2.11BSD, as released, Fakecracker: NetBSD as a Function Based MicroVM, First powerpc64 snapshots available for OpenBSD, OPNsense 20.1.8 released, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nEmulex: The Cheapest 10gbe for Your Homelab\n\n\nYears ago, the hunt for the cheapest 10gbe NICs resulted in buying Mellanox ConnectX-2 single-port 10gbe network cards from eBay for around $10. Nowadays those cards have increased in cost to around $20-30. While still cheap, not quite the cheapest. There are now alternatives!\nBefore diving into details, let’s get something very clear. If you want the absolute simplest plug-and-play 10gbe LAN for your homelab, pay the extra for Mellanox. If you’re willing to go hands-on, do some simple manual configuration and installation, read on for my experiences with Emulex 10gbe NICs.\nEmulex NICs can often be had for around $15 on eBay, sometimes even cheaper. I recently picked up a set of 4 of these cards, which came bundled with 6 SFP+ 10g-SR modules for a grand total of $47.48. Considering I can usually find SFP+ modules for about $5/ea, these alone were worth $30.\n\n\nI have also tried some Solarflare cards that I found cheap, they work ok, but are pickier about optics, and tend to be focused on low-latency, so often don’t manage to saturate the full 10 gbps, topping out around 8 gbps.\nI have been using fs.com for optics, patch cables, and DACs. I find DACs are usually cheaper if you are just going between a server and a switch in the same rack, or direct between 2 servers.\n***\n\n\n\nIn Search of 2.11BSD, as released\n\n\nAlmost all of the BSD releases have been well preserved. If you want to find 1BSD, or 2BSD or 4.3-TAHOE BSD you can find them online with little fuss. However, if you search for 2.11BSD, you'll find it easily enough, but it won't be the original. You'll find either the latest patched version (2.11BSD pl 469), or one of the earlier popular version (pl 430 is popular). You can even find the RetroBSD project which used 2.11BSD as a starting point to create systems for tiny mips-based PIC controllers. You'll find every single patch that's been issued for the system.\n\n\n\n\nNews Roundup\n\nFakecracker: NetBSD as a Function Based MicroVM\n\n\nIn November 2018 AWS published an Open Source tool called Firecracker, mostly a virtual machine monitor relying on KVM, a small sized Linux kernel, and a stripped down version of Qemu. What baffled me was the speed at which the virtual machine would fire up and run the service. The whole process is to be compared to a container, but safer, as it does not share the kernel nor any resource, it is a separate and dedicated virtual machine.\nIf you want to learn more on Firecracker‘s internals, here’s a very well put article.\n\n\n\n\nFirst powerpc64 snapshots available for OpenBSD\n\n\nSince we reported the first bits of powerpc64 support going into the tree on 16 May, work has progressed at a steady pace, resulting in snapshots now being available for this platform.\nSo, if you have a POWER9 system idling around, go to your nearest mirror and fetch this snapshot. Keep in mind that as this is still very early days, very little handholding is available - you are basically on your own.\n\n\n\n\nOPNsense 20.1.8 released\n\n\nSorry about the delay while we chased a race condition in the updates back to an issue with the latest FreeBSD package manager updates. For now we reverted to our current version but all relevant third party packages have been updated as updates became available over the last weeks, e.g. cURL and Python, and hostapd / wpa_supplicant amongst others.\n\n\n\n\nBeastie Bits\n\n\nOld School Disk Partitioning\nNomad BSD 1.3.2 Released\nChai-Fi\n\n\n\n\nTarsnap\n\n\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\n\n\nFeedback/Questions\n\n\nPoojan - ZFS Question\ngraceon - supermicro\nzenbum - groff\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\nSpecial Guest: Warner Losh.","content_html":"Emulex: The Cheapest 10gbe for Your Homelab, In Search of 2.11BSD, as released, Fakecracker: NetBSD as a Function Based MicroVM, First powerpc64 snapshots available for OpenBSD, OPNsense 20.1.8 released, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nYears ago, the hunt for the cheapest 10gbe NICs resulted in buying Mellanox ConnectX-2 single-port 10gbe network cards from eBay for around $10. Nowadays those cards have increased in cost to around $20-30. While still cheap, not quite the cheapest. There are now alternatives!
\n\n
\nBefore diving into details, let’s get something very clear. If you want the absolute simplest plug-and-play 10gbe LAN for your homelab, pay the extra for Mellanox. If you’re willing to go hands-on, do some simple manual configuration and installation, read on for my experiences with Emulex 10gbe NICs.
\nEmulex NICs can often be had for around $15 on eBay, sometimes even cheaper. I recently picked up a set of 4 of these cards, which came bundled with 6 SFP+ 10g-SR modules for a grand total of $47.48. Considering I can usually find SFP+ modules for about $5/ea, these alone were worth $30.\n
\n- I have also tried some Solarflare cards that I found cheap, they work ok, but are pickier about optics, and tend to be focused on low-latency, so often don’t manage to saturate the full 10 gbps, topping out around 8 gbps.
\n- I have been using fs.com for optics, patch cables, and DACs. I find DACs are usually cheaper if you are just going between a server and a switch in the same rack, or direct between 2 servers.\n***
\n
\n\n\nAlmost all of the BSD releases have been well preserved. If you want to find 1BSD, or 2BSD or 4.3-TAHOE BSD you can find them online with little fuss. However, if you search for 2.11BSD, you'll find it easily enough, but it won't be the original. You'll find either the latest patched version (2.11BSD pl 469), or one of the earlier popular version (pl 430 is popular). You can even find the RetroBSD project which used 2.11BSD as a starting point to create systems for tiny mips-based PIC controllers. You'll find every single patch that's been issued for the system.
\n\n
\n
\n\n\nIn November 2018 AWS published an Open Source tool called Firecracker, mostly a virtual machine monitor relying on KVM, a small sized Linux kernel, and a stripped down version of Qemu. What baffled me was the speed at which the virtual machine would fire up and run the service. The whole process is to be compared to a container, but safer, as it does not share the kernel nor any resource, it is a separate and dedicated virtual machine.
\n\n
\nIf you want to learn more on Firecracker‘s internals, here’s a very well put article.
\n
\n\n\nSince we reported the first bits of powerpc64 support going into the tree on 16 May, work has progressed at a steady pace, resulting in snapshots now being available for this platform.
\n\n
\nSo, if you have a POWER9 system idling around, go to your nearest mirror and fetch this snapshot. Keep in mind that as this is still very early days, very little handholding is available - you are basically on your own.
\n
\n\n\nSorry about the delay while we chased a race condition in the updates back to an issue with the latest FreeBSD package manager updates. For now we reverted to our current version but all relevant third party packages have been updated as updates became available over the last weeks, e.g. cURL and Python, and hostapd / wpa_supplicant amongst others.
\n\n
\n
Special Guest: Warner Losh.
","summary":"Emulex: The Cheapest 10gbe for Your Homelab, In Search of 2.11BSD, as released, Fakecracker: NetBSD as a Function Based MicroVM, First powerpc64 snapshots available for OpenBSD, OPNsense 20.1.8 released, and more.\r\n","date_published":"2020-07-30T07:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/e7930697-b2c2-4603-b015-19d1070a7c69.mp3","mime_type":"audio/mpeg","size_in_bytes":64248344,"duration_in_seconds":3730}]},{"id":"69d88af7-54da-4612-9fc2-84ffae001c46","title":"360: Full circle","url":"https://www.bsdnow.tv/360","content_text":"Chasing a bad commit, New FreeBSD Core Team elected, Getting Started with NetBSD on the Pinebook Pro, FreeBSD on the Intel 10th Gen i3 NUC, pf table size check and change, and more.\n\nNOTES\nThis episode of BSDNow is brought to you by Tarsnap\n\nHeadlines\n\nChasing a bad commit\n\n\nWhile working on a big project where multiple teams merge their feature branches frequently into a release Git branch, developers often run into situations where they find that some of their work have been either removed, modified or affected by someone else's work accidentally. It can happen in smaller teams as well. Two features could have been working perfectly fine until they got merged together and broke something. That's a highly possible case. There are many other cases which could cause such hard to understand and subtle bugs which even continuous integration (CI) systems running the entire test suite of our projects couldn't catch.\nWe are not going to discuss how such subtle bugs can get into our release branch because that's just a wild territory out there. Instead, we can definitely discuss about how to find a commit that deviated from an expected outcome of a certain feature. The deviation could be any behaviour of our code that we can measure distinctively — either good or bad in general.\n\n\n\n\nNew FreeBSD Core Team Elected\n\n\nThe FreeBSD Project is pleased to announce the completion of the 2020 Core Team election. Active committers to the project have elected your Eleventh FreeBSD Core Team.!\n\n\n\nBaptiste Daroussin (bapt)\nEd Maste (emaste)\nGeorge V. Neville-Neil (gnn)\nHiroki Sato (hrs)\nKyle Evans (kevans)\nMark Johnston (markj)\nScott Long (scottl)\nSean Chittenden (seanc)\nWarner Losh (imp)\n***\n\n\nNews Roundup\n\nGetting Started with NetBSD on the Pinebook Pro\n\n\nIf you buy a Pinebook Pro now, it comes with Manjaro Linux on the internal eMMC storage. Let’s install NetBSD instead!\nThe easiest way to get started is to buy a decent micro-SD card (what sort of markings it should have is a science of its own, by the way) and install NetBSD on that. On a warm boot (i.e. when rebooting a running system), the micro-SD card has priority compared to the eMMC, so the system will boot from there.\n\n\nA FreeBSD developer has borrowed some of the NetBSD code to get audio working on RockPro64 and Pinebook Pro: https://twitter.com/kernelnomicon/status/1282790609778905088\n***\n\n\n\nFreeBSD on the Intel 10th Gen i3 NUC\n\n\nI have ended up with some 10th Gen i3 NUC's (NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020. Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is.\n\n\n\n\npf table size check and change\n\n\nDid you know there’s a default size limit to pf’s state table? I did not, but it makes sense that there is one. If for some reason you bump into this limit (difficult for home use, I’d think), here’s how you change it\nThere is a table-entries limit specified, you can see current settings with\n'pfctl -s all'. You can adjust the limits in the /etc/pf.conf file\ncontaining the rules with a line like this near the top:\nset limit table-entries 100000\n\n\nIn the original mail thread, there is mention of the FreeBSD sysctl net.pf.request_maxcount, which controls the maximum number of entries that can be sent as a single ioctl(). This allows the user to adjust the memory limit for how big of a list the kernel is willing to allocate memory for.\n***\n\n\n\nBeastie Bits\n\n\ntmux and bhyve\nAzure and FreeBSD\nGroff Tutorial\n***\n###Tarsnap\nThis weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.\nTarsnap Mastery\n\n\nFeedback/Questions\n\n\nChris - ZFS Question\nPatrick - Tarsnap\nPin - pkgsrc\n***\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n***\n","content_html":"Chasing a bad commit, New FreeBSD Core Team elected, Getting Started with NetBSD on the Pinebook Pro, FreeBSD on the Intel 10th Gen i3 NUC, pf table size check and change, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nWhile working on a big project where multiple teams merge their feature branches frequently into a release Git branch, developers often run into situations where they find that some of their work have been either removed, modified or affected by someone else's work accidentally. It can happen in smaller teams as well. Two features could have been working perfectly fine until they got merged together and broke something. That's a highly possible case. There are many other cases which could cause such hard to understand and subtle bugs which even continuous integration (CI) systems running the entire test suite of our projects couldn't catch.
\n
\nWe are not going to discuss how such subtle bugs can get into our release branch because that's just a wild territory out there. Instead, we can definitely discuss about how to find a commit that deviated from an expected outcome of a certain feature. The deviation could be any behaviour of our code that we can measure distinctively — either good or bad in general.
\n\n\nThe FreeBSD Project is pleased to announce the completion of the 2020 Core Team election. Active committers to the project have elected your Eleventh FreeBSD Core Team.!
\n
\n\n\nIf you buy a Pinebook Pro now, it comes with Manjaro Linux on the internal eMMC storage. Let’s install NetBSD instead!
\n\n
\nThe easiest way to get started is to buy a decent micro-SD card (what sort of markings it should have is a science of its own, by the way) and install NetBSD on that. On a warm boot (i.e. when rebooting a running system), the micro-SD card has priority compared to the eMMC, so the system will boot from there.\n
\n- A FreeBSD developer has borrowed some of the NetBSD code to get audio working on RockPro64 and Pinebook Pro: https://twitter.com/kernelnomicon/status/1282790609778905088\n***
\n
\n\n\nI have ended up with some 10th Gen i3 NUC's (NUC10i3FNH to be specific) to put to work in my testbed. These are quite new devices, the build date on the boxes is 13APR2020. Before I figure out what their true role is (one of them might have to run linux) I need to install FreeBSD -CURRENT and see how performance and hardware support is.
\n
\n\n\nDid you know there’s a default size limit to pf’s state table? I did not, but it makes sense that there is one. If for some reason you bump into this limit (difficult for home use, I’d think), here’s how you change it
\n\n
\nThere is a table-entries limit specified, you can see current settings with
\n'pfctl -s all'. You can adjust the limits in the /etc/pf.conf file
\ncontaining the rules with a line like this near the top:
\nset limit table-entries 100000
\n
\n- In the original mail thread, there is mention of the FreeBSD sysctl net.pf.request_maxcount, which controls the maximum number of entries that can be sent as a single ioctl(). This allows the user to adjust the memory limit for how big of a list the kernel is willing to allocate memory for.\n***
\n
Throw-Away Browser on FreeBSD With "pot" within 5 minutes, OmniOS as OpenBSD guest with bhyve, BSD vs Linux distro development, My FreeBSD Laptop Build, FreeBSD CURRENT Binary Upgrades, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\npot is a great and relatively new jail management tool. It offers DevOps style provisioning and can even be used to provide Docker-like, scalable cloud services together with nomad and consul (more about this in Orchestrating jails with nomad and pot).
\n
\n\n\nToday I will be creating a OpenBSD guest via bhyve on OmniOS. I will also be adding a Pass Through Ethernet Controller so I can have a multi-homed guest that will serve as a firewall/router.
\n
\nThis post will cover setting up bhyve on OmniOS, so it will also be a good introduction to bhyve. As well, I look into OpenBSD’s uEFI boot loader so if you have had trouble with this, then you are in the right place.
\n\n\nQ: Comparing-apples-to-BSDs asks: I was reading one of the old articles from the archive. One of the things mentioned was how the BSDs have a distinct approach in terms of packaging the base system relative to userland apps, and that the Linux distros at the time were not following the same practice. Are there Linux distros that have adopted the same approach in modern times? If not, are there technical limitations that are preventing them from doing so, such as some distros supporting multiple kernel versions maybe?
\n\n
\nDistroWatch answers: In the article mentioned above, I made the observation that Linux distributions tend to take one of two approaches when it comes to packaging software. Generally a Linux distribution will either offer a rolling release, where virtually all packages are regularly upgraded to their latest stable releases, or a fixed release where almost all packages are kept at a set version number and only receive bug fixes for the life cycle of the distribution. Projects like Arch Linux and Void are popular examples of rolling, always-up-to-date distributions while Fedora and Ubuntu offer fixed platforms.
\n\nMy FreeBSD Laptop Build
\n\nI have always liked Thinkpad hardware and when I started to do more commuting I decided I needed something that had a decent sized screen but fit well on a bus. Luckily about this time Lenovo gave me a nice gift in the Thinkpad X390. Its basically the famous X2xx series but with a 13” screen and smaller bezel.
\n\n
\nSo with this laptop I figured it was time to actually put the docs together on how I got my FreeBSD workstation working on it. I will here in the near future have another post that will cover this for HardenedBSD as well since the steps are similar but have a few extra gotchas due to the extra hardening.
\n\nFreeBSD CURRENT Binary Upgrades
\n\n\n
\n- Disclaimer\nThis proof-of-concept is not a publication of FreeBSD.
\n- Description\nup.bsd.lv is a proof-of-concept of binary updates for FreeBSD/amd64 CURRENT/HEAD to facilitate the exhaustive testing of FreeBSD and the bhyve hypervisor and OpenZFS 2.0 specifically. Updates are based on the SVN revisions of official FreeBSD Release Engineering bi-monthly snapshots.
\n
lars - openbsd router hardware
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nYubikey-agent on FreeBSD, Managing Kubernetes clusters from OpenBSD, History of FreeBSD part 1, Running Jitsi-Meet in a FreeBSD Jail, Command Line Bug Hunting in FreeBSD, Game of Github, Wireguard official merged into OpenBSD, and more
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nSome time ago Filippo Valsorda wrote yubikey-agent, seamless SSH agent for YubiKeys. I really like YubiKeys and worked on the FreeBSD support for U2F in Chromium and pyu2f, getting yubikey-agent ported looked like an interesting project. It took some hacking to make it work but overall it wasn’t hard. Following is the roadmap on how to get it set up on FreeBSD. The actual details depend on your system (as you will see)
\n\n
\n
\n\n\nThis should work with OpenBSD 6.7. I write this while the source tree is locked for release, so even if I use -current this is as close as -current gets to -release
\n\n
\nUpdate 2020-06-05: we now have a port for kubectl. So, at least in -current things get a bit easier.
\n
\n\n\nFreeBSD, a free and open-source Unix-like operating system has been around since 1993. However, its origins are directly linked to that of BSD, and further back, those of Unix. During this History of FreeBSD series, we will talk about how Unix came to be, and how Berkeley’s Unix developed at Bell Labs.
\n\n
\n
\n\n\nDue to the situation with COVID-19 that also lead to people being confined to their homes in South Africa as well, we decided to provide a (freely usable of course) Jitsi Meet instance to the community being hosted in South Africa on our FreeBSD environment.
\n\n
\nThat way, communities in South Africa and beyond have a free alternative to the commercial conferencing solutions with sometimes dubious security and privacy histories and at the same time improved user experience due to the lower latency of local hosting.\n
\n- Grafana for Jitsi-Meet\n***
\n
\n\n\nFreeBSD uses bugzilla for tracking bugs, taking feature requests, regressions and issues in the Operating System. The web interface for bugzilla is okay, but if you want to do a lot of batch operations it is slow to deal with. We are planning to run a bugsquash on July 11th and that really needs some tooling to help any hackers that show up process the giant bug list we have.
\n\n
\n
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nOpenBSD 6.7 on PC Engines, NetBSD code study, DRM Update on OpenBSD, Booting FreeBSD on HPE Microserver SATA port, 3 ways to multiboot, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nI just got myself a PC Engines APU4D4. I miss an OpenBSD box providing home services. It’s quite simple to install and run OpenBSD on this machine. And you can even update the BIOS from OpenBSD.
\n\n
\n\nNetBSD code study
\n\n
\n
\n\n\nMy small homelab post generated a ton of questions and comments, most of them specific to running FreeBSD on the HP MicroServer. I’ll try and answer these over the coming week.
\n\n
\nJosh Paxton emailed to ask how I got FreeBSD booting on it, given the unconventional booting limitations of the hardware. I thought I wrote about it a few years ago, but maybe it’s on my proverbial draft heap. If you’re impatient, the script is in my lunchbox.
\n
\n\n\nmultiboot installation of a BSD system with other operating systems
\n\n
\n(OSs) on UEFI hardware is not officially supported by any of the
\npopular
\n
Michael - Jordyns ZFS Question
\n\nTrueNAS is Multi-OS, Encrypted ZFS on NetBSD, FreeBSD’s new Code of Conduct, Gaming on OpenBSD, dig a little deeper, Hammer2 and periodic snapshots, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThere was a time in history where all that mattered was an Operating System (OS) and the hardware it ran on — the “pre-software era”, if you will. Your hardware dictated the OS you used.
\n\n
\nOnce software applications became prominent, your hardware’s OS determined the applications you could run. Application vendors were forced to juggle the burden of “portability” between OS platforms, choosing carefully the operating systems they’d develop their software to. Then, there were the great OS Wars of the 1990s, replete with the rampant competition, licensing battles, and nasty lawsuits, which more or less gave birth to the “open source OS” era.
\nThe advent of the hypervisor simultaneously gave way to the “virtual era” which set us on a path of agnosticism toward the OS. Instead of choosing from the applications available for your chosen OS, you could simply install another OS on the same hardware for your chosen application. The OS became nothing but a necessary cog in the stack.
\nTrueNAS open storage enables this “post-OS era” with support for storage clients of all UNIX flavors, Linux, FreeBSD, Windows, MacOS, VMware, Citrix, and many others. Containerization has carried that mentality even further. An operating system, like the hardware that runs it, is now just thought of as part of the “infrastructure”.
\n\nEncrypted ZFS on NetBSD 9.0, for a FreeBSD guy
\n\nI had one of my other HP Microservers brought back from the office last week to help with this working-from-home world we’re in right now. I was going to wipe an old version of Debian Wheezy/Xen and install FreeBSD to mirror my other machines before thinking: why not NetBSD?
\n\n
\n
\n\n\nWhile no one would expect this, there are huge efforts from a small team to bring more games into OpenBSD. In fact, now some commercial games works natively now, thanks to Mono or Java. There are no wine or linux emulation layer in OpenBSD.
\n\n
\nHere is a small list of most well known games that run on OpenBSD:
\n\n'dig' a little deeper
\n\nI knew the existence of the dig command but didn't exactly know when and how to use it. Then, just recently I encountered an issue that allowed me to learn and make use of it.
\n\n
\n\nHAMMER2 and periodic snapshots
\n\nThe first version of HAMMER took automatic snapshots, set within the config for each filesystem. HAMMER2 now also takes automatic snapshots, via periodic(8) like most every repeating task on your DragonFly system.
\n\n\n
Upgrading OpenBSD, Where do Unix man pages come from?, Help for NetBSD’s VAX port, FreeBSD on Dell Latitude 7390, PFS Tool changes in DragonflyBSD, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nLet's see how to upgrade your OpenBSD system. Maybe you are doing this because the latest release just came out. If so, this is pretty simple: back up your data, boot from install media, and select "Upgrade" instead of "Install". But maybe the latest release has been out for a few months. Why would we go through the trouble of building and installing a new kernel or other core system components? Maybe some patches have been released to improve system security or stability. It is pretty easy to build and install a kernel on OpenBSD, easier and simpler in many ways than it is on Linux.
\n
\n\n\nWhere do UNIX manpages come from? Who introduced the section-based layout of NAME, SYNOPSIS, and so on? And for manpage authors: where were those economical two- and three-letter instructions developed?
\n\n
\n
\n\n\nThe VAX is the oldest machine architecture still supported by NetBSD.
\n\n
\nUnfortunately there is another challenge, totally outside of NetBSD, but affecting the VAX port big time: the compiler support for VAX is ... let's say sub-optimal. It is also risking to be dropped completely by gcc upstream.
\nNow here is where people can help: there is a bounty campaign to finance a gcc hacker to fix the hardest and most immediate issue with gcc for VAX. Without this being resolved, gcc will drop support for VAX in a near future version.
\n
\n\n\nAs a FreeBSD developer, I make a point of using FreeBSD whenever I can — including on the desktop. I've been running FreeBSD on laptops since 2004; this hasn't always been easy, but over the years I've found that the situation has generally been improving. One of the things we still lack is adequate documentation, however — so I'm writing this to provide an example for users and also Google bait in case anyone runs into some of the problems I had to address.
\n\n
\n
\n\n\nHAMMER2 just became a little more DWIM: the pfs-list and pfs-delete directives will now look across all mounted filesystems, not just the current directory’s mount path. pfs-delete won’t delete any filesystem name that appears in more than one place, though
\n\n\n
\n- git: hammer2 - Enhance pfs-list and pfs-delete\nEnhance pfs-list to list PFSs available across all mounted hammer2 filesystems instead of just the current directory's mount. A specific mount may be specified via -s mountpt.\nEnhance pfs-delete to look for the PFS name across all mounted hammer2 filesystems instead of just the current directory's mount.\nAs a safety, pfs-delete will refuse to delete PFS names which are duplicated across multiple mounts. A specific mount may be specified via -s mountpt.
\n
FreeBSD 11.4-RC 2 available, OpenBSD 6.7 on a PineBook Pro 64, How OpenZFS Keeps Your Data Safe, Bringing FreeBSD to EC2, FreeBSD 2020 Community Survey, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nThe second RC build of the 11.4-RELEASE release cycle is now available.
\n\n\n
\n- 11.4-RELEASE notes (still in progress at the time of recording)\n***
\n
\n\n\nThis document is work in progress and I'll update the date above once I change something. If you have something to add, remarks, etc please contact me. Preferably via Mastodon but other means of communication are also fine.
\n\n
\n
\n\n\nVeteran technology writer Jim Salter wrote an excellent guide on the ZFS file system’s features and performance that we absolutely had to share. There’s plenty of information in the article for ZFS newbies and advanced users alike. Be sure to check out the article over at Ars Technica to learn more about ZFS concepts including pools, vdevs, datasets, snapshots, and replication, just to name a few.
\n\n
\n
\n\n\nColin is the founder of Tarsnap, a secure online backup service which combines the flexibility and scriptability of the standard UNIX "tar" utility with strong encryption, deduplication, and the reliability of Amazon S3 storage. Having started work on Tarsnap in 2006, Colin is among the first generation of users of Amazon Web Services, and has written dozens of articles about his experiences with AWS on his blog.
\n\n
\n
\n\n\nThe FreeBSD Core Team invites you to complete the 2020 FreeBSD Community Survey. The purpose of this survey is to collect quantitative data from the public in order to help guide the project’s priorities and efforts. This is only the second time a survey has been conducted by the FreeBSD Project and your input is valued.
\n\n
\nThe survey will remain open for 14 days and will close on June 16th at 17:00 UTC (Tuesday 10am PDT).
\n
Morgan - Can I get some commentary on this issue
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nSponsored By:
Scheduling in NetBSD, ZFS vs. RAID on Ironwolf disks, OpenBSD on Microsoft Surface Go 2, FreeBSD for Linux sysadmins, FreeBSD on Lenovo T480, and more.
\n\nNOTES
\nThis episode of BSDNow is brought to you by Tarsnap
\n\n\nIn this blog, we will discuss about the 4.4BSD Thread scheduler one of the two schedulers in NetBSD and a few OS APIs that can be used to control the schedulers and get information while executing.
\n
\n\n\nThis has been a long while in the making—it's test results time. To truly understand the fundamentals of computer storage, it's important to explore the impact of various conventional RAID (Redundant Array of Inexpensive Disks) topologies on performance. It's also important to understand what ZFS is and how it works. But at some point, people (particularly computer enthusiasts on the Internet) want numbers.
\n
\n\n\nI used OpenBSD on the original Surface Go back in 2018 and many things worked with the big exception of the internal Atheros WiFi. This meant I had to keep it tethered to a USB-C dock for Ethernet or use a small USB-A WiFi dongle plugged into a less-than-small USB-A-to-USB-C adapter.
\n
\n\n\nIf you’ve ever installed and explored another Linux distro (what Linux sysadmin hasn’t?!?), then exploring FreeBSD is going be somewhat similar with a few key differences.
\n
\nWhile there is no graphical installation, the installation process is straightforward and similar to installing a server-based Linux distro. Just make sure you choose the local_unbound package when prompted if you want to cache DNS lookups locally, as FreeBSD doesn’t have a built-in local DNS resolver that does this.
\nFollowing installation, the directory structure is almost identical to Linux. Of course, you’ll notice some small differences here and there (e.g. regular user home directories are located under /usr/home instead of /home). Standard UNIX commands such as ls, chmod, find, which, ps, nice, ifconfig, netstat, sockstat (the ss command in Linux) are exactly as you’d expect, but with some different options here and there that you’ll see in the man pages. And yes, reboot and poweroff are there too.
\n\n\nRecently I replaced my 2014 MacBook Air with a Lenovo Thinkpad T480, on which I've installed FreeBSD, currently 12.1-RELEASE. This page documents my set-up along with various configuration tweaks and fixes.
\n
Sponsored By:
A brief introduction to randomness, logs grinding netatalk to a halt, NetBSD core team changes, Using qemu guest agent on OpenBSD kvm/qemu guests, WireGuard patchset for OpenBSD, FreeBSD 12.1 on a laptop, and more.
\n\n\n\n\n\n\nA brief introduction to randomness
\n
\n\n\nBut what if we want them to act unpredictably? This is very useful if we want to secure our private communications with randomized keys, or not let people cheat at video games, or if we're doing statistical simulations or similar.
\n
\n\n\n\n\nI’ve heard it said the cobbler’s children walk barefoot. While posessing the qualities of a famed financial investment strategy, it speaks to how we generally put more effort into things for others than ourselves; at least in business.
\n
\nThe HP Microserver I share with Clara is a modest affair compared to what we run at work. It has six spinning rust drives and two SSDs which are ZFS-mirrored; not even in a RAID 10 equivalent. This is underlaid with GELI for encryption, and served to our Macs with Netatalk over gigabit Ethernet with jumbo frames.
\n\n\nMatt Thomas (matt@) has served on the NetBSD core team for over ten years, and has made many contributions, including ELF functionality, being the long-time VAX maintainer, gcc contributor, the generic pmap, and also networking functionality, and platform bring-up over the years. Matt has stepped down from the NetBSD core team, and we thank him for his many, extensive contributions.
\n\n
\nRobert Elz (kre@), a long time BSD contributor, has kindly accepted the offer to join the core team, and help us out with the benefit of his experience and advice over many years. Amongst other things, Robert has been maintaining our shell, liaising with the Austin Group, and bringing it up to date with modern functionality.
\n
\n\n\nIn a post to the ports@ mailing list, Landry Breuil (landry@) shared some of his notes on using qemu guest agent on OpenBSD kvm/qemu guests.
\n
\n\n\nA while ago I wanted to learn more about OpenBSD development. So I picked a project, in this case WireGuard, to develop a native client for. Over the last two years, with many different iterations, and working closely with the WireGuard's creator (Jason [Jason A. Donenfeld - Ed.], CC'd), it started to become a serious project eventually reaching parity with other official implementations. Finally, we are here and I think it is time for any further development to happen inside the src tree.
\n\n
\n
\n\n\nI’m using FreeBSD again on a laptop for some reasons so expect to read more about FreeBSD here. This tutorial explain how to get a graphical desktop using FreeBSD 12.1.
\n\n
\n
Backup and Restore on NetBSD, OpenBSD 6.7 available, Building a WireGuard Jail with FreeBSD's standard tools, who gets to chown things and quotas, influence TrueNAS CORE roadmap, and more.
\n\n\n\n\nPutting together the bits and pieces of a backup and restore concept, while not being rocket science, always seems to be a little bit ungrateful. Most Admin Handbooks handle this topic only within few pages. After replacing my old Mac Mini's OS by NetBSD, I tried to implement an automated backup, allowing me to handle it similarly to the time machine backups I've been using before. Suggestions on how to improve are always welcome.
\n
\n\n\n\n\nThe OpenBSD project produces and operating system which places focus on portability, standardisation, code correctness, proactive security and integrated cryptography. The project's latest release is OpenBSD 6.7 which introduces several new improvements to the cron scheduling daemon, improvements to the web server daemon, and the top command now offers scrollable output. These and many more changes can be found in the project's release announcement: "This is a partial list of new features and systems included in OpenBSD 6.7. For a comprehensive list, see the changelog leading to 6.7. General improvements and bugfixes: Reduced the minimum allowed number of chunks in a CONCAT volume from 2 to 1, increasing the number of volumes which can be created on a single disk with bioctl(8) from 7 to 15. This can be used to create more partitions than previously. Rewrote the cron(8) flag-parsing code to be getopt-like, allowing tight formations like -ns and flag repetition. Renamed the 'options' field in crontab(5) to 'flags'. Added crontab(5) -s flag to the command field, indicating that only a single instance of the job should run concurrently. Added cron(8) support for random time values using the ~ operator. Allowed cwm(1) configuration of window size based on percentage of the master window during horizontal and vertical tiling actions."
\n
\n\n\nRecently, I had an opportunity to build a WireGuard jail on a FreeBSD 12.1 host.
\n
\nAs it was really quick and easy to setup and it has been working completely fine for a month, I’d like to share my experience with anyone interested in this topic.
\n\n\nOne of the famous big splits between the BSD Unix world and the System V world is whether ordinary users can use chown (the command and the system call) to give away their own files. In System V derived Unixes you were generally allowed to; in BSD derived Unixes you weren't. Until I looked it up now to make sure, I thought that BSD changed this behavior from V7 and that V7 had an unrestricted chown. However, this turns out to be wrong; in V7 Unix, chown(2) was restricted to root only.
\n
\n\n\nAs many of you know, we’ve historically had three ticket types available in our tracker: Bugs, Features, and Improvements, which are all fairly self-explanatory. After some discussion internally, we’ve decided to implement a new type of ticket, a “Suggestion”. These will be replacing Feature and Improvement requests for the TrueNAS Community, simplifying things down to two options: Bugs and Suggestions. This change also introduces a slightly different workflow than before.
\n\n
\n
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
5x if_bridge Performance Improvement, How Unix Won, Understanding VLAN Configuration on FreeBSD, Using bhyve PCI passthrough on OmniOS, TrueNAS 11.3-U2 Available, and more.
\n\n\n\n\nWith FreeBSD Foundation grant, Kristof Provost harnesses new parallel techniques to uncork performance bottleneck
\n\n\n
\n- Kristof also streamed some of his work, providing an interesting insight into how such development work happens
\n- > https://www.twitch.tv/provostk/videos\n***
\n
+> Unix has won in every conceivable way. And in true mythic style, it contains the seeds of its own eclipse. This is my subjective historical narrative of how that happened.
\n\n\n\n\nI’m using the name “Unix” to include the entire family of operating systems descended from it, or that have been heavily influenced by it. That includes Linux, SunOS, Solaris, BSD, Mac OS X, and many, many others.
\n\n
\nBoth major mobile OSs, Android and iOS, have Unix roots. Their billions of users dwarf those using clunky things like laptops and desktops, but even there, Windows is only the non-Unix viable OS. Almost everything running server-side in giant datacenters is Linux.
\nHow did Unix win?
\n
\n\n\nThis blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not to difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host. And the solution presented in this blog post works pretty well for me!
\n\n
\n
\n\n\nUntil recently, I’ve never had a chance to use VLANs on FreeBSD hosts, though I sometimes configure them on ethernet switches.
\n\n
\nBut when I was playing with vnet jails, I suddenly got interested in VLAN configuration on FreeBSD and experimented with it for some time.
\nI wrote this short article to summarize my current understanding of how to configure VLANs on FreeBSD.
\n
\n\n\nSome hardware is not supported in illumos yet, but luckily there is bhyve which supports pci passthrough to any guest operating system. To continue with my OmniOS desktop on "modern" hardware I would love wifi support, so why not using a bhyve guest as router zone which provide the required drivers?
\n\n
\n
\n\n\nTrueNAS 11.3-U2.1 is generally available as of 4/22/2020. This update is based on FreeNAS 11.3-U2 which has had over 50k deployments and received excellent community and third party reviews. The Release Notes are available on the iXsystems.com website.
\n\n
\n
HardenedBSD April 2020 Status Report
\nNYC Bug’s Mailing List - Listing of open Dev Jobs
Morgan - Performance
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
\n\nEncrypted Crash Dumps in FreeBSD, Time on Unix, Improve ZVOL sync write performance with a taskq, central log host with syslog-ng, NetBSD Entropy overhaul, Setting Up NetBSD Kernel Dev Environment, and more.
\n\n\n\n\nSome time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features.
\n\nThe main issue with crash dumps is that they may include sensitive information available in memory during a crash. They will contain all the data from the kernel and the userland, like passwords, private keys, etc. While dumping them, they are written to unencrypted storage, so if somebody took out the hard drive, they could access sensitive data. If you are sending a crash dump through the network, it may be captured by third parties. Locally the data are written directly to a dump device, skipping the GEOM subsystem. The purpose of that is to allow a kernel to write a crash dump even in case a panic occurs in the GEOM subsystem. It means that a crash dump cannot be automatically encrypted with GELI.
\n
\n\n\nTime, a word that is entangled in everything in our lives, something we’re intimately familiar with. Keeping track of it is important for many activities we do.
\n\nOver millennia we’ve developed different ways to calculate it. Most prominently, we’ve relied on the position the sun appears to be at in the sky, what is called apparent solar time.
\n\nWe’ve decided to split it as seasons pass, counting one full cycle of the 4 seasons as a year, a full rotation around the sun. We’ve also divided the passing of light to the lack thereof as days, a rotation of the earth on itself. Moving on to more precise clock divisions such as seconds, minutes, and hours, units that meant different things at different points in history. Ultimately, as travel got faster, the different ways of counting time that evolved in multiple places had to converge. People had to agree on what it all meant.
\n
See the article for more
\n\n\n\n\nsyslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.
\n
\n\n\nThis week I committed an overhaul of the kernel entropy system. Please let me know if you observe any snags! For the technical background, see the thread on tech-kern a few months ago: https://mail-index.NetBSD.org/tech-kern/2019/12/21/msg025876.html.
\n
\n\n\nI used T_PAGEFLT’s blog post as a reference for setting my NetBSD kernel development environment since his website is down I’m putting down the steps here so it would be helpful for starters.
\n
FuryBSD 2020Q2 Images Available, Technical reasons to choose FreeBSD over GNU/Linux, Ars technica reviews GhostBSD, “TLS Mastery” sponsorships open, BSD community show their various collections, a tale of OpenBSD secure memory allocator internals, learn to stop worrying and love SSDs, and more.
\n\n\n\n\nThe Q2 2020 images are not a visible leap forward but a functional leap forward. Most effort was spent creating a better out of box experience for automatic Ethernet configuration, working WiFi, webcam, and improved hypervisor support.
\n
\n\n\nSince I wrote my article "Why you should migrate everything from Linux to BSD" I have been wanting to write something about the technical reasons to choose FreeBSD over GNU/Linux and while I cannot possibly cover every single reason, I can write about some of the things that I consider worth noting.
\n
\n\n\nWhen I began work on the FreeBSD 12.1-RELEASE review last week, it didn't take long to figure out that the desktop portion wasn't going very smoothly.
\n\nI think it's important for BSD-curious users to know of easier, gentler alternatives, so I did a little looking around and settled on GhostBSD for a follow-up review.
\n\nGhostBSD is based on TrueOS, which itself derives from FreeBSD Stable. It was originally a Canadian distro, but—like most successful distributions—it has transcended its country of origin and can now be considered worldwide. Significant GhostBSD development takes place now in Canada, Italy, Germany, and the United States.
\n
\n\n\nMy next book will be TLS Mastery, all about Transport Layer Encryption, Let’s Encrypt, OCSP, and so on.
\n\nThis should be a shorter book, more like my DNSSEC or Tarsnap titles, or the first edition of Sudo Mastery. I would like a break from writing doorstops like the SNMP and jails books.
\n
JT's post: https://twitter.com/q5sys/status/1251194823589138432
\n\nOthers jumped in with their collections:
\n\nDo you have a nice collection, take a picture and send it in!
\n\n\n\n\nHi there,
\n\nIt's been a very long time I haven't written anything after my last OpenBSD blogs, that is,
\n\nOpenBSD Kernel Internals — Creation of process from user-space to kernel space.
\n\nOpenBSD: Introduction to
\n\nexecpromises
in the pledge(2)pledge(2): OpenBSD's defensive approach to OS Security
\n\nSo, again I started reading OpenBSD source codes with debugger after reducing my sleep timings and managing to get some time after professional life. This time I have picked one of my favourite item from my wishlist to learn and share, that is, OpenBSD malloc(3), secure allocator
\n
\n\n\nmy home FreeNAS runs two pools for data. One RAIDZ2 with four spinning disk drives and one mirror with two SSDs. Toying with InfluxDB and Grafana in the last couple of days I found that I seem to have a constant write load of 1 Megabyte (!) per second on the SSDs. What the ...?
\n\nSo I run three VMs on the SSDs in total. One with Windows 10, two with Ubuntu running Confluence, A wiki essentially, with files for attachments and MySQL as the backend database. Clearly the writes had to stop when the wikis were not used at all, just sitting idle, right?
\n\nWell even with a full query log and quite some experience in the operation of web applications I could not figure out what Confluence is doing (productively, no doubt) but trust me, it writes a couple of hundred kbytes to the database each second just sitting idle.
\n
\n\n\nI've wanted to write about my infrastructure for a while, but I kept thinking, "I'll wait until after I've done $next_thing_on_my_todo." Of course this cycle never ends, so I decided to write about its state at the end of 2019. Maybe I'll write an update on it in a couple of moons; who knows?
\n
Rethinking OpenBSD security, FreeBSD 2020 Q1 status report, the notion of progress and user interfaces, Comments about Thomas E. Dickey on NetBSD curses, making Unix a little more Plan9-like, Not-actually Linux distro review: FreeBSD, and more.
\n\n\n\n\nOpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.
\n
\nI picked a few errata, not all of them, that were interesting and happened to suit my narrative.
\n\n\nWelcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020.
\n
\n\n\nOne trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better.
\n\nHow should we think about progress? Both in general and regarding technology?
\n
\n\n\nI was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.
\n
\n\n\nI’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up.
\n\nA Warning
\n\nThe suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone.
\n
\n\n\nThis month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems.
\n\nThe first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then.
\n\nBefore we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it.
\n\nFreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution.
\n
Jordyn - ZFS Pool Problem
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Tales from a core file, Lenovo X260 BIOS Update with OpenBSD, the problem of Unix iowait and multi-CPU machines, Hugo workflow using FreeBSD Jails, Caddy, Restic; extending NetBSD-7 branch support, a tale of two hypervisor bugs, and more.
\n\n\n\n\nOn the side, I’ve been wrapping up some improvements to the classic Unix stdio libraries in illumos. stdio contains the classic functions like fopen(), printf(), and the security nightmare gets(). While working on support for fmemopen() and friends I got to reacquaint myself with some of the joys of the stdio ABI and its history from 7th Edition Unix. With that in mind, let’s dive into this, history, and some mistakes not to repeat. While this is written from the perspective of the C programming language, aspects of it apply to many other languages.
\n
\n\n\nMy X260 only runs OpenBSD and has no CD driver. But I still need to upgrade its BIOS from time to time. And this is possible using the ISO BIOS image.
\n\nFirst off all, you need to download the “BIOS Update (Bootable CD)” from the Lenovo Support Website.
\n
\n\n\nVarious Unixes have had a 'iowait' statistic for a long time now (although I can't find a source for where it originated; it's not in 4.x BSD, so it may have come through System V and sar). The traditional and standard definition of iowait is that it's the amount of time the system was idle but had at least one process waiting on disk IO. Rather than count this time as 'idle' (as you would if you had a three-way division of CPU time between user, system, and idle), some Unixes evolved to count this as a new category, 'iowait'.
\n
\n\n\nAfter hosting with Netlify for a few years, I decided to head back to self hosting. Theres a few reasons for that but the main reasoning was that I had more control over how things worked.
\n\nIn this post, i’ll show you my workflow for deploying my Hugo generated site (www.jaredwolff.com). Instead of using what most people would go for, i’ll be doing all of this using a FreeBSD Jails based server. Plus i’ll show you some tricks i’ve learned over the years on bulk image resizing and more.
\n\nLet’s get to it.
\n
\n\n\nTypically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.
\n\nWe've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.
\n\nSecurity fixes will still be made to the NetBSD-7 branch.
\n\nWe hope you're all safe. Stay home.
\n
\n\n\nVM escape has become a popular topic of discussion over the last few years. A good amount of research on this topic has been published for various hypervisors like VMware, QEMU, VirtualBox, Xen and Hyper-V. Bhyve is a hypervisor for FreeBSD supporting hardware-assisted virtualization. This paper details the exploitation of two bugs in bhyve - FreeBSD-SA-16:32.bhyve (VGA emulation heap overflow) and CVE-2018-17160 (Firmware Configuration device bss buffer overflow) and some generic techniques which could be used for exploiting other bhyve bugs. Further, the paper also discusses sandbox escapes using PCI device passthrough, and Control-Flow Integrity bypasses in HardenedBSD 12-CURRENT
\n
NetBSD 8.2 is available, NextCloud on OpenBSD, X11 screen locking, NetBSD and RISC OS running parallel, community feedback about switching to BSD, and more.
\n\n\n\n\nThe third release in the NetBSD-8 is now available.
\n\nThis release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.
\n
\n\n\nNextCloud and OpenBSD are complementary to one another. NextCloud is an awesome, secure and private alternative for proprietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial.
\n
\n\n\nBack when this tutorial was initially written, things were different. The OpenBSD port relied on PHP 5.6 and there were no package updates. But the port improved (hats off, Gonzalo!) and package updates were introduced to the -stable branch (hats off, Solene!).
\n\nA rewrite of this tutorial was long overdue. Right now, it is written for 6.6 -stable and will be updated once 6.7 is released. If you have any questions or desire some help, feel free to reach out.
\n
\n\n\nFor years I’ve been using XScreenSaver as a default, but I recently learned about xsecurelock and re-evaluated my screen-saving requirements
\n
\n\n\nI have been experimenting with running two systems at the same time on the RK3399 SoC.
\n\n
\nIt all begun when I figured out how to switch to the A72 cpu for RISC OS. When the switch was done, the A53 cpu just continued to execute code.
\nOK I thought why not give it something to do!
\nMy first step was to run some small programs.
\nIt worked!\n
\n- Thanks to Tom Jones for the pointer to this article
\n
Shell text processing, data rebalancing on ZFS mirrors, Add Security Headers with OpenBSD relayd, ZFS filesystem hierarchy in ZFS pools, speeding up ZSH, How Unix pipes work, grow ZFS pools over time, the real reason ifconfig on Linux is deprecated, clear your terminal in style, and more.
\n\n\n\n\nThis article is part of a self-published book project by Balthazar Rouberol and Etienne Brodu, ex-roommates, friends and colleagues, aiming at empowering the up and coming generation of developers. We currently are hard at work on it!
\n\nOne of the things that makes the shell an invaluable tool is the amount of available text processing commands, and the ability to easily pipe them into each other to build complex text processing workflows. These commands can make it trivial to perform text and data analysis, convert data between different formats, filter lines, etc.
\n\nWhen working with text data, the philosophy is to break any complex problem you have into a set of smaller ones, and to solve each of them with a specialized tool.
\n
\n\n\nOne of the questions that comes up time and time again about ZFS is “how can I migrate my data to a pool on a few of my disks, then add the rest of the disks afterward?”
\n\nIf you just want to get the data moved and don’t care about balance, you can just copy the data over, then add the new disks and be done with it. But, it won’t be distributed evenly over the vdevs in your pool.
\n\nDon’t fret, though, it’s actually pretty easy to rebalance mirrors. In the following example, we’ll assume you’ve got four disks in a RAID array on an old machine, and two disks available to copy the data to in the short term.
\n
\n\n\nI am a huge fan of OpenBSD’s built-in httpd server as it is simple, secure, and quite performant. With the modern push of the large search providers pushing secure websites, it is now important to add security headers to your website or risk having the search results for your website downgraded. Fortunately, it is very easy to do this when you combine httpd with relayd. While relayd is principally designed for layer 3 redirections and layer 7 relays, it just so happens that it makes a handy tool for adding the recommended security headers. My website automatically redirects users from http to https and this gets achieved using a simple redirection in /etc/httpd.conf So if you have a configuration similar to mine, then you will still want to have httpd listen on the egress interface on port 80. The key thing to change here is to have httpd listen on 127.0.0.1 on port 443.
\n
\n\n\nOur long standing practice here, predating even the first generation of our ZFS fileservers, is that we have two main sorts of filesystems, home directories (homedir filesystems) and what we call 'work directory' (workdir) filesystems. Homedir filesystems are called /h/NNN (for some NNN) and workdir filesystems are called /w/NNN; the NNN is unique across all of the different sorts of filesystems. Users are encouraged to put as much stuff as possible in workdirs and can have as many of them as they want, which mattered a lot more in the days when we used Solaris DiskSuite and had fixed-sized filesystems.
\n
https://web.archive.org/web/20200315184849/https://blog.jonlu.ca/posts/speeding-up-zsh
\n\n\n\n\nI was opening multiple shells for an unrelated project today and noticed how abysmal my shell load speed was. After the initial load it was relatively fast, but the actual shell start up was noticeably slow. I timed it with time and these were the results.
\n\nIn the future I hope to actually recompile zsh with additional profiling techniques and debug information - keeping an internal timer and having a flag output current time for each command in a tree fashion would make building heat maps really easy.
\n
\n\n\nPipes are cool! We saw how handy they are in a previous blog post. Let’s look at a typical way to use the pipe operator. We have some output, and we want to look at the first lines of the output. Let’s download The Brothers Karamazov by Fyodor Dostoevsky, a fairly long novel.
\n
\n\n\nIn my entry on why ZFS isn't good at growing and reshaping pools, I mentioned that we go to quite some lengths in our ZFS environment to be able to incrementally expand our pools. Today I want to put together all of the pieces of that in one place to discuss what those lengths are.
\n
\nOur big constraint is that not only do we need to add space to pools over time, but we have a fairly large number of pools and which pools will have space added to them is unpredictable. We need a solution to pool expansion that leaves us with as much flexibility as possible for as long as possible. This pretty much requires being able to expand pools in relatively small increments of space.
\n\n\nIn my third installment of FreeBSD vs Linux, I will discuss underlying reasons for why Linux moved away from ifconfig(8) to ip(8).
\n
In the past, when people said, “Linux is a kernel, not an operating system”, I knew that was true but I always thought it was a rather pedantic criticism. Of course no one runs just the Linux kernel, you run a distribution of Linux. But after reviewing userland code, I understand the significant drawbacks to developing “just a kernel” in isolation from the rest of the system.
\n\n\n\n\nif you’re someone like me who habitually clears their terminal, sometimes you want a little excitement in your life. Here is a way to do just that.
\n\nThis post revolves around the idea of giving a command a percent chance of running. While the topic at hand is not serious, this simple technique has potential in your scripts.
\n
Fighting the Coronavirus with FreeBSD, Wireguard VPN Howto in OPNsense, NomadBSD 1.3.1 available, fresh GhostBSD 20.02, New FuryBSD XFCE and KDE images, pf-badhost 0.3 released, and more.
\n\n\n\n\nHere is a quick HOWTO for those who want to provide some FreeBSD based compute resources to help finding vaccines.
\n\nUPDATE 2020-03-22: 0mp@ made a port out of this, it is in “biology/linux-foldingathome”.
\n\nPer default it will now pick up some SARS-CoV‑2 (COVID-19) related folding tasks. There are some more config options (e.g. how much of the system resources are used). Please refer to the official Folding@Home site for more information about that. Be also aware that there is a big rise in compute resources donated to Folding@Home, so the pool of available work units may be empty from time to time, but they are working on adding more work units. Be patient.
\n
\n\n\nWireGuard is a modern designed VPN that uses the latest cryptography for stronger security, is very lightweight, and is relatively easy to set up (mostly). I say ‘mostly’ because I found setting up WireGuard in OPNsense to be more difficult than I anticipated. The basic setup of the WireGuard VPN itself was as easy as the authors claim on their website, but I came across a few gotcha's. The gotcha's occur with functionality that is beyond the scope of the WireGuard protocol so I cannot fault them for that. My greatest struggle was configuring WireGuard to function similarly to my OpenVPN server. I want the ability to connect remotely to my home network from my iPhone or iPad, tunnel all traffic through the VPN, have access to certain devices and services on my network, and have the VPN devices use my home's Internet connection.
\n\nWireGuard behaves more like a SSH server than a typical VPN server. With WireGuard, devices which have shared their cryptographic keys with each other are able to connect via an encrypted tunnel (like a SSH server configured to use keys instead of passwords). The devices that are connecting to one another are referred to as “peer” devices. When the peer device is an OPNsense router with WireGuard installed, for instance, it can be configured to allow access to various resources on your network. It becomes a tunnel into your network similar to OpenVPN (with the appropriate firewall rules enabled). I will refer to the WireGuard installation on OPNsense as the server rather than a “peer” to make it more clear which device I am configuring unless I am describing the user interface because that is the terminology used interchangeably by WireGuard.
\n\nThe documentation I found on WireGuard in OPNsense is straightforward and relatively easy to understand, but I had to wrestle with it for a little while to gain a better understanding on how it should be configured. I believe it was partially due to differing end goals – I was trying to achieve something a little different than the authors of other wiki/blog/forum posts. Piecing together various sources of information, I finally ended up with a configuration that met the goals stated above.
\n
\n\n\nNomadBSD 1.3.1 has recently been made available. NomadBSD is a lightweight and portable FreeBSD distribution, designed to run on live on a USB flash drive, allowing you to plug, test, and play on different hardware. They have also started a forum as of yesterday, where you can ask questions and mingle with the NomadBSD community. Notable changes in 1.3.1 are base system upgraded to FreeBSD 12.1-p2. automatic network interface setup improved, image size increased to over 4GB, Thunderbird, Zeroconf, and some more listed below.
\n
\n\n\nEric Turgeon, main developer of GhostBSD, has announced version 20.02 of the FreeBSD based operating system. Notable changes are ZFS partition into the custom partition editor installer, allowing you to install alongside with Windows, Linux, or macOS. Other changes are force upgrade all packages on system upgrade, improved update station, and powerd by default for laptop battery performance.
\n
\n\n\nThis new release is now based on FreeBSD 12.1 with the latest FreeBSD quarterly packages. This brings XFCE up to 4.14, and KDE up to 5.17. In addition to updates this new ISO mostly addresses community bugs, community enhancement requests, and community pull requests. Due to the overwhelming amount of reports with GitHub hosting all new releases are now being pushed to SourceForge only for the time being. Previous releases will still be kept for archive purposes.
\n
\n\n\npf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet's biggest irritants. Annoyances such as SSH and SMTP bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.
\n
OpenBSD Full disk encryption with coreboot and tianocore, FreeBSD 12.0 EOL, ZFS DVA layout, OpenBSD’s Go situation, AD updates requires changes in TrueNAS and FreeNAS, full name of FreeBSD’s root account, and more.
\n\n\n\n\nIt has been a while since I have posted here so I wanted to share something that was surprisingly difficult for me to figure out. I have a Thinkpad T440p that I have flashed with Coreboot 4.11 with some special patches that allow the newer machine to work. When I got the laptop, the default BIOS was UEFI and I installed two operating systems.
\n\nWindows 10 with bitlocker full disk encryption on the “normal” drive (I replaced the spinning 2.5″ disk with an SSD)
\n\nUbuntu 19.10 on the m.2 SATA drive that I installed using LUKS full disk encryption
\n\nI purchased one of those carriers for the optical bay that allows you to install a third SSD and so I did that with the intent of putting OpenBSD on it. Since my other two operating systems were running full disk encryption, I wanted to do the same on OpenBSD.
\n
\n\n\n\n\nDear FreeBSD community,
\n\nAs of February 29, 2020, FreeBSD 12.0 will reach end-of-life and will no longer be supported by the FreeBSD Security Team. Users of FreeBSD 12.0 are strongly encouraged to upgrade to a newer release as soon as possible.
\n
\n\n\nOne piece of ZFS terminology is DVA and DVAs, which is short for Data Virtual Address. For ZFS, a DVA is the equivalent of a block number in other filesystems; it tells ZFS where to find whatever data we're talking about. The short summary of what fields DVAs have and what they mean is that DVAs tell us how to find blocks by giving us their vdev (by number) and their byte offset into that particular vdev (and then their size). A typical DVA might say that you find what it's talking about on vdev 0 at byte offset 0x53a40ed000. There are some consequences of this that I hadn't really thought about until the other day.
\n\nRight away we can see why ZFS has a problem removing a vdev; the vdev's number is burned into every DVA that refers to data on it. If there's no vdev 0 in the pool, ZFS has no idea where to even start looking for data because all addressing is relative to the vdev. ZFS pool shrinking gets around this by adding a translation layer that says where to find the portions of vdev 0 that you care about after it's been removed.
\n
\n\n\nMicrosoft is changing the security defaults for Active Directory to eliminate some security vulnerabilities in its protocols. Unfortunately, these new security defaults may disrupt existing FreeNAS/TrueNAS deployments once Windows systems are updated. The Windows updates may appear sometime in March 2020; no official date has been announced as of yet.
\n\nFreeNAS and TrueNAS users that utilize Active Directory should update to version 11.3 (or 11.2-U8) to avoid potential disruption of their networks when updating to the latest versions of Windows software after March 1, 2020. Version 11.3 has been released and version 11.2-U8 will be available in early March.
\n
\n\n\nNetBSD now has a users(7) and groups(7) manual. Looking into what entries existed in the passwd and group files I wondered about root’s full name who we now know as Charlie Root in the BSDs....
\n
\n\n\nOver in the fediverse, Pete Zaitcev had a reaction to my entry on OpenBSD versus Prometheus for us:
\n\nI don't think the situation is usually that bad. Our situation with Prometheus is basically a worst case scenario for Go on OpenBSD, and most people will have much better results, especially if you stick to supported OpenBSD versions.
\n\nIf you stick to supported OpenBSD versions, upgrading your machines as older OpenBSD releases fall out of support (as the OpenBSD people want you to do), you should not have any problems with your own Go programs. The latest Go release will support the currently supported OpenBSD versions (as long as OpenBSD remains a supported platform for Go), and the Go 1.0 compatibility guarantee means that you can always rebuild your current Go programs with newer versions of Go. You might have problems with compiled binaries that you don't want to rebuild, but my understanding is that this is the case for OpenBSD in general; it doesn't guarantee a stable ABI even for C programs (cf). If you use OpenBSD, you have to be prepared to rebuild your code after OpenBSD upgrades regardless of what language it's written in.
\n
FreeBSD on Power, DragonflyBSD 5.8 is here, Unifying FreeNAS/TrueNAS, OpenBSD vs. Prometheus and Go, gcc 4.2.1 removed from FreeBSD base, and more.
\n\n\n\n\nThe power and promise of all open source software is freedom. Another way to express freedom is choice — choice of platforms, deployment models, stacks, configurations, etc.
\n\nThe FreeBSD Foundation is dedicated to supporting and promoting the FreeBSD Project and community worldwide. But, what does this mean, exactly, you may wonder. The truth is it means many different things, but in all cases the Foundation acts to expand freedom and choice so that FreeBSD users have the power to serve their varied compute needs.
\n\nThis blog tells the story of one specific way the Foundation helps a member of the community provide greater hardware choice for all FreeBSD users.
\n
\n\n\nDragonFly version 5.8 brings a new dsynth utility for building your own binary dports packages, plus significant support work to speed up that build - up to and including the entire collection. Additional progress has been made on GPU and signal support.
\n\nThe details of all commits between the 5.6 and 5.8 branches are available in the associated commit messages for 5.8.0rc1 and 5.8.0. Also see /usr/src/UPDATING for specific file changes in PAM.
\n
\n\n\nFreeNAS and TrueNAS have been separate-but-related members of the #1 Open Source storage software family since 2012. FreeNAS is the free Open Source version with an expert community and has led the pursuit of innovations like Plugins and VMs. TrueNAS is the enterprise version for organizations of all sizes that need additional uptime and performance, as well as the enterprise-grade support necessary for critical data and applications.
\n\nFrom the beginning at iXsystems, we’ve developed, tested, documented, and released both as separate products, even though the vast majority of code is shared. This was a deliberate technical decision in the beginning but over time became less of a necessity and more of “just how we’ve always done it”. Furthermore, to change it was going to require a serious overhaul to how we build and package both products, among other things, so we continued to kick the can down the road. As we made systematic improvements to development and QA efficiency over the past few years, the redundant release process became almost impossible to ignore as our next major efficiency roadblock to overcome. So, we’ve finally rolled up our sleeves.
\n\nWith the recent 11.3 release, TrueNAS gained parity with FreeNAS on features like VMs and Plugins, further homogenizing the code. Today, we announce the next phase of evolution for FreeNAS and TrueNAS.
\n
\n\n\nWe have a decent number of OpenBSD machines that do important things (and that have sometimes experienced problems like running out of disk space), and we have a Prometheus based metrics and monitoring system. The Prometheus host agent has enough support for OpenBSD to be able to report on critical metrics, including things like local disk space. Despite all of this, after some investigation I've determined that it's not really sensible to even try to deploy the host agent on our OpenBSD machines. This is due to a combination of factors that have at their root OpenBSD's lack of ABI stability
\n
\n\n\nAs described in Warner's email message[1] to the FreeBSD-arch mailing list we have reached GCC 4.2.1's retirement date. At this time all supported architectures either use in-tree Clang, or rely on external toolchain (i.e., a contemporary GCC version from ports).
\n\nGCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later that year, in r171825. GCC has served us well, but version 4.2.1 is obsolete and not used by default on any architecture in FreeBSD. It does not support modern C and does not support arm64 or RISC-V.
\n
Why ZFS is doing filesystem checksumming right, better TMPFS throughput performance on DragonFlyBSD, reshaping pools with ZFS, PKGSRC on Manjaro aarch64 Pinebook-pro, central log host with syslog-ng on FreeBSD, and more.
\n\n\n\n\nOne of the best aspects of ZFS is its reliability. This can be accomplished using a few features like copy-on-write approach and checksumming. Today we will look at how ZFS does checksumming and why it does it the proper way. Most of the file systems don’t provide any integrity checking and fail in several scenarios:
\n
\n\n\nChecksumming may help us detect errors in a few of those situations.
\n
\n\n\nIt's been a while since last having any new magical optimizations to talk about by DragonFlyBSD lead developer Matthew Dillon, but on Wednesday he landed some significant temporary file-system "TMPFS" optimizations for better throughput including with swap.
\n\nOf several interesting commits merged tonight, the improved write clustering is a big one. In particular, "Reduces low-memory tmpfs paging I/O overheads by 4x and generally increases paging throughput to SSD-based swap by 2x-4x. Tmpfs is now able to issue a lot more 64KB I/Os when under memory pressure."
\n
\n\n\n\n\nThere's also a new tunable in the VM space as well as part of his commits on Wednesday night. This follows a lot of recent work on dsynth, improved page-out daemon pipelining, and other routine work.
\n
\n\n\nThis work is building up towards the eventual DragonFlyBSD 5.8 while those wanting to try the latest improvements right away can find their daily snapshots.
\n
\n\n\nrecently read Mark McBride's Five Years of Btrfs (via), which has a significant discussion of why McBride chose Btrfs over ZFS that boils down to ZFS not being very good at evolving your pool structure. You might doubt this judgment from a Btrfs user, so let me say as both a fan of ZFS and a long term user of it that this is unfortunately quite true; ZFS is not a good choice if you want to modify your pool disk layout significantly over time. ZFS works best if the only change in your pools that you do is replacing drives with bigger drives. In our ZFS environment we go to quite some lengths to be able to expand pools incrementally over time, and while this works it both leaves us with unbalanced pools and means that we're basically forced to use mirroring instead of RAIDZ.
\n\n(An unbalanced pool is one where some vdevs and disks have much more data than others. This is less of an issue for us now that we're using SSDs instead of HDs.)
\n
\n\n\nI wanted to see how pkgsrc works on aarch64 Linux Manjaro since it is a very mature framework that is very portable and supported by many architectures – pkgsrc (package source) is a package management system for Unix-like operating systems. It was forked from the FreeBSD ports collection in 1997 as the primary package management system for NetBSD.
\n\nOne might question why use pkgsrc on Arch based Manjaro, since the pacman package repository is very good on its own. I see alternative pkgsrc as a good automated build framework that offers a way to produce independent build environment /usr/pkg that does not interfere with the current Linux distribution in any way (all libraries are statically built)
\n\nI have used the latest Manjaro for Pinebookpro and standard recommended tools as mentioned here https://wiki.netbsd.org/pkgsrc/how_to_use_pkgsrc_on_linux/
\n
\n\n\nsyslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.
\n
\n\n\nThis blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not too difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host.
\n
Meet FuryBSD, NetBSD 9.0 has been released, OpenBSD Foundation 2019 campaign wrapup, a retrospective on OmniOS ZFS-based NFS fileservers, NetBSD Fundraising 2020 goal, OpenSSH 8.2 released, and more.## Headlines
\n\n\n\n\nAt its heart, FuryBSD is a very simple beast. According to the site, “FuryBSD is a back to basics lightweight desktop distribution based on stock FreeBSD.” It is basically FreeBSD with a desktop environment pre-configured and several apps preinstalled. The goal is to quickly get a FreeBSD-based system running on your computer.
\n\nYou might be thinking that this sounds a lot like a couple of other BSDs that are available, such as NomadBSD and GhostBSD. The major difference between those BSDs and FuryBSD is that FuryBSD is much closer to stock FreeBSD. For example, FuryBSD uses the FreeBSD installer, while others have created their own installers and utilities.
\n\nAs it states on the site, “Although FuryBSD may resemble past graphical BSD projects like PC-BSD and TrueOS, FuryBSD is created by a different team and takes a different approach focusing on tight integration with FreeBSD. This keeps overhead low and maintains compatibility with upstream.” The lead dev also told me that “One key focus for FuryBSD is for it to be a small live media with a few assistive tools to test drivers for hardware.”
\n\nCurrently, you can go to the FuryBSD homepage and download either an XFCE or KDE LiveCD. A GNOME version is in the works.
\n
\n\n\nThe NetBSD Project is pleased to announce NetBSD 9.0, the seventeenth major release of the NetBSD operating system.
\n\nThis release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes. Here are some highlights of this new release.
\n
\n\n\n\n\nOur target for 2019 was CDN$300K. Our community's continued generosity combined with our corporate donors exceeded that nicely. In addition we received the largest single donation in our history, CDN$380K from Smartisan. The return of Google was another welcome event. Altogether 2019 was our most successful campaign to date, yielding CDN$692K in total.
\n\nWe thank all our donors, Iridium (Smartisan), Platinum (Yandex, Google), Gold (Microsoft, Facebook) Silver (2Keys) and Bronze (genua, Thinkst Canary). But especially our community of smaller donors whose contributions are the bedrock of our support. Thank you all!
\n
\n\n\nOur OmniOS fileservers have now been out of service for about six months, which makes it somewhat past time for a retrospective on them. Our OmniOS fileservers followed on our Solaris fileservers, which I wrote a two part retrospective on (part 1, part 2), and have now been replaced by our Linux fileservers. To be honest, I have been sitting on my hands about writing this retrospective because we have mixed feelings about our OmniOS fileservers.
\n\nI will put the summary up front. OmniOS worked reasonably well for us over its lifespan here and looking back I think it was almost certainly the right choice for us at the time we made that choice (which was 2013 and 2014). However it was not without issues that marred our experience with it in practice, although not enough to make me regret that we ran it (and ran it for as long as we did). Part of our issues are likely due to a design mistake in making our fileservers too big, although this design mistake was probably magnified when we were unable to use Intel 10G-T networking in OmniOS.
\n\nOn the one hand, our OmniOS fileservers worked, almost always reliably. Like our Solaris fileservers before them, they ran quietly for years without needing much attention, delivering NFS fileservice to our Ubuntu servers; specifically, we ran them for about five years (2014 through 2019, although we started migrating away at the end of 2018). Over this time we had only minor hardware issues and not all that many disk failures, and we suffered no data loss (with ZFS checksums likely saving us several times, and certainly providing good reassurances). Our overall environment was easy to manage and was pretty much problem free in the face of things like failed disks. I'm pretty sure that our users saw a NFS environment that was solid, reliable, and performed well pretty much all of the time, which is the important thing. So OmniOS basically delivered the fileserver environment we wanted.
\n
\n\n\nIs it really more than 10 years since we last had an official fundraising drive?
\n\nLooking at old TNF financial reports I noticed that we have been doing quite well financially over the last years, with a steady stream of small and medium donations, and most of the time only moderate expenditures. The last fundraising drive back in 2009 was a giant success, and we have lived off it until now.
\n
\n\n\n\n\nOpenSSH 8.2 was released on 2020-02-14. It is available from the mirrors listed at https://www.openssh.com/.
\n\nOpenSSH is a 100% complete SSH protocol 2.0 implementation and includes sftp client and server support.
\n\nOnce again, we would like to thank the OpenSSH community for their continued support of the project, especially those who contributed code or patches, reported bugs, tested snapshots or donated to the project. More information on donations may be found at:
\n
Distrowatch reviews FuryBSD, LLDB on i386 for NetBSD, wpa_supplicant as lower-class citizen, KDE on FreeBSD updates, Travel Grant for BSDCan open, ZFS dataset for testing iocage within a jail, and more.
\n\n\n\n\nFuryBSD is the most recent addition to the DistroWatch database and provides a live desktop operating system based on FreeBSD. FuryBSD is not entirely different in its goals from NomadBSD, which we discussed recently. I wanted to take this FreeBSD-based project for a test drive and see how it compares to NomadBSD and other desktop-oriented projects in the FreeBSD family.
\n\nFuryBSD supplies hybrid ISO/USB images which can be used to run a live desktop. There are two desktop editions currently, both for 64-bit (x86_64) machines: Xfce and KDE Plasma. The Xfce edition is 1.4GB in size and is the flavour I downloaded. The KDE Plasma edition is about 3.0GB in size.
\n\nMy fresh install of FuryBSD booted to a graphical login screen. From there I could sign into my account, which brings up the Xfce desktop. The installed version of Xfce is the same as the live version, with a few minor changes. Most of the desktop icons have been removed with just the file manager launchers remaining. The Getting Started and System Information icons have been removed. Otherwise the experience is virtually identical to the live media.
\n\nFuryBSD uses a theme that is mostly grey and white with creamy yellow folder icons. The application menu launchers tend to have neutral icons, neither particularly bright and detailed or minimal.
\n
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n\nIn February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support.
\n\nThe original NetBSD port of LLDB was focused on amd64 only. In January, I have extended it to support i386 executables. This includes both 32-bit builds of LLDB (running natively on i386 kernel or via compat32) and debugging 32-bit programs from 64-bit LLDB.
\n
\n\n\nwpa_supplicant is definitely a lower-class citizen, sorry.
\n\nI increasingly wonder why this stuff matters; transit costs are so much lower than the period when eduroam was setup, and their reliance on 802.11x is super weird in a world where, for the most part
\n\n
\n + entire cities have open wifi in their downtown core
\n + edu vs edu+transit split horizon problems have to be solved anyways
\n + many universities have parallel open wifi
\n + rate limiting / fare-share approaches for the open-net, on unmetered
\n + flat-rate solves the problem
\n + LTE hotspot off a phone isn't a rip off anymore
\n + other open networks existessentially no one else feels compelled to do use 802.11x for a so called "semi-open access network", so I think they've lost the plot on friction vs benefit.
\n\n(we've held hackathons at EDU campus that are locked down like that, and in every case we've said no way, gotten a wire with open net, and built our own wifi. we will not subject our developers to that extra complexity).
\n
\n\n\nSome bits and bobs from the KDE FreeBSD team in february 2020. We met at the FreeBSD devsummit before FOSDEM, along with other FreeBSD people. Plans were made, schemes were forged, and Groff the Goat was introduced to some new people.
\n
\n\n\nHi everyone,
\n\nThe Travel Grant Application for BSDCan 2020 is now open. The Foundation can help you attend BSDCan through our travel grant program. Travel grants are available to FreeBSD developers and advocates who need assistance with travel expenses for attending conferences related to FreeBSD development. BSDCan 2020 applications are due April 9, 2020. Find out more and apply at: https://www.freebsdfoundation.org/what-we-do/grants/travel-grants/
\n\nDid you know the Foundation also provides grants for technical events not specifically focused on BSD? If you feel that your attendance at one of these events will benefit the FreeBSD Project and Community and you need assistance getting there, please fill out the general travel grant application. Your application must be received 7 weeks prior to the event. The general application can be found here: https://goo.gl/forms/QzsOMR8Jra0vqFYH2
\n
\n\n\nI’m going to do jails within a jail. I already do that with poudriere in a jail but here I want to test an older version of iocage before upgrading my current jail hosts to a newer version.
\n
\n\n\nThis post includes my errors and mistakes. Perhaps you should proceed carefully and read it all first.
\n
Happinesses and stresses of full-time FOSS work, building a FreeBSD fileserver, Kubernetes on FreeBSD bhyve, NetBSD 9 RC1 available, OPNSense 20.1 is here, HardenedBSD’s idealistic future, and more.
\n\n\n\n\nIn the past few days, several free software maintainers have come out to discuss the stresses of their work. Though the timing was suggestive, my article last week on the philosophy of project governance was, at best, only tangentially related to this topic - I had been working on that article for a while. I do have some thoughts that I’d like to share about what kind of stresses I’ve dealt with as a FOSS maintainer, and how I’ve managed (or often mismanaged) it.
\n\nFebruary will mark one year that I’ve been working on self-directed free software projects full-time. I was planning on writing an optimistic retrospective article around this time, but given the current mood of the ecosystem I think it would be better to be realistic. In this stage of my career, I now feel at once happier, busier, more fulfilled, more engaged, more stressed, and more depressed than I have at any other point in my life.
\n\nThe good parts are numerous. I’m able to work on my life’s passions, and my projects are in the best shape they’ve ever been thanks to the attention I’m able to pour into them. I’ve also been able to do more thoughtful, careful work; with the extra time I’ve been able to make my software more robust and reliable than it’s ever been. The variety of projects I can invest my time into has also increased substantially, with what was once relegated to minor curiosities now receiving a similar amount of attention as my larger projects were receiving in my spare time before. I can work from anywhere in the world, at any time, not worrying about when to take time off and when to put my head down and crank out a lot of code.
\n\nThe frustrations are numerous, as well. I often feel like I’ve bit off more than I can chew. This has been the default state of affairs for me for a long time; I’m often neglecting half of my projects in order to obtain progress by leaps and bounds in just a few. Working on FOSS full-time has cast this model’s disadvantages into greater relief, as I focus on a greater breadth of projects and spend more time on them.
\n
\n\n\nRecently at my job, I was faced with a task to develop a file server explicitly suited for the requirements of the company. Needless to say, any configuration of a kind depends on what the infrastructure needs. So, drawing from my personal experience and numerous materials on the web, I came up with the combination FreeBSD+SAMBA+AD as the most appropriate. It appears to be a perfect choice for this environment, and harmonic addition to the existing network configuration since FreeBSD + SAMBA + AD enables admins with the broad range of possibilities for access control. However, as nothing is perfect, this configuration isn’t the best choice if your priority is data protection because it won’t be able to reach the necessary levels of reliability and fault tolerance without outside improvements.
\n\nNow, since we’ve established that, let’s move on to the next point. This article’s describing the process of building a test environment while concentrating primarily on the details of the configuration. As the author, though, I must say I’m in no way suggesting that this is the only way! The following configuration will be presented in its initial stage, with the minimum requirements necessary to get the job done, and its purpose in one specific situation only. Here, look at this as a useful strategy to solve similar tasks. Well, let’s get started!
\n
\n\n\nFebruary 11th was the first meeting of this new user group, founded by John Young and myself
\n\n11 people attended, and a lot of good discussions were had
\n\nOne of the attendees already owns a domain that fits well for the group, so we will be getting that setup over the next few weeks, as well as the twitter account, and other organization stuff.
\n\nSpecial thanks to the illumos users who drove in from Buffalo to attend, although they may have actually had a shorter drive than a few of the other attendees.
\n\nThe next meeting is scheduled again for the 2nd Tuesday of the month, March 10th.
\n\nWe are still discussing if we should meet at a restaurant again, or try to get a space at the local college or innovation hub where we can have a projector etc.
\n
\n\n\nThere are quite a few solutions for container orchestration, but the most popular (or the most famous and highly advertised, is probably, a Kubernetes) Since I plan to conduct many experiments with installing and configuring k8s, I need a laboratory in which I can quickly and easily deploy a cluster in any quantities for myself. In my work and everyday life I use two OS very tightly - Linux and FreeBSD OS. Kubernetes and docker are Linux-centric projects, and at first glance, you should not expect any useful participation and help from FreeBSD here. As the saying goes, an elephant can be made out of a fly, but it will no longer fly. However, two tempting things come to mind - this is very good integration and work in the FreeBSD ZFS file system, from which it would be nice to use the snapshot mechanism, COW and reliability. And the second is the bhyve hypervisor, because we still need the docker and k8s loader in the form of the Linux kernel. Thus, we need to connect a certain number of actions in various ways, most of which are related to starting and pre-configuring virtual machines. This is typical of both a Linux-based server and FreeBSD. What exactly will work under the hood to run virtual machines does not play a big role. And if so - let's take a FreeBSD here!
\n
\n\n\nWe hope this will lead to the best NetBSD release ever (only to be topped by NetBSD 10 next year).
\n
Here are a few highlights of the new release:
\n\nYou can download binaries of NetBSD 9.0_RC1 from our Fastly-provided CDN: https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0_RC1/
\n\n\nFor over 5 years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
\n\n20.1, nicknamed "Keen Kingfisher", is a subtle improvement on sustainable firewall experience. This release adds VXLAN and additional loopback device support, IPsec public key authentication and elliptic curve TLS certificate creation amongst others. Third party software has been updated to their latest versions. The logging frontend was rewritten for MVC with seamless API support. On the far side the documentation increased in quality as well as quantity and now presents itself in a familiar menu layout.
\n
\n\n\nOver the past month, we purchased and deployed the new 13-CURRENT/amd64 package building server. We published our first 13-CURRENT/amd64 production package build using that server. We then rebuilt the old package building server to act as the 12-STABLE/amd64 package building server. This post signifies a very important milestone: we have now fully recovered from last year's death of our infrastructure. Our 12-STABLE/amd64 repo, previously out-of-date by many months, is now fully up-to-date!
\n\nHardenedBSD is in a very unique position to provide innovative solutions to at-risk and underprivileged populations. As such, we are making human rights endeavors a defining area of focus. Our infrastructure will integrate various privacy and anonymity enhancing technologies and techniques to protect lives. Our operating system's security posture will increase, especially with our focus on exploit mitigations.
\n\nNavigating the intersection between human rights and information security directly impacts lives. HardenedBSD's 2020 mission and focus is to deliver an entire hardened ecosystem that is unfriendly towards those who would oppress or censor their people. This includes a subtle shift in priorities to match this new mission and focus. While we implement exploit mitigations and further harden the ecosystem, we will seek out opportunities to contribute a tangible and unique impact on human rights issues. Providing Tor Onion Services for our core infrastructure is the first step in likely many to come towards securely helping those in need.
\n
Linux couldn’t duplicate OpenBSD, FreeBSD Q4 status report, OPNsense 19.7.9 released, archives retain and pass on knowledge, HardenedBSD Tor Onion Service v3 Nodes, and more.
\n\n\n\n\nOpenBSD has a well deserved reputation for putting security and a clean system (for code, documentation, and so on) first, and everything else second. OpenBSD is of course based on BSD (it's right there in the name) and descends from FreeBSD NetBSD (you can read the history here). But one of the questions you could ask about it is whether it had to be that way, and in particular if you could build something like OpenBSD on top of Linux. I believe that the answer is no.
\n\nLinux and the *BSDs have a significantly different model of what they are. BSDs have a 'base system' that provides an integrated and fully operational core Unix, covering the kernel, C library and compiler, and the normal Unix user level programs, all maintained and distributed by the particular BSD. Linux is not a single unit this way, and instead all of the component parts are maintained separately and assembled in various ways by various Linux distributions. Both approaches have their advantages, but one big one for the BSD approach is that it enables global changes.
\n\nMaking global changes is an important part of what makes OpenBSD's approach to improving security, code maintenance, and so on work. Because it directly maintains everything as a unit, OpenBSD is in a position to introduce new C library or kernel APIs (or change them) and then immediately update all sorts of things in user level programs to use the new API. This takes a certain amount of work, of course, but it's possible to do it at all. And because OpenBSD can do this sort of ambitious global change, it does.
\n\nThis goes further than just the ability to make global changes, because in theory you can patch in global changes on top of a bunch of separate upstream projects. Because OpenBSD is in control of its entire base system, it's not forced to try to reconcile different development priorities or integrate clashing changes. OpenBSD can decide (and has) that only certain sorts of changes will be accepted into its system at all, no matter what people want. If there are features or entire programs that don't fit into what OpenBSD will accept, they just lose out.
\n
\n\n\nHere is the last quarterly status report for 2019. As you might remember from last report, we changed our timeline: now we collect reports the last month of each quarter and we edit and publish the full document the next month. Thus, we cover here the period October 2019 - December 2019.
\n\nIf you thought that the FreeBSD community was less active in the Christmas' quarter you will be glad to be proven wrong: a quick glance at the summary will be sufficient to see that much work has been done in the last months.
\n\nHave a nice read!
\n
\n\n\nAs 20.1 nears we will be making adjustments to the scope of the release with an announcement following shortly.
\n\nFor now, this update brings you a GeoIP database configuration page for aliases which is now required due to upstream database policy changes and a number of prominent third-party software updates we are happy to see included.
\n
\n\n\nArchives are important. When they are public and available for searching, it retains and passes on knowledge. It saves vast amounts of time.
\n
\n\n\nI've been working today on deploying Tor Onion Service v3 nodes across our build infrastructure. I'm happy to announce that the public portion of this is now completed. Below you will find various onion service hostnames and their match to our infrastructure.
\n
Hyperbola Developer interview, why you should migrate from Linux to BSD, FreeBSD is an amazing OS, improving the ptrace(2) API in LLVM 10, First FreeBSD conference in Australia, and a guide to containers on FreeNAS.
\n\n\n\n\nUpdate 2020-01-21: Since I wrote this article it got posted on Hacker News, Reddit and Lobster, and a few people have emailed me with comments. I have updated the article with comments where I have found it needed. As an important side note I would like to point out that I am not a FreeBSD developer, there may be things going on in the FreeBSD world that I know absolutely nothing about. I am also not glued to the FreeBSD developer mailing lists. I am not a FreeBSD "fanboy". I have been using GNU/Linux a ton more for the past two decades than FreeBSD, mainly due to hardware incompatibility (lacking or buggy drivers), and I love both Debian GNU/Linux and Arch Linux just as much as FreeBSD. However, I am concerned about the development of GNU/Linux as of late. Also this article is not about me trying to make anyone switch from something else to FreeBSD. It's about why I like FreeBSD and that I recommend you try it out if you're into messing with operating systems.
\n\nI think the year was late 1999 or mid 2000 when I one day was browsing computer books at my favorite bookshop and I discovered the book The Complete FreeBSD third edition from 1999 by Greg Lehey. With the book came 4 CD Roms with FreeBSD 3.3.
\n\nI had already familiarized myself with GNU/Linux in 1998, and I was in the process of migrating every server and desktop operating system away from Microsoft Windows, both at home and at my company, to GNU/Linux, initially Red Hat Linux and then later Debian GNU/Linux, which eventually became my favorite GNU/Linux distribution for many years.
\n\nWhen I first saw The Complete FreeBSD book by Greg Lehey I remember noticing the text on the front page that said, "The Free Version of Berkeley UNIX" and "Rock Solid Stability", and I was immediately intrigued! What was that all about? A free UNIX operating system! And rock solid stability? That sounded amazing.
\n
\n\n\nIn late December 2019, Hyperbola announced that they would be making major changes to their project. They have decided to drop the Linux kernel in favor of forking the OpenBSD kernel. This announcement only came months after Project Trident announced that they were going in the opposite direction (from BSD to Linux).
\n\nHyperbola also plans to replace all software that is not GPL v3 compliant with new versions that are.
\n\nTo get more insight into the future of their new project, I interviewed Andre, co-founder of Hyperbola.
\n
\n\n\nThis month I have improved the NetBSD ptrace(2) API, removing one legacy interface with a few flaws and replacing it with two new calls with new features, and removing technical debt.
\n\nAs LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0).
\n
\n\n\nFreeBSD has existed as an operating system, project, and foundation for more than twenty years, and its earlier incantations have exited for far longer. The old guard have been developing code, porting software, and writing documentation for longer than I’ve existed. I’ve been using it for more than a decade for personal projects, and professionally for half that time.
\n\nWhile there are many prominent Australian FreeBSD contributors, sysadmins, and users, we’ve always had to venture overseas for conferences. We’re always told Australians are among the most ardent travellers, but I always wondered if we could do a domestic event as well.
\n\nAnd on Tuesday, we did! Deb Goodkin and the FreeBSD Foundation graciously organised and chaired a dedicated FreeBSD miniconf at the long-running linux.conf.au event held each year in a different city in Australia and New Zealand.
\n
\n\n\nThis is a simple write-up to setup Docker on FreeNAS 11 or FreeBSD 11.
\n
But muh jails?
\n\n\n\n\nYou know that jails are dope and you know that jails are dope, yet no one else knows it. So here we are stuck with docker. Two years ago I would be the last person to recommend using docker, but a whole lot of things has changes past years…
\n
So jails are dead then?
\n\n\n\n\nNo, jails are still dope, but jails lack tools to manage them. Yes, there are a few tools, but they meant for hard-core FreeBSD users who used to suffering. Docker allows you to run applications without deep knowledge of application you’re running. It will also allow you to run applications that are not ported to FreeBSD.
\n
\n\n\nAs an operating system GNU/Linux has become a real mess because of the fragmented nature of the project, the bloatware in the kernel, and because of the jerking around by commercial interests.
\n
Upgrading FreeBSD from 11.3 to 12.1, Distrowatch switching to FreeBSD, Torvalds says don’t run ZFS, iked(8) removed automatic IPv6 blocking, working towards LLDB on i386, and memory-hard Argon2 hashing scheme in NetBSD.
\n\n\n\n\nNow here’s something more like what I was originally expecting the content on this blog to look like. I’m in the process of moving all of our FreeBSD servers (about 30 in total) from 11.3 to 12.1. We have our own local build of the OS, and until “packaged base” gets to a state where it’s reliably usable, we’re stuck doing upgrades the old-fashioned way. I created a set of notes for myself while cranking through these upgrades and I wanted to share them since they are not really work-specific and this process isn’t very well documented for people who haven’t been doing this sort of upgrade process for 25 years.
\n\nOur source and object trees are read-only exported from the build server over NFS, which causes things to be slow. /etc/make.conf and /etc/src.conf are symbolic links on all of our servers to the master copies in /usr/src so that make installworld can find the configuration parameters the system was built with.
\n
\n\n\nThis may be a little off-topic for this board (forgive me if it is, please). However, I wanted to say that I'm one of the people who works on DistroWatch (distrowatch.com) and this past week we had to deal with a server facing hardware failure. We had a discussion about whether to continue running Debian or switch to something else.
\n\nThe primary "something else" option turned out to be FreeBSD and it is what we eventually went with. It took a while to convert everything over from working with Debian GNU/Linux to FreeBSD 12 (some script incompatibilities, different paths, some changes to web server configuration, networking IPv6 troubles). But in the end we ended up with a good, FreeBSD-based experience.
\n\nSince the transition was successful, though certainly not seamless, I thought people might want to do a Q&A on the migration process. Especially for those thinking of making the same switch.
\n
\n\n\niked(8) no longer automatically blocks unencrypted outbound IPv6 packets. This feature was intended to avoid accidental leakage, but in practice was found to mostly be a cause of misconfiguration.
\n\nIf you previously used iked(8)'s -6 flag to disable this feature, it is no longer needed and should be removed from /etc/rc.conf.local if used.
\n
\n\n\n“Don’t use ZFS. It’s that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me.”
\n\nThis is what Linus Torvalds said in a mailing list to once again express his disliking for ZFS filesystem specially over its licensing.
\n\nTo avoid unnecessary confusion, this is more intended for Linux distributions, kernel developers and maintainers rather than individual Linux users.
\n
\n\n\nWe successfully incorporated the Argon2 reference implementation into NetBSD/amd64 for our 2019 Google Summer of Coding project. We introduced our project here and provided some hints on how to select parameters here. For our final report, we will provide an overview of what changes were made to complete the project.
\n\nThe Argon2 reference implementation, available here, is available under both the Creative Commons CC0 1.0 and the Apache Public License 2.0. To import the reference implementation into src/external, we chose to use the Apache 2.0 license for this project.
\n
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n\nIn February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support.
\n\nThroughout December I've continued working on our build bot maintenance, in particular enabling compiler-rt tests. I've revived and finished my old patch for extended register state (XState) in core dumps. I've started working on bringing proper i386 support to LLDB.
\n
Your Impact on FreeBSD in 2019, Wireguard on OpenBSD Router, Amazon now has FreeBSD/ARM 12, pkgsrc-2019Q4, The Joys of UNIX Keyboards, OpenBSD on Digital Ocean, and more.
\n\n\n\n\nIt’s hard to believe that 2019 is nearly over. It has been an amazing year for supporting the FreeBSD Project and community! Why do I say that? Because as I reflect over the past 12 months, I realize how many events we’ve attended all over the world, and how many lives we’ve touched in so many ways. From advocating for FreeBSD to implementing FreeBSD features, my team has been there to help make FreeBSD the best open source project and operating system out there.
\n\nIn 2019, we focused on supporting a few key areas where the Project needed the most help. The first area was software development. Whether it was contracting FreeBSD developers to work on projects like wifi support, to providing internal staff to quickly implement hardware workarounds, we’ve stepped in to help keep FreeBSD innovative, secure, and reliable. Software development includes supporting the tools and infrastructure that make the development process go smoothly, and we’re on it with team members heading up the Continuous Integration efforts, and actively involved in the clusteradmin and security teams.
\n\nOur advocacy efforts focused on recruiting new users and contributors to the Project. We attended and participated in 38 conferences and events in 21 countries. From giving FreeBSD presentations and workshops to staffing tables, we were able to have 1:1 conversations with thousands of attendees.
\n\nOur travels also provided opportunities to talk directly with FreeBSD commercial and individual users, contributors, and future FreeBSD user/contributors. We’ve seen an increase in use and interest in FreeBSD from all of these organizations and individuals. These meetings give us a chance to learn more about what organizations need and what they and other individuals are working on. The information helps inform the work we should fund.
\n
\n\n\nwireguard (wg) is a modern vpn protocol, using the latest class of encryption algorithms while at the same time promising speed and a small code base.
\n\nmodern crypto and lean code are also tenants of openbsd, thus it was a no brainer to migrate my router from openvpn over to wireguard.
\n\nmy setup : a collection of devices, both wired and wireless, that are nat’d through my router (openbsd 6.6) out via my vpn provider azire* and out to the internet using wg-quick to start wg.
\n\nrunning : doubtless this could be improved on, but currently i start wg manually when my router boots. this, and the nat'ing on the vpn interface mean its impossible for clients to connect to the internet without the vpn being up. as my router is on a ups and only reboots when a kernel patch requires it, it’s a compromise i can live with. run wg-quick (please replace vpn with whatever you named your wg .conf file.) and reload pf rules.
\n
\n\n\n\n\nAWS, the cloud division of Amazon, announced in December the next generation of its ARM processors, the Graviton2. This is a custom chip design with a 7nm architecture. It is based on 64-bit ARM Neoverse cores.
\n\nCompared to first-generation Graviton processors (A1), today’s new chips should deliver up to 7x the performance of A1 instances in some cases. Floating point performance is now twice as fast. There are additional memory channels and cache speed memory access should be much faster.
\n\nThe company is working on three types of Graviton2 EC2 instances that should be available soon. Instances with a “g” suffix are powered by Graviton2 chips. If they have a “d” suffix, it also means that they have NVMe local storage.
\n\n\n
\n\n- \n
General-purpose instances (M6g and M6gd)
- \n
Compute-optimized instances (C6g and C6gd)
- \n
Memory-optimized instances (R6g and R6gd)
You can choose instances with up to 64 vCPUs, 512 GiB of memory and 25 Gbps networking.
\n\nAnd you can see that ARM-powered servers are not just a fad. AWS already promises a 40% better price/performance ratio with ARM-based instances when you compare them with x86-based instances.
\n\nAWS has been working with operating system vendors and independent software vendors to help them release software that runs on ARM. ARM-based EC2 instances support Amazon Linux 2, Ubuntu, Red Hat, SUSE, Fedora, Debian and FreeBSD. It also works with multiple container services (Docker, Amazon ECS, and Amazon Elastic Kubernetes Service).
\n
\n\n\nThe pkgsrc developers are proud to announce the 65th quarterly release of pkgsrc, the cross-platform packaging system. pkgsrc is available with more than 20,000 packages, running on 23 separate platforms; more information on pkgsrc itself is available at https://www.pkgsrc.org/
\n\nIn total, 190 packages were added, 96 packages were removed, and 1,868 package updates (to 1388 unique packages) were processed since the pkgsrc-2019Q3 release. As usual, a large number of updates and additions were processed for packages for go (14), guile (11), perl (170), php (10), python (426), and ruby (110). This continues pkgsrc's tradition of adding useful packages, updating many packages to more current versions, and pruning unmaintained packages that are believed to have essentially no users.
\n
\n\n\nI fell in love with a dead keyboard layout.
\n\nA decade or so ago while helping a friends father clean out an old building, we came across an ancient Sun Microsystems server. We found it curious. Everything about it was different from what we were used to. The command line was black on white, the connectors strange and foreign, and the keyboard layout was bizarre.
\n\nWe never did much with it; turning it on made all the lights in his home dim, and our joint knowledge of UNIX was nonexistent. It sat in his bedroom for years supporting his television at the foot of his bed.
\n\nI never forgot that keyboard though. The thought that there was this alternative layout out there seemed intriguing to me.
\n
\n\n\nLast night I had a need to put together a new OpenBSD machine. Since I already use DigitalOcean for one of my public DNS servers I wanted to use them for this need but sadly like all too many of the cloud providers they don't support OpenBSD. Now they do support FreeBSD and I found a couple writeups that show how to use FreeBSD as a shim to install OpenBSD.
\n\nThey are both sort of old at this point and with OpenBSD 6.6 out I ran into a bit of a snag. The default these days is to use a GPT partition table to enable EFI booting. This is generally pretty sane but it looks to me like the FreeBSD droplet doesn't support this. After the installer rebooted the VM failed to boot, being unable to find the bootloader.
\n\nThankfully DigitalOcean has a recovery ISO that you can boot by simply switching to it and powering off and then on your Droplet.
\n
Announcing HyperbolaBSD, IPFW In-Kernel NAT setup on FreeBSD, Wayland and WebRTC enabled for NetBSD 9/Linux, LLDB Threading support ready for mainline, OpenSSH U2F/FIDO support in base, Dragonfly drm/i915: Update, and more.
\n\n\n\n\nDue to the Linux kernel rapidly proceeding down an unstable path, we are planning on implementing a completely new OS derived from several BSD implementations.
\n\nThis was not an easy decision to make, but we wish to use our time and resources to create a viable alternative to the current operating system trends which are actively seeking to undermine user choice and freedom.
\n\nThis will not be a "distro", but a hard fork of the OpenBSD kernel and userspace including new code written under GPLv3 and LGPLv3 to replace GPL-incompatible parts and non-free ones.
\n
\n\n\nFuture versions of Hyperbola will be using HyperbolaBSD which will have the new kernel, userspace and not be ABI compatible with previous versions.
\n\nHyperbolaBSD is intended to be modular and minimalist so other projects will be able to re-use the code under free license.
\n
\n\n\nAfter graduating college, I am moving from Brooklyn, NY to Redmond, WA (guess where I got a job). I always wanted to re-do my OPNsense firewall (currently a HP T730) with stock FreeBSD and IPFW’s in-kernel NAT.
\n\nWhy IPFW? Benchmarks have shown IPFW to be faster which is especially good for my Tor relay, and because I can! However, one downside of IPFW is less documentation vs PF, even less without natd (which we’re not using), and this took me time to figure this out.
\n\nBut since my T730 is already packed, I am testing this on a old PC with two NICs, and my laptop [1] as a client with an USB-to-Ethernet adapter.
\n
\n\n\nThis is just a heads up that the Wayland option is now turned on by
\n
default for NetBSD 9 and Linux in cases where it peacefully coexists
\nwith X11.
\n\n\nThe WebRTC option has also been enabled by default on NetBSD 9 for two Firefox versions: www/firefox, www/firefox68
\n\nPlease keep me informed of any fallout. Hopefully, there will be none.
\n\nIf you want to try out Wayland-related things on NetBSD 9, wm/velox/MESSAGE may be interesting for you.
\n
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n\nIn February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.
\n\nSo far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.
\n
\n\n\nHardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.
\n\nYou'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.
\n\nSo, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.
\n
How learning OpenBSD makes computers suck a little less, How Unix works, FreeBSD 12.1 Runs Well on Ryzen Threadripper 3970X, BSDCan CFP, HardenedBSD Infrastructure Goals, and more.
\n\n\n\n\nHow much better could things actually be if we abandoned the enterprise development model?
\n\nNext I will compare this enterprise development approach with non-enterprise development - projects such as OpenBSD, which do not hesitate to introduce ABI breaking changes to improve the codebase.
\n\nOne of the most commonly referred to pillars of the project's philosophy has long been its emphasis on clean functional code. Any code which makes it into OpenBSD is subject to ongoing aggressive audits for deprecated, or otherwise unmaintained code in order to reduce cruft and attack surface. Additionally the project creator, Theo de Raadt, and his team of core developers engage in ongoing development for proactive mitigations for various attack classes many of which are directly adopted by various multi-platform userland applications as well as the operating systems themselves (Windows, Linux, and the other BSDs). Frequently it is the case that introducing new features (not just deprecating old ones) introduces new incompatibilities against previously functional binaries compiled for OpenBSD.
\n\nTo prevent the sort of kernel memory bloat that has plagued so many other operating systems for years, the project enforces a hard ceiling on the number of lines of code that can ever be in ring 0 at a given time. Current estimates guess the number of bugs per line of code in the Linux kernel are around 1 bug per every 10,000 lines of code. Think of this in the context of the scope creep seen in the Linux kernel (which if I recall correctly is currently at around 100,000,000 lines of code), as well as the Windows NT kernel (500,000,000 lines of code) and you quickly begin to understand how adding more and more functionality into the most privileged components of the operating system without first removing old components begins to add up in terms of the drastic difference seen between these systems in the number of zero day exploits caught in the wild respectively.
\n
\n\n\nUnix is beautiful. Allow me to paint some happy little trees for you. I’m not going to explain a bunch of commands – that’s boring, and there’s a million tutorials on the web doing that already. I’m going to leave you with the ability to reason about the system.
\n\nEvery fancy thing you want done is one google search away.
\n\nBut understanding why the solution does what you want is not the same.
\n\nThat’s what gives you real power, the power to not be afraid.
\n\nAnd since it rhymes, it must be true.
\n
\n\n\nFor those of you interested in AMD's new Ryzen Threadripper 3960X/3970X processors with TRX40 motherboards for running FreeBSD, the experience in our initial testing has been surprisingly pleasant. In fact, it works out-of-the-box which one could argue is better than the current Linux support that needs the MCE workaround for booting. Here are some benchmarks of FreeBSD 12.1 on the Threadripper 3970X compared to Linux and Windows for this new HEDT platform.
\n\nIt was refreshing to see FreeBSD 12.1 booting and running just fine with the Ryzen Threadripper 3970X 32-core/64-thread processor from the ASUS ROG ZENITH II EXTREME motherboard and all core functionality working including the PCIe 4.0 NVMe SSD storage, onboard networking, etc. The system was running with 4 x 16GB DDR4-3600 memory, 1TB Corsair Force MP600 NVMe SSD, and Radeon RX 580 graphics. It was refreshing to see FreeBSD 12.1 running well with this high-end AMD Threadripper system considering Linux even needed a boot workaround.
\n\nWhile the FreeBSD 12.1 experience was trouble-free with the ASUS TRX40 motherboard (ROG Zenith II Extreme) and AMD Ryzen Threadripper 3970X, DragonFlyBSD unfortunately was not. Both DragonFlyBSD 5.6.2 stable and the DragonFlyBSD daily development snapshot from last week were yielding a panic on boot. So with that, DragonFlyBSD wasn't tested for this Threadripper 3970X comparison but just FreeBSD 12.1.
\n\nFreeBSD 12.1 on the Threadripper 3970X was benchmarked both with its default LLVM Clang 8.0.1 compiler and again with GCC 9.2 from ports for ruling out compiler differences. The FreeBSD 12.1 performance was compared to last week's Windows 10 vs. Linux benchmarks with the same system.
\n
\n\n\nBSDCan 2020 will be held 5-6 (Fri-Sat) June, 2020 in Ottawa, at the University of Ottawa. It will be preceded by two days of tutorials on 3-4 June (Wed-Thu).
\n\nNOTE the change of month in 2020 back to June Also: do not miss out on the Goat BOF on Tuesday 2 June.
\n\nWe are now accepting proposals for talks. The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.
\n
\n\n\nIf you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include:
\n
\n\n\nFrom the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples.
\n\nBoth users and developers are encouraged to share their experiences.
\n
\n\n\n2019 has been an extremely productive year with regards to HardenedBSD's infrastructure. Several opportunities aligned themselves in such a way as to open a door for a near-complete rebuild with a vast expansion.
\n\nThe last few months especially have seen a major expansion of our infrastructure. We obtained a number of to-be-retired Dell R410 servers. The crash of our nightly build server provided the opportunity to deploy these R410 servers, doubling our build capacity.
\n\nMy available time to spend on HardenedBSD has decreased compared to this time last year. As part of rebuilding our infrastructure, I wanted to enable the community to be able to contribute. I'm structuring the work such that help is just a pull request away. Those in the HardenedBSD community who want to contribute to the infrastructure work can simply open a pull request. I'll review the code, and deploy it after a successful review. Users/contributors don't need access to our servers in order to improve them.
\n\nMy primary goal for the rest of 2019 and into 2020 is to become fully self-hosted, with the sole exception of email. I want to transition the source-of-truth git repos to our own infrastructure. We will still provide a read-only mirror on GitHub.
\n\nAs I develop this infrastructure, I'm doing so with human rights in mind. HardenedBSD is in a very unique position. In 2020, I plan to provide production Tor Onion Services for the various bits of our infrastructure. HardenedBSD will provide access to its various internal services to its developers and contributors. The entire development lifecycle, going from dev to prod, will be able to happen over Tor.
\n\nTransparency will be key moving forward. Logs for the auto-sync script are now published directly to GitHub. Build logs will be, soon, too. Logs of all automated processes, and the code for those processes, will be tracked publicly via git. This will be especially crucial for development over Tor.
\n\nIntegrating Tor into our infrastructure so deeply increases risk and maintenance burden. However, I believe that through added transparency, we will be able to mitigate risk. Periodic audits will need to be performed and published.
\n\nI hope to migrate HardenedBSD's site away from Drupal to a static site generator. We don't really need the dynamic capabilities Drupal gives us. The many security issues Drupal and PHP both bring also leave much to be desired.
\n\nSo, that's about it. I spent the last few months of 2019 laying the foundation for a successful 2020. I'm excited to see how the project grows.
\n
Authentication Vulnerabilities in OpenBSD, NetBSD 9.0 RC1 is available, Running FreeNAS on a DigitalOcean droplet, NomadBSD 1.3 is here, at e2k19 nobody can hear you scream, and more.
\n\n\n\n\nOpenBSD uses BSD Authentication, which is made up of a variety of authentication styles. The authentication styles currently provided are:
\n
\n passwd Request a password and check it against the password in the master.passwd file. See login_passwd(8).
\n skey Send a challenge and request a response, checking it with S/Key (tm) authentication. See login_skey(8).
\n yubikey Authenticate using a Yubico YubiKey token. See login_yubikey(8).
\n For any given style, the program /usr/libexec/auth/login_style is used to
\n perform the authentication. The synopsis of this program is:
\n /usr/libexec/auth/login_style [-v name=value] [-s service] username class
\n\n\n\nlogin_passwd [-s service] [-v wheel=yes|no] [-v lastchance=yes|no] user [class] The service argument specifies which protocol to use with the invoking program. The allowed protocols are login, challenge, and response. (The challenge protocol is silently ignored but will report success as passwd-style authentication is not challenge-response based).\n
Here are a few highlights of the new release:
\n\n\nSupport for Arm AArch64 (64-bit Armv8-A) machines, including "Arm ServerReady"
\n
\ncompliant machines (SBBR+SBSA)
\nEnhanced hardware support for Armv7-A
\nUpdated GPU drivers (e.g. support for Intel Kabylake)
\nEnhanced virtualization support
\nSupport for hardware-accelerated virtualization (NVMM)
\nSupport for Performance Monitoring Counters
\nSupport for Kernel ASLR
\nSupport several kernel sanitizers (KLEAK, KASAN, KUBSAN)
\nSupport for userland sanitizers
\nAudit of the network stack
\nMany improvements in NPF
\nUpdated ZFS
\nReworked error handling and NCQ support in the SATA subsystem
\nSupport a common framework for USB Ethernet drivers (usbnet)
More information on the RC can be found on the NetBSD 9 release page
\n\n\nBase of a FreeBSD droplet, we'll re-image our boot block device with FreeNAS iso. We'll then install FreeNAS on the second block device. Once done we're going to do the ol' switcheroo: we're going to re-image our original boot block device using the now FreeNAS-installed second block device.
\n
\n\n\nThe base system has been changed to FreeBSD 12.1-RELEASE-p1
\n\n
\n Due to a deadlock problem, FreeBSD's unionfs has been replaced by unionfs-fuse
\n The GPT layout has been changed to MBR. This prevents problems with Lenovo
\n systems that refuse to boot from GPT if "lenovofix" is not set, and systems that
\n hang on boot if "lenovofix" is set.
\n Support for ZFS installations has been added to the NomadBSD installer.
\n The rc-script for setting up the network interfaces has been fixed and improved.
\n Support for setting the country code for the wlan device has been added.
\n Auto configuration for running in VirtualBox has been added.
\n A check for the default display has been added to the graphics configuration scripts. This fixes problems where users with Optimus have their NVIDIA card disabled, and use the integrated graphics chip instead.
\n NVIDIA driver version 440 has been added.
\n nomadbsd-dmconfig, a Qt tool for selecting the display manager theme, setting the
\ndefault user and autologin has been added.
\n nomadbsd-adduser, a Qt tool for added preconfigured user accounts to the system has been added.
\n Martin Orszulik added Czech translations to the setup and installation wizard.
\n The NomadBSD logo, designed by Ian Grindley, has been changed.
\n Support for localized error messages has been added.
\n Support for localizing the password prompts has been added.
\n Some templates for starting other DEs have been added to ~/.xinitrc.
\n The interfaces of nomadbsd-setup-gui and nomadbsd-install-gui have been improved.
\n A script that helps users to configure a multihead systems has been added.
\n The Xorg driver for newer Intel GPUs has been changed from "intel" to "modesetting".
\n /proc has been added to /etc/fstab
\n A D-Bus session issue has been fixed which prevented thunar from accessing samba shares.
\n DSBBg which allows users to change and manage wallpapers has been added.
\n The latest version of update_obmenu now supports auto-updating the Openbox menu. Manually updating the Openbox menu after packet (de)installation is therefore no longer needed.Support for multiple keyboard layouts has been added.
\n
\n www/palemoon has been removed.
\n mail/thunderbird has been removed.
\n audio/audacity has been added.
\n deskutils/orage has been added.
\n the password manager fpm2 has been replaced by KeePassXC
\n mail/sylpheed has been replaced by mail/claws-mail
\n multimedia/simplescreenrecorder has been added.
\n DSBMC has been changed to DSBMC-Qt
\n Many small improvements and bug fixes.
Special Guest: Mariusz Zaborski.
","summary":"Authentication Vulnerabilities in OpenBSD, NetBSD 9.0 RC1 is available, Running FreeNAS on a DigitalOcean droplet, NomadBSD 1.3 is here, at e2k19 nobody can hear you scream, and more.","date_published":"2019-12-26T08:00:00.000-05:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/af84425c-c562-4d3b-b28c-cce7a148a3ad.mp3","mime_type":"audio/mp3","size_in_bytes":54074955,"duration_in_seconds":4506}]},{"id":"ca9f1431-2af7-48ad-98d6-e68c253ec75b","title":"329: Lucas’ Arts","url":"https://www.bsdnow.tv/329","content_text":"In this episode, we interview Michael W. Lucas about his latest book projects, including the upcoming SNMP Mastery book.\n\nInterview - Michael Lucas\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n\n\n\n\n\n \n Your browser does not support the HTML5 video tag.\nSpecial Guest: Michael W Lucas.","content_html":"In this episode, we interview Michael W. Lucas about his latest book projects, including the upcoming SNMP Mastery book.
\n\nSpecial Guest: Michael W Lucas.
","summary":"In this episode, we interview Michael W. Lucas about his latest book projects, including the upcoming SNMP Mastery book.","date_published":"2019-12-19T08:00:00.000-05:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/ca9f1431-2af7-48ad-98d6-e68c253ec75b.mp3","mime_type":"audio/mp3","size_in_bytes":36780535,"duration_in_seconds":3065}]},{"id":"be8ded86-58b0-46af-ba11-af5a748bc3d8","title":"328: EPYC Netflix Stack","url":"https://www.bsdnow.tv/328","content_text":"LLDB Threading support now ready, Multiple IPSec VPN tunnels with FreeBSD, Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance, happy eyeballs with unwind(8), AWS got FreeBSD ARM 12, OpenSSH U2F/FIDO support, and more.\n\nHeadlines\n\nLLDB Threading support now ready for mainline\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.\n\nIn February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.\n\nSo far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.\n\n\n\n\nMultiple IPSec VPN tunnels with FreeBSD\n\n\nThe FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html)\n\n\nBut it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below.\n\n\nThe requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE).\n\nEach location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C).\n\nVPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C).\n\n\n\n\nNews Roundup\n\nNetflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance\n\n\nDrew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel.\n\nNetflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made.\n\nFor those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%.\n\n\n\n\nunwind(8); \"happy eyeballs\"\n\n\nIn case you are wondering why happy eyeballs: It's a variation on this:\nhttps://en.wikipedia.org/wiki/Happy_Eyeballs\n\nunwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers.\n\nThis diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix. \n\nOne other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E):\n 17 files changed, 385 insertions(+), 1683 deletions(-)\n\nPlease test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals.\n\n\n\n\nAmazon now has FreeBSD ARM 12\n\n\nProduct Overview\n\nFreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years.\n\nFreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems.\n\n\n\n\nOpenSSH U2F/FIDO support in base\n\n\nI just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys.\n\nHardware backed keys can be generated using \"ssh-keygen -t ecdsa-sk\" (or \"ed25519-sk\" if your token supports it). Many tokens require to be touched/tapped to confirm this step.\n\nYou'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a \"key handle\" that is used by the security key to derive the real private key at signing time.\n\nSo, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything. \n\nOnce you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too.\n\nPlease test this thoroughly - it's a big change that we want to have stable before the next release.\n\n\n\n\nBeastie Bits\n\n\nDragonFly - git: virtio - Fix LUN scan issue w/ Google Cloud\nReally fast Markov chains in ~20 lines of sh, grep, cut and awk\nFreeBSD Journal Sept/Oct 2019\nMichael Dexter is raising money for Bhyve development\nsyscall call-from verification\nFreeBSD Forums Howto Section\n\n\n\n\nFeedback/Questions\n\n\nJeroen - Feedback\nSavo - pfsense ports\nTin - I want to learn C\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n\n\n\n\n\n \n Your browser does not support the HTML5 video tag.\n","content_html":"LLDB Threading support now ready, Multiple IPSec VPN tunnels with FreeBSD, Netflix Optimized FreeBSD's Network Stack More Than Doubled AMD EPYC Performance, happy eyeballs with unwind(8), AWS got FreeBSD ARM 12, OpenSSH U2F/FIDO support, and more.
\n\n\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n\nIn February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.
\n\nSo far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.
\n
\n\n\nThe FreeBSD handbook describes an IPSec VPN tunnel between 2 FreeBSD hosts (see https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html)
\n
But it is also possible to have multiple, 2 or more, IPSec VPN tunnels created and running on a FreeBSD host. How to implement and configure this is described below.
\n\n\n\n\nThe requirements is to have 3 locations (A, B and C) connected with IPSec VPN tunnels using FreeBSD (11.3-RELEASE).
\n\nEach location has 1 IPSec VPN host running FreeBSD (VPN host A, B and C).
\n\nVPN host A has 2 IPSec VPN tunnels: 1 to location B (VPN host B) and 1 to location C (VPN host C).
\n
\n\n\nDrew Gallatin of Netflix presented at the recent EuroBSDcon 2019 conference in Norway on the company's network stack optimizations to FreeBSD. Netflix was working on being able to deliver 200Gb/s network performance for video streaming out of Intel Xeon and AMD EPYC servers, to which they are now at 190Gb/s+ and in the process that doubled the potential of EPYC Naples/Rome servers and also very hefty upgrades too for Intel.
\n\nNetflix has long been known to be using FreeBSD in their data centers particularly where network performance is concerned. But in wanting to deliver 200Gb/s throughput from individual servers led them to making NUMA optimizations to the FreeBSD network stack. Allocating NUMA local memory for kernel TLS crypto buffers and for backing files sent via sentfile were among their optimizations. Changes to network connection handling and dealing with incoming connections to Nginx were also made.
\n\nFor those just wanting the end result, Netflix's NUMA optimizations to FreeBSD resulted in their Intel Xeon servers going from 105Gb/s to 191Gb/s while the NUMA fabric utilization dropped from 40% to 13%.
\n
\n\n\nIn case you are wondering why happy eyeballs: It's a variation on this:
\n\n
\nhttps://en.wikipedia.org/wiki/Happy_Eyeballsunwind has a concept of a best nameserver type. It considers a configured DoT nameserver to be better than doing it's own recursive resolving. Recursive resolving is considered to be better than asking the dhcp provided nameservers.
\n\nThis diff sorts the nameserver types by quality, as above (validation, resolving, dead...), and as a tie breaker it adds the median of the round trip time of previous queries into the mix.
\n\nOne other interesting thing about this is that it gets us past captive portals without a check URL, that's why this diff is so huge, it rips out all the captive portal stuff (please apply with patch -E):
\n\n
\n 17 files changed, 385 insertions(+), 1683 deletions(-)Please test this. I'm particularly interested in reports from people who move between networks and need to get past captive portals.
\n
\n\n\nProduct Overview
\n\nFreeBSD is an operating system used to power servers, desktops, and embedded systems. Derived from BSD, the version of UNIX developed at the University of California, Berkeley, FreeBSD has been continually developed by a large community for more than 30 years.
\n\nFreeBSD's networking, security, storage, and monitoring features, including the pf firewall, the Capsicum and CloudABI capability frameworks, the ZFS filesystem, and the DTrace dynamic tracing framework, make FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage systems.
\n
\n\n\nI just committed all the dependencies for OpenSSH security key (U2F) support to base and tweaked OpenSSH to use them directly. This means there will be no additional configuration hoops to jump through to use U2F/FIDO2 security keys.
\n\nHardware backed keys can be generated using "ssh-keygen -t ecdsa-sk" (or "ed25519-sk" if your token supports it). Many tokens require to be touched/tapped to confirm this step.
\n\nYou'll get a public/private keypair back as usual, except in this case, the private key file does not contain a highly-sensitive private key but instead holds a "key handle" that is used by the security key to derive the real private key at signing time.
\n\nSo, stealing a copy of the private key file without also stealing your security key (or access to it) should not give the attacker anything.
\n\nOnce you have generated a key, you can use it normally - i.e. add it to an agent, copy it to your destination's authorized_keys files (assuming they are running -current too), etc. At authentication time, you will be prompted to tap your security key to confirm the signature operation - this makes theft-of-access attacks against security keys more difficult too.
\n\nPlease test this thoroughly - it's a big change that we want to have stable before the next release.
\n
We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more.
\n\n\n\n\nThis quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work).
\n\nEfficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner.
\n\nStarting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October.
\n\nNext quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January.
\n
\n\n\nOpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available.
\n\nFirst I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release.
\n\nVersion 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions.
\n
\n\n\nBecause it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs.
\n\nThe next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020.
\n
\n\n\nA long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback.
\n
\n\n\nSometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device.
\n\nGNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks.
\n
\n\n\nThis is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD.
\n\nGoals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.
\n\nThis tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform.
\n
LPI releases BSD Certification, openzfs trip report, Using FreeBSD with ports, LLDB threading support ready, Linux versus Open Source Unix, and more.
\n\n\n\n\nLinux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018.
\n\nG. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.”
\n
\n\n\nThe seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling.
\n
\n\n\nIn the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient.
\n\nIn this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today.
\n
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n\nIn February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report.
\n\nSo far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report.
\n
FreeBSD 12.1 is here, A history of Unix before Berkeley, FreeBSD development setup, HardenedBSD 2019 Status Report, DNSSEC, compiling RainbowCrack on OpenBSD, and more.
\n\nSome of the highlights:
\n\nFor a complete list of new features and known problems, please see the online release notes and errata list, available at: https://www.FreeBSD.org/releases/12.1R/relnotes.html
\n\n\nNobody needs to be told that UNIX is popular today. In this article we will show you a little of where it was yesterday and over the past decade. And, without meaning in the least to minimise the incredible contributions of Ken Thompson and Dennis Ritchie, we will bring to light many of the others who worked on early versions, and try to show where some of the key ideas came from, and how they got into the UNIX of today.
\n\nOur title says we are talking about UNIX evolution. Evolution means different things to different people. We use the term loosely, to describe the change over time among the many different UNIX variants in use both inside and outside Bell Labs. Ideas, code, and useful programs seem to have made their way back and forth - like mutant genes - among all the many UNIXes living in the phone company over the decade in question.
\n\nPart One looks at some of the major components of the current UNIX system - the text formatting tools, the compilers and program development tools, and so on. Most of the work described in Part One took place at
\nResearch'', a part of Bell Laboratories (now AT&T Bell Laboratories, then as now
the Labs''), and the ancestral home of UNIX. In planned (but not written) later parts, we would have looked at some of the myriad versions of UNIX - there are far more than one might suspect. This includes a look at Columbus and USG and at Berkeley Unix. You'll begin to get a glimpse inside the history of the major streams of development of the system during that time.
\n\n\nI do my FreeBSD development using git, tmux, vim and cscope.
\n\nI keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd
\n
\n\n\nAs we are experiencing the Suricata community first hand in Amsterdam we thought to release this version a bit earlier than planned. Included is the latest Suricata 5.0.0 release in the development version. That means later this November we will releasing version 5 to the production version as we finish up tweaking the integration and maybe pick up 5.0.1 as it becomes available.
\n\nLDAP TLS connectivity is now integrated into the system trust store, which ensures that all required root and intermediate certificates will be seen by the connection setup when they have been added to the authorities section. The same is true for trusting self-signed certificates. On top of this, IPsec now supports public key authentication as contributed by Pascal Mathis.
\n
\n\n\nWe at HardenedBSD have a lot of news to share. On 05 Nov 2019, Oliver Pinter resigned amicably from the project. All of us at HardenedBSD owe Oliver our gratitude and appreciation. This humble project, named by Oliver, was born out of his thesis work and the collaboration with Shawn Webb. Oliver created the HardenedBSD repo on GitHub in April 2013. The HardenedBSD Foundation was formed five years later to carry on this great work.
\n
\n\n\nDNSSEC validation has been enabled in the default unbound.conf(5) in -current. The relevant commits were from Job Snijders (job@)
\n
\n\n\nShopware is the next generation of open source e-commerce software. Based on bleeding edge technologies like Symfony 3, Doctrine2 and Zend Framework Shopware comes as the perfect platform for your next e-commerce project. This tutorial will walk you through the Shopware Community Edition (CE) installation on FreeBSD 12 system by using NGINX as a web server.
\n
\n\n\nMake sure your system meets the following minimum requirements:
\n\n\n
\n- Linux-based operating system with NGINX or Apache 2.x (with mod_rewrite) web server installed.
\n- PHP 5.6.4 or higher with ctype, gd, curl, dom, hash, iconv, zip, json, mbstring, openssl, session, simplexml, xml, zlib, fileinfo, and pdo/mysql extensions. PHP 7.1 or above is strongly recommended.
\n- MySQL 5.5.0 or higher.
\n- Possibility to set up cron jobs.
\n- Minimum 4 GB available hard disk space.
\n- IonCube Loader version 5.0.0 or higher (optional).
\n
\n\n\nProject RainbowCrack was originally Zhu Shuanglei's implementation, it's not clear to me if the project is still just his or if it's even been maintained for a while. His page seems to have been last updated in August 2007.
\n\nThe Project RainbowCrack web page now has just binaries for Windows XP and Linux, both 32-bit and 64-bit versions.
\n\nEarlier versions were available as source code. The version 1.2 source code does not compile on OpenBSD, and in my experience it doesn't compile on Linux, either. It seems to date from 2004 at the earliest, and I think it makes some version-2.4 assumptions about Linux kernel headers.
\n
Migrating drives and zpool between hosts, OpenBSD in 2019, Dragonfly’s new zlib and dhcpcd, Batch renaming images and resolution with awk, a rant on the X11 ICCCM selection system, hammer 2 emergency space mode, and more.
\n\n\n\n\nToday is the day.
\n\nToday I move a zpool from an R710 into an R720. The goal: all services on that zpool start running on the new host.
\n\nFortunately, that zpool is dedicated to jails, more or less. I have done some planning about this, including moving a poudriere on the R710 into a jail.
\n\nNow it is almost noon on Saturday, I am sitting in the basement (just outside the server room), and I’m typing this up.
\n
In this post:
\n\n\n\n\nI’ve used OpenBSD on and off since 2.1. More back then than in the last 10 years or so though, so I thought I’d try it again.
\n\nWhat triggered this was me finding a silly bug in GNU cpio that has existed with a “FIXME” comment since at least 1994. I checked OpenBSD to see if it had a related bug, but as expected no it was just fine.
\n\nI don’t quite remember why I stopped using OpenBSD for servers, but I do remember filesystem corruption on “unexpected power disconnections” (even with softdep turned on), which I’ve never really seen on Linux.
\n\nThat and that fewer things “just worked” than with Linux, which matters more when I installed more random things than I do now. I’ve become a lot more minimalist. Probably due to less spare time. Life is better when you don’t run things like PHP (not that OpenBSD doesn’t support PHP, just an example) or your own email server with various antispam tooling, and other things.
\n\nThis is all experience from running OpenBSD on a server. On my next laptop I intend to try running OpenBSD on the dektop, and will see if that more ad-hoc environment works well. E.g. will gnuradio work? Lack of other-OS VM support may be a problem.
\n
\n\n\nOuch, that’s a long list of bad stuff. Still, I like it. I’ll continue to run it, and will make sure my stuff continues working on OpenBSD.
\n\nAnd maybe in a year I’ll have a review of OpenBSD on a laptop.
\n
\n\n\nzlib and dhcpcd are both updated in DragonFly… but my quick perusal of the commits makes it sound like bugfix only; no usage changes needed.
\n
\n\n\nThe most recent item on my list of “Geeky things I did that made me feel pretty awesome” is an hour’s adventure that culminated in this code:
\n
$ file IMG* | awk 'BEGIN{a=0} {print substr($1, 1, length($1)-5),a++"_"substr($8,1, length($8)-1)}' | while read fn fr; do echo $(rename -v "s/$fn/img_$fr/g" *); done\nIMG_20170808_172653_425.jpg renamed as img_0_4032x3024.jpg\nIMG_20170808_173020_267.jpg renamed as img_1_3024x3506.jpg\nIMG_20170808_173130_616.jpg renamed as img_2_3024x3779.jpg\nIMG_20170808_173221_425.jpg renamed as img_3_3024x3780.jpg\nIMG_20170808_173417_059.jpg renamed as img_4_2956x2980.jpg\nIMG_20170808_173450_971.jpg renamed as img_5_3024x3024.jpg\nIMG_20170808_173536_034.jpg renamed as img_6_4032x3024.jpg\nIMG_20170808_173602_732.jpg renamed as img_7_1617x1617.jpg\nIMG_20170808_173645_339.jpg renamed as img_8_3024x3780.jpg\nIMG_20170909_170146_585.jpg renamed as img_9_3036x3036.jpg\nIMG_20170911_211522_543.jpg renamed as img_10_3036x3036.jpg\nIMG_20170913_071608_288.jpg renamed as img_11_2760x2760.jpg\nIMG_20170913_073205_522.jpg renamed as img_12_2738x2738.jpg\n// ... etc etc\n
\n\n\n\n\nThe last item on the aforementioned list is “TODO: come up with a shorter title for this list.”
\n
\n\n\nd00d, that document is devilspawn. I've recently spent my nights in pain
\n\n
\nimplementing the selection mechanism. WHY OH WHY OH WHY? why me? why did I choose to do this? and what sick evil twisted mind wrote this damn spec? I don't know why I'm working with it, I just wanted to make a useful program.I didn't know what I was getting myself in to. Nobody knows until they try it. And once you start, you're unable to stop. You can't stop, if you stop then you haven't completed it to spec. You can't fail on this, it's just a few pages of text, how can that be so hard? So what if they use Atoms for everything. So what if there's no explicit correlation between the target type of a SelectionNotify event and the type of the property it indicates?
\n\nSo what if the distinction is ambiguous? So what if the document is littered with such atrocities? It's not the spec's fault, the spec is authoritative. It's obviously YOUR (the implementor's) fault for misunderstanding it. If you didn't misunderstand it, you wouldn't be here complaining about it would you?
\n
\n\n\nAs anyone who has been running HAMMER1 or HAMMER2 has noticed, snapshots and copy on write and infinite history can eat a lot of disk space, even if the actual file volume isn’t changing much. There’s now an ‘emergency mode‘ for HAMMER2, where disk operations can happen even if there isn’t space for the normal history activity. It’s dangerous, in that the normal protections against data loss if power is cut go away, and snapshots created while in this mode will be mangled. So definitely don’t leave it on!
\n
The earliest Unix code, how to replace fail2ban with blacklistd, OpenBSD crossed 400k commits, how to install Bolt CMS on FreeBSD, optimized hammer2, appeasing the OSI 7-layer burrito guys, and more.
\n\n\n\n\nWhat is it that runs the servers that hold our online world, be it the web or the cloud? What enables the mobile apps that are at the center of increasingly on-demand lives in the developed world and of mobile banking and messaging in the developing world? The answer is the operating system Unix and its many descendants: Linux, Android, BSD Unix, MacOS, iOS—the list goes on and on. Want to glimpse the Unix in your Mac? Open a Terminal window and enter “man roff” to view the Unix manual entry for an early text formatting program that lives within your operating system.
\n\n2019 marks the 50th anniversary of the start of Unix. In the summer of 1969, that same summer that saw humankind’s first steps on the surface of the Moon, computer scientists at the Bell Telephone Laboratories—most centrally Ken Thompson and Dennis Ritchie—began the construction of a new operating system, using a then-aging DEC PDP-7 computer at the labs.
\n
\n\n\nIt was supposed to say "log," but the computer sending the message — based at UCLA — crashed before the letter "g" was typed. A computer at Stanford 560 kilometres away was supposed to fill in the remaining characters "in," as in "log in."
\n
\n\n\n"The idea of the network was you could sit at one computer, log on through the network to a remote computer and use its services there,"
\n\n50 years later, the internet has become so ubiquitous that it has almost been rendered invisible. There's hardly an aspect in our daily lives that hasn't been touched and transformed by it.
\n\nQ: Take us back to that day 50 years ago. Did you have the sense that this was going to be something you'd be talking about a half a century later?
\n\nA: Well, yes and no. Four months before that message was sent, there was a press release that came out of UCLA in which it quotes me as describing what my vision for this network would become. Basically what it said is that this network would be always on, always available. Anybody with any device could get on at anytime from any location, and it would be invisible.
\n\nWell, what I missed ... was that this is going to become a social network. People talking to people. Not computers talking to computers, but [the] human element.
\n\nQ: Can you briefly explain what you were working on in that lab? Why were you trying to get computers to actually talk to one another?
\n\nA: As an MIT graduate student, years before, I recognized I was surrounded by computers and I realized there was no effective [or efficient] way for them to communicate. I did my dissertation, my research, on establishing a mathematical theory of how these networks would work. But there was no such network existing. AT&T said it won't work and, even if it does, we want nothing to do with it.
\n\nSo I had to wait around for years until the Advanced Research Projects Agency within the Department of Defence decided they needed a network to connect together the computer scientists they were supervising and supporting.
\n\nQ: For all the promise of the internet, it has also developed some dark sides that I'm guessing pioneers like yourselves never anticipated.
\n\nA: We did not. I knew everybody on the internet at that time, and they were all well-behaved and they all believed in an open, shared free network. So we did not put in any security controls.
\n\nWhen the first spam email occurred, we began to see the dark side emerge as this network reached nefarious people sitting in basements with a high-speed connection, reaching out to millions of people instantaneously, at no cost in time or money, anonymously until all sorts of unpleasant events occurred, which we called the dark side.
\n\nBut in those early days, I considered the network to be going through its teenage years. Hacking to spam, annoying kinds of effects. I thought that one day this network would mature and grow up. Well, in fact, it took a turn for the worse when nation states, organized crime and extremists came in and began to abuse the network in severe ways.
\n\nQ: Is there any part of you that regrets giving birth to this?
\n\nA: Absolutely not. The greater good is much more important.
\n
\n\n\n\n\nblacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy.
\n\nThe interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf
\n\nNow, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use.
\n\nUnfortunately (dont' ask me why ??) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen.
\n
\n\n\nSometime in the last week OpenBSD crossed 400,000 commits (*) upon all our repositories since starting at 1995/10/18 08:37:01 Canada/Mountain. That's a lot of commits by a lot of amazing people.
\n\n(*) by one measure. Since the repository is so large and old, there are a variety of quirks including ChangeLog missing entries and branches not convertible to other repo forms, so measuring is hard. If you think you've got a great way of measuring, don't be so sure of yourself -- you may have overcounted or undercounted.
\n
\n\n\nBolt is a sophisticated, lightweight and simple CMS built with PHP. It is released under the open-source MIT-license and source code is hosted as a public repository on Github. A bolt is a tool for Content Management, which strives to be as simple and straightforward as possible. It is quick to set up, easy to configure, uses elegant templates. Bolt is created using modern open-source libraries and is best suited to build sites in HTML5 with modern markup. In this tutorial, we will go through the Bolt CMS installation on FreeBSD 12 system by using Nginx as a web server, MySQL as a database server, and optionally you can secure the transport layer by using acme.sh client and Let's Encrypt certificate authority to add SSL support.
\n
\n\n\nRefactor the XOP groups in order to be able to queue strategy calls, whenever possible, to the same CPU as the issuer. This optimizes several cases and reduces unnecessary IPI traffic between cores. The next best thing to do would be to not queue certain XOPs to an H2 support thread at all, but I would like to keep the threads intact for later clustering work.
\n\n
\nThe best scaling case for this is when one has a large number of user threads doing I/O. One instance of a single-threaded program on an otherwise idle machine might see a slightly reduction in performance but at the same time we completely avoid unnecessarily spamming all cores in the system on the behalf of a single program, so overhead is also significantly lower.This will tend to increase the number of H2 support threads since we need a certain degree of multiplication for domain separation.
\n\nThis should significantly increase I/O performance for multi-threaded workloads.
\n
\n\n\nI've seen the writing on the wall, and while for now you can configure Firefox not to use DoH, I'm not confident enough to think it will remain that way. To that end, I've finally set up my own DoH server for use at Chez Boca. It only involved setting up my own CA to generate the appropriate certificates, install my CA certificate into Firefox, configure Apache to run over HTTP/2 (THANK YOU SO VERY XXXXXXX MUCH GOOGLE FOR SHOVING THIS HTTP/2 XXXXXXXX DOWN OUR THROATS!—no, I'm not bitter) and write a 150 line script that just queries my own local DNS, because, you know, it's more XXXXXXX secure or some XXXXXXXX reason like that.
\n\nSigh.
\n
Michael - FreeNAS inside a Jail
\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more.
\n\n\n\n\nIn the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing.
\n
\n\n\nIn my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix.
\n
\n\n\nHello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version
\n
12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so.
\n\nHere are the full patch notes:
\n\n\n\n\nSince the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE.
\n\nOur official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release.
\n\nThere is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum.
\n
\n\n\nLooking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy.
\n
\n\n\nsshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN.
\n
Security
\n\nThis release includes a number of changes that may affect existing configurations:
\n\nNew Features
\n\nAn interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more.
\n\nRobot OS on FreeBSD
\n\nSpecial Guest: Trenton Shulz.
","summary":"An interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more.","date_published":"2019-10-23T23:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/fca983bf-93c9-460f-8c32-3b32663d463d.mp3","mime_type":"audio/mp3","size_in_bytes":39796738,"duration_in_seconds":3316}]},{"id":"11b9f24e-1789-4328-8396-4b9654aa2dfc","title":"320: Codebase: Neck Deep","url":"https://www.bsdnow.tv/320","content_text":"Headlines\n\nFreeBSD and custom firmware on the Google Pixelbook\n\n\nFreeBSD and custom firmware on the Google Pixelbook\n\n\n\nBack in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\\u2019t use the trackpoint much. But this summer I\\u2019ve decided that it was time for something newer. I wanted something..\n\n\n\nlighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);\nwith a 3:2 display (why is Lenovo making these Serious Work\\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially);\nwith a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);\nwith USB-C ports;\nwithout a dGPU, especially without an NVIDIA GPU;\nassembled with screws and not glue (I don\\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good);\nsupported by FreeBSD of course (\\u201csome development required\\u201d is okay but I\\u2019m not going to write big drivers);\nhow about something with open source firmware, that would be fun.\n\n\n\nI was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\\u2019s fun. But going back in processor generations just doesn\\u2019t feel great. I want something more efficient, not less!\n\nAnd then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\\u2019t every laptop ever doing that?!), the \\u201cconvertibleness\\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\\u2019 stock firmware is coreboot + depthcharge).\n\nSo here it is, my new laptop, a Google Pixelbook.\n\n\n\nConclusion\n\n\n\nPixelbook, FreeBSD, coreboot, EDK2 good.\n\nSeriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)\n\n\n\n\nPorting NetBSD to the AMD x86-64: a case study in OS portability\n\n\nAbstract\n\n\n\nNetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.\n\n\n\nPortability\n\n\n\nSupporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.\n\n\n\nGeneral\n\n\n\nGenerally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.\n\n\n\nTypes\n\n\n\nAssumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc.\n\n\n\nConclusions and future work\n\n\n\nThe port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.\n\n\n\n\nNews Roundup\n\nZFS performance really does degrade as you approach quota limits\n\n\nEvery so often (currently monthly), there is an \"OpenZFS leadership meeting\". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones. \n\n\n\nIn the September meeting notes, I read a very interesting (to me) agenda item: \n\n\nRelax quota semantics for improved performance (Allan Jude)\nProblem: As you approach quotas, ZFS performance degrades.\nProposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased.\n\n\n\n\nThis is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).\n\nSecond, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)\n\nWith that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.\n\n\n\n\nFixing up KA9Q-unix, or \"neck deep in 30 year old codebases..\"\n\n\nI'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.\n\nAnyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.\n\nSo yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.\n\nBut! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!\n\nI grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.\n\n\n\n\nHAMMER2 and fsck for review\n\n\nHAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.\n\n\n\n\n[The return of startx(1) for non-root users with some caveats\n\nMark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:\n\nCVSROOT: /cvs\nModule name: src\nChanges by: kettenis@cvs.openbsd.org 2019/09/15 06:25:41\n\nModified files:\n etc/etc.amd64 : fbtab \n etc/etc.arm64 : fbtab \n etc/etc.hppa : fbtab \n etc/etc.i386 : fbtab \n etc/etc.loongson: fbtab \n etc/etc.luna88k: fbtab \n etc/etc.macppc : fbtab \n etc/etc.octeon : fbtab \n etc/etc.sgi : fbtab \n etc/etc.sparc64: fbtab \n\nLog message:\nAdd ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.\n\nok jsg@, matthieu@\nCVSROOT: /cvs\nModule name: xenocara\nChanges by: kettenis@cvs.openbsd.org 2019/09/15 06:31:08\n\nModified files:\n xserver/hw/xfree86/common: xf86AutoConfig.c \n\nLog message:\nAdd modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.\n\nThis makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).\n\nok jsg@, matthieu@\n\n\n\n\nBeastie Bits\n\n\nASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal?\nSourcehut makes BSD software better\nChaosnet for Unx\nThe Vim-Inspired Editor with a Linguistic Twist\nbhyvearm64: CPU and Memory Virtualization on Armv8.0-A\nDefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities\n\n\n\n\nFeedback/Questions\n\n\nTim - GSoC project ideas for pf rule syntax translation\nBrad - Steam on FreeBSD\nRuslan - FreeBSD Quarterly Status Report - Q2 2019\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n\n\n\n\n\n \n Your browser does not support the HTML5 video tag.\n","content_html":"\n\n\nBack in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\\u2019t use the trackpoint much. But this summer I\\u2019ve decided that it was time for something newer. I wanted something..
\n
\n\n\nI was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\\u2019s fun. But going back in processor generations just doesn\\u2019t feel great. I want something more efficient, not less!
\n\nAnd then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\\u2019t every laptop ever doing that?!), the \\u201cconvertibleness\\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\\u2019 stock firmware is coreboot + depthcharge).
\n\nSo here it is, my new laptop, a Google Pixelbook.
\n
\n\n\nPixelbook, FreeBSD, coreboot, EDK2 good.
\n\nSeriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)
\n
\n\n\nNetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.
\n
\n\n\nSupporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.
\n
\n\n\nGenerally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.
\n
\n\n\nAssumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc.
\n
\n\n\nThe port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.
\n
\n\n\nEvery so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones.
\n
\n\n\nThis is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).
\n\nSecond, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)
\n\nWith that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.
\n
\n\n\nI'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.
\n\nAnyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.
\n\nSo yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.
\n\nBut! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!
\n\nI grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.
\n
\n\n\nHAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.
\n
Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:
\n\nCVSROOT: /cvs\nModule name: src\nChanges by: kettenis@cvs.openbsd.org 2019/09/15 06:25:41\n\nModified files:\n etc/etc.amd64 : fbtab \n etc/etc.arm64 : fbtab \n etc/etc.hppa : fbtab \n etc/etc.i386 : fbtab \n etc/etc.loongson: fbtab \n etc/etc.luna88k: fbtab \n etc/etc.macppc : fbtab \n etc/etc.octeon : fbtab \n etc/etc.sgi : fbtab \n etc/etc.sparc64: fbtab \n\nLog message:\nAdd ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.\n\nok jsg@, matthieu@\nCVSROOT: /cvs\nModule name: xenocara\nChanges by: kettenis@cvs.openbsd.org 2019/09/15 06:31:08\n\nModified files:\n xserver/hw/xfree86/common: xf86AutoConfig.c \n\nLog message:\nAdd modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.\n\nThis makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).\n\nok jsg@, matthieu@\n
\n\nCausing ZFS corruption for fun, NetBSD Assembly Programming Tutorial, The IKEA Lack Rack for Servers, a new OmniOS Community Edition LTS has been published, List Block Devices on FreeBSD lsblk(8) Style, Project Trident 19.10 available, and more.
\n\n\n\n\nDatto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios.
\n
\n\n\nSince this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets.
\n
\n\n\nAt the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events.
\n\nTo others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject.
\n
\n\n\nA sparc64 version is also being prepared and will be added when done
\n\nThis post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel.
\n
\n\n\nWhy not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs
\n
\n\n\nFirst occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009.
\n\nThe LackRack will certainly make its appearance again this summer at eth0:2010 Summer.
\n
\n\n\nWhen temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks.
\n\nThe LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented!
\n
\n\n\nYou can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat!
\n
\n\n\nThe OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030.
\n\nOmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details.
\n\nThis is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details.
\n\nIf you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28.
\n
\n\n\nWhen I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few: sockstat, gstat, top -b -o res, top -m io -o total, usbconfig, rcorder, beadm/bectl, idprio/rtprio,… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world.
\n
Example lsblk(8) output from Linux system:
\n\n$ lsblk\nNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT\nsr0 11:0 1 1024M 0 rom\nsda 8:0 0 931.5G 0 disk\n|-sda1 8:1 0 500M 0 part /boot\n`-sda2 8:2 0 931G 0 part\n |-vg_local-lv_root (dm-0) 253:0 0 50G 0 lvm /\n |-vg_local-lv_swap (dm-1) 253:1 0 17.7G 0 lvm [SWAP]\n `-vg_local-lv_home (dm-2) 253:2 0 1.8T 0 lvm /home\nsdc 8:32 0 232.9G 0 disk\n`-sdc1 8:33 0 232.9G 0 part\n `-md1 9:1 0 232.9G 0 raid10 /data\nsdd 8:48 0 232.9G 0 disk\n`-sdd1 8:49 0 232.9G 0 part\n `-md1 9:1 0 232.9G 0 raid10 /data\n
\n\n\n\n\nWhat FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs.
\n
\n\n\nThis is a general package update to the CURRENT release repository based upon TrueOS 19.10
\n
DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more.
\n\n\n\n\nFor those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04.
\n\nBack in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs.
\n\nWe've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs.
\n\nFor comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round.
\n\nAll of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test.
\n
\n\n\niXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete.
\n\nHaving first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection.
\n\nNot only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors.
\n\nWith precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes.
\n
\n\n\n\n\nFreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system.
\n\nFreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes.
\n\nFor those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures.
\n\nThe FreeBSD release team is planning for at least another beta or two and around three release candidates. If all goes well, FreeBSD 12.1 will be out in early November.
\n
\n\n\nThe fourth biennial vBSDCon was held in Reston, VA on September 5th through 7th and attracted attendees and presenters from not only the Washington, DC area, but also Canada, Germany, Kenya, and beyond. While MeetBSD caters to Silicon Valley BSD enthusiasts on even years, vBSDcon caters to East Coast and DC area enthusiasts on odd years. Verisign was again the key sponsor of vBSDcon 2019 but this year made a conscious effort to entrust the organization of the event to a team of community members led by Dan Langille, who you probably know as the lead BSDCan organizer. The result of this shift was a low key but professional event that fostered great conversation and brainstorming at every turn.
\n
\n\n\nI fear we're drifting a bit here and the S/N ratio is dropping a bit w.r.t the actual history of Unix. Please no more on the relative merits of version control systems or alternative text processing systems.
\n\nSo I'll try to distract you by saying this. I'm sitting on two artifacts that have recently been given to me:
\n
\n\n\nand I am going slowly crazy as I wait for them to be offically released. Now you have a new topic to talk about :-)
\n\nCheers, Warren
\n
* for some definition of "soon"
\n\nSetting up buildbot in FreeBSD jails, Set up a mail server with OpenSMTPD, Dovecot and Rspamd, OpenBSD amateur packet radio with HamBSD, DragonFlyBSD's HAMMER2 gets fsck, return of startx for users.
\n\n\n\n\nWe’re back from EuroBSDcon in Lillehammer, Norway. It was a great conference with 212 people attending. 2 days of tutorials, parallel to the FreeBSD Devsummit, followed by two days of talks. Some speakers uploaded their slides to papers.freebsd.org already with more to come.
\n\nThe social event was also interesting. We visited an open air museum with building preserved from different time periods. In the older section they had a collection of farm buildings, a church originally built in the 1200s and relocated to the museum, and a school house. In the more modern area, they had houses from 1915, and each decade from 1930 to 1990, plus a “house of the future” as imagined in 2001. Many had open doors to allow you to tour the inside, and some were even “inhabited”. The latter fact gave a much more interactive experience and we could learn additional things about the history of that particular house. The town at the end included a general store, a post office, and more. Then, we all had a nice dinner together in the museum’s restaurant.
\n
\n\n\nIn this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here.
\n
\n\n\nFirst of all, I was not clear enough about the political consequences of centralizing mail services at Big Mailer Corps.
\n\nIt doesn’t make sense for Random Joe, sharing kitten pictures with his family and friends, to build a personal mail infrastructure when multiple Big Mailer Corps offer “for free” an amazing quality of service. They provide him with an e-mail address that is immediately available and which will generally work reliably. It really doesn’t make sense for Random Joe not to go there, and particularly if even techies go there without hesitation, proving it is a sound choice.
\n\nThere is nothing wrong with Random Joes using a service that works.
\n\nWhat is terribly wrong though is the centralization of a communication protocol in the hands of a few commercial companies, EVERY SINGLE ONE OF THEM coming from the same country (currently led by a lunatic who abuses power and probably suffers from NPD), EVERY SINGLE ONE OF THEM having been in the news and/or in a court for random/assorted “unpleasant” behaviors (privacy abuses, eavesdropping, monopoly abuse, sexual or professional harassment, you just name it…), and EVERY SINGLE ONE OF THEM growing user bases that far exceeds the total population of multiple countries combined.
\n
\n\n\nThe HamBSD project aims to bring amateur packet radio to OpenBSD, including support for TCP/IP over AX.25 and APRS tracking/digipeating in the base system.
\n\nHamBSD will not provide a full AX.25 stack but instead only implement support for UI frames. There will be a focus on simplicity, security and readable code.
\n\nThe amateur radio community needs a reliable platform for packet radio for use in both leisure and emergency scenarios. It should be expected that the system is stable and resilient (but as yet it is neither).
\n
\n\n\nHAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rather than any manual repair process.
\n
\n\n\nAdd initial fsck support for HAMMER2, although CoW fs doesn't require fsck as a concept. Currently no repairing (no write), just verifying.
\n\nKeep this as a separate command for now.
\n\n
\nhttps://i.redd.it/vkdss0mtdpo31.jpg
\n
\n\n\nAdd modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.
\n\nThis makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).
\n
NetBSD LLVM sanitizers and GDB regression test suite, Ada—The Language of Cost Savings, Homura - a Windows Games Launcher for FreeBSD, FreeBSD core team appoints a WG to explore transition to Git, OpenBSD 6.6 Beta tagged, Project Trident 12-U5 update now available, and more.
\n\n\n\n\nAs NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.
\n\nI have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.
\n
\n\n\nI have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).
\n
\n\n\nInspired by lutris (a Linux gaming platform), we would like to provide a game launcher to play windows games on FreeBSD.
\n
\n\n\nMany myths surround the Ada programming language, but it continues to be used and evolve at the same time. And while the increased adoption of Ada and SPARK, its provable subset, is slow, it’s noticeable. Ada already addresses more of the features found in found in heavily used embedded languages like C+ and C#. It also tackles problems addressed by upcoming languages like Rust.
\n\nChris concludes, “Development technologies have a profound impact on one of the largest and most variable costs associated with embedded-system engineering—labor. At a time when on-time system deployment can not only impact customer satisfaction, but access to services revenue streams, engineering team efficiency is at a premium. Our research showed that programming language choices can have significant influence in this area, leading to shorter projects, better schedules and, ultimately, lower development costs. While a variety of factors can influence and dictate language choice, our research showed that Ada’s evolution has made it an increasingly compelling option for engineering organizations, providing both technically and financially sound solution.”
\n\nIn general, Ada already makes embedded “programming in the large” much easier by handling issues that aren’t even addressed in other languages. Though these features are often provided by third-party software, it results in inconsistent practices among developers. Ada also supports the gamut of embedded platforms from systems like Arm’s Cortex-M through supercomputers. Learning Ada isn’t as hard as one might think and the benefits can be significant.
\n
\n\n\nCore approved source commit bits for Doug Moore (dougm), Chuck Silvers (chs), Brandon Bergren (bdragon), and a vendor commit bit for Scott Phillips (scottph).
\n\nThe annual developer survey closed on 2019-04-02. Of the 397 developers, 243 took the survey with an average completion time of 12 minutes. The public survey closed on 2019-05-13. It was taken by 3637 users and had a 79% completion rate. A presentation of the survey results took place at BSDCan 2019.
\n\nThe core team voted to appoint a working group to explore transitioning our source code 'source of truth' from Subversion to Git. Core asked Ed Maste to chair the group as Ed has been researching this topic for some time. For example, Ed gave a MeetBSD 2018 talk on the topic.
\n\nThere is a variety of viewpoints within core regarding where and how to host a Git repository, however core feels that Git is the prudent path forward.
\n
CVSROOT: /cvs\nModule name: src\nChanges by: deraadt@cvs.openbsd.org 2019/08/09 21:56:02\n\nModified files:\n etc/root : root.mail\n share/mk : sys.mk\n sys/arch/macppc/stand/tbxidata: bsd.tbxi\n sys/conf : newvers.sh\n sys/sys : param.h\n usr.bin/signify: signify.1\n\nLog message:\nmove to 6.6-beta\n
\n\n\n\nImproved hardware support, including:
\n\n\n\n\nThis is the fifth general package update to the STABLE release repository based upon TrueOS 12-Stable.
\n
Package Summary
\n\nNew Packages (20)
\n\nDeleted Packages (24)
\n\n[CHVT feedback]
\nDJ - Feedback
\nBen - chvt
\nHarri - Marc's chvt question
vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more.
\n\nAllan and Benedict attended vBSDcon 2019, which ended last week.
\n\nIt was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks.
\n\nThe first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails.
\n\nJohn Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract and the recent commit we covered in episode 313.
\n\nMeanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs.
\n\nDavid Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD.
\n\nShawn Webb followed with his overview talk about the “State of the Hardened Union”.
\n\nBenedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality.
\n\nEntertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale.
\n\nPeople formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts.
\n\nColin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”.
\n\nAllan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads.
\n\n“By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms.
\n\nConor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”.
\n\nTwo OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”.
\n\nA dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night.
\n\nWe want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees!
\n\n\n\n\nThe InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.).
\n\nI’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop.
\n\nThe dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta.
\n
\n\n\nMaybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey.
\n\nIt was a bright, cold Monday, the last day of March 1969, and the computer sciences department was hosting distinguished guests: Bill Baker, a Bell Labs vice president, and Ed David, the director of research. Baker was about to pull the plug on Multics (a condensed form of MULTiplexed Information and Computing Service), a software project that the computer sciences department had been working on for four years. Multics was two years overdue, way over budget, and functional only in the loosest possible understanding of the term.
\n\nTrying to put the best spin possible on what was clearly an abject failure, Baker gave a speech in which he claimed that Bell Labs had accomplished everything it was trying to accomplish in Multics and that they no longer needed to work on the project. As Berk Tague, a staffer present at the meeting, later told Princeton University, “Like Vietnam, he declared victory and got out of Multics.”
\n\nWithin the department, this announcement was hardly unexpected. The programmers were acutely aware of the various issues with both the scope of the project and the computer they had been asked to build it for.
\n\nStill, it was something to work on, and as long as Bell Labs was working on Multics, they would also have a $7 million mainframe computer to play around with in their spare time. Dennis Ritchie, one of the programmers working on Multics, later said they all felt some stake in the success of the project, even though they knew the odds of that success were exceedingly remote.
\n\nCancellation of Multics meant the end of the only project that the programmers in the Computer science department had to work on—and it also meant the loss of the only computer in the Computer science department. After the GE 645 mainframe was taken apart and hauled off, the computer science department’s resources were reduced to little more than office supplies and a few terminals.
\n
\n\n\nIn the early '60s, Bill Ninke, a researcher in acoustics, had demonstrated a rudimentary graphical user interface with a DEC PDP-7 minicomputer. Acoustics still had that computer, but they weren’t using it and had stuck it somewhere out of the way up on the sixth floor.
\n\nAnd so Thompson, an indefatigable explorer of the labs’ nooks and crannies, finally found that PDP-7 shortly after Davis and Baker cancelled Multics.
\n\nWith the rest of the team’s help, Thompson bundled up the various pieces of the PDP-7—a machine about the size of a refrigerator, not counting the terminal—moved it into a closet assigned to the acoustics department, and got it up and running. One way or another, they convinced acoustics to provide space for the computer and also to pay for the not infrequent repairs to it out of that department’s budget.
\n\nMcIlroy’s programmers suddenly had a computer, kind of. So during the summer of 1969, Thompson, Ritchie, and Canaday hashed out the basics of a file manager that would run on the PDP-7. This was no simple task. Batch computing—running programs one after the other—rarely required that a computer be able to permanently store information, and many mainframes did not have any permanent storage device (whether a tape or a hard disk) attached to them. But the time-sharing environment that these programmers had fallen in love with required attached storage. And with multiple users connected to the same computer at the same time, the file manager had to be written well enough to keep one user’s files from being written over another user’s. When a file was read, the output from that file had to be sent to the user that was opening it.
\n\nIt was a challenge that McIlroy’s team was willing to accept. They had seen the future of computing and wanted to explore it. They knew that Multics was a dead-end, but they had discovered the possibilities opened up by shared development, shared access, and real-time computing. Twenty years later, Ritchie characterized it for Princeton as such: “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form.”
\n\nEventually when they had the file management system more or less fleshed out conceptually, it came time to actually write the code. The trio—all of whom had terrible handwriting—decided to use the Labs’ dictating service. One of them called up a lab extension and dictated the entire code base into a tape recorder. And thus, some unidentified clerical worker or workers soon had the unenviable task of trying to convert that into a typewritten document.
\n\nOf course, it was done imperfectly. Among various errors, “inode” came back as “eye node,” but the output was still viewed as a decided improvement over their assorted scribbles.
\n\nIn August 1969, Thompson’s wife and son went on a three-week vacation to see her family out in Berkeley, and Thompson decided to spend that time writing an assembler, a file editor, and a kernel to manage the PDP-7 processor. This would turn the group’s file manager into a full-fledged operating system. He generously allocated himself one week for each task.
\n\nThompson finished his tasks more or less on schedule. And by September, the computer science department at Bell Labs had an operating system running on a PDP-7—and it wasn’t Multics.
\n\nBy the summer of 1970, the team had attached a tape drive to the PDP-7, and their blossoming OS also had a growing selection of tools for programmers (several of which persist down to this day). But despite the successes, Thompson, Canaday, and Ritchie were still being rebuffed by labs management in their efforts to get a brand-new computer.
\n\nIt wasn’t until late 1971 that the computer science department got a truly modern computer. The Unix team had developed several tools designed to automatically format text files for printing over the past year or so. They had done so to simplify the production of documentation for their pet project, but their tools had escaped and were being used by several researchers elsewhere on the top floor. At the same time, the legal department was prepared to spend a fortune on a mainframe program called “AstroText.” Catching wind of this, the Unix crew realized that they could, with only a little effort, upgrade the tools they had written for their own use into something that the legal department could use to prepare patent applications.
\n\nThe computer science department pitched lab management on the purchase of a DEC PDP-11 for document production purposes, and Max Mathews offered to pay for the machine out of the acoustics department budget. Finally, management gave in and purchased a computer for the Unix team to play with. Eventually, word leaked out about this operating system, and businesses and institutions with PDP-11s began contacting Bell Labs about their new operating system. The Labs made it available for free—requesting only the cost of postage and media from anyone who wanted a copy.
\n\nThe rest has quite literally made tech history.
\n
\n\n\nA network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump.
\n\nSo, first, let’s talk a little bit about history. The first implementation of the network dumps was implemented around 2000 for the FreeBSD 4.x as a kernel module. The code was implemented in 2010 with the intention of being part of FreeBSD 9.0. However, the code never landed in FreeBSD. Finally, in 2018 with the commit r333283 by Mark Johnston the netdump client code landed in the FreeBSD. Subsequently, many other commitments were then implemented to add support for the different drivers (for example r333289). The first official release of FreeBSD, which support netdump is FreeBSD 12.0.
\n\nNow, let’s get back to the main topic. How to configure the network dump? Two machines are needed. One machine is to collect core dump, let’s call it server. We will use the second one to send us the core dump - the client.
\n
Unix virtual memory when you have no swap space, Dsynth details on Dragonfly, Instant Workstation on FreeBSD, new servers new tech, Experimenting with streaming setups on NetBSD, NetBSD’s progress towards Steam support thanks to GSoC, and more.
\n\n\n\n\nRecently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine):
\n\nOnce you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...]
\n\nI'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM.
\n\nTo simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program.
\n
\n\n\nFirst, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load.
\n\nMatthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes.
\n
\n\n\nDSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's).
\n\nThe original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied.
\n\n\n
\n- \n
The intent is to make dsynth compatible with synth's configuration files and directory structure.
- \n
This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts).
\n\n\n\n\nSome considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly.
\n\nSo – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager.
\n\nThe tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think.
\n\nIn any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around.
\n
\n\n\n\n\nFollowing up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure.
\n
\n\n\nWe have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting) and two 3900X based systems (thor and loki) which all together burn only around half the wattage that monster burned (500W vs 1000W) and 3 times the performance. That's at least a 6:1 improvement in performance efficiency.
\n\nWith SSD prices down significantly the new machines have all-SSDs. These new machines allow us to build dports binary packages for release, master, and staged at the same time and reduces the full-on bulk build times for getting all three done down from 2 weeks to 2 days. It will allow us to more promptly synchronize updates to ports with dports and get binary packages up sooner.
\n\nMonster, our venerable 48-core quad-socket opteron is being retired. This was a wonderful dev machine for working on DragonFly's SMP algorithms over the last 6+ years precisely because its inter-core and inter-socket latencies were quite high. If a SMP algorithm wasn't spot-on, you could feel it. Over the years DragonFly's performance on monster in doing things like bulk builds increased radically as the SMP algorithms got better and the cores became more and more localized. This kept monster relevant far longer than I thought it would be.
\n\nBut we are at a point now where improvements in efficiency are just too good to ignore. Monster's quad-socket opteron (4 x 12 core 6168's) pulls 1000W under full load while a single Ryzen 3900X (12 core / 24 thread) in a server configuration pulls only 150W, and is slightly faster on the same workload to boot.
\n\nI would like to thank everyone's generous donations over the last few years! We burned a few thousand on the new machines (as well as the major SSD upgrades we did to the blades) and made very good use of the money, particularly this year as prices for all major components (RAM, SSDs, CPUs, Mobos, etc) have dropped significantly.
\n
\n\n\nEver since OBS was successfully ported to NetBSD, I’ve been trying it out, seeing what works and what doesn’t. I’ve only just gotten started, and there’ll definitely be a lot of tweaking going forward.
\n\nCapturing a specific application’s windows seems to work okay. Capturing an entire display works, too. I actually haven’t tried streaming to Twitch or YouTube yet, but in a previous experiment a few weeks ago, I was able to run a FFmpeg command line and that could stream to Twitch mostly OK.
\n\nMy laptop combined with my external monitor allows me to have a dual-monitor setup wherein the smaller laptop screen can be my “broadcasting station” while the bigger screen is where all the action takes place. I can make OBS visible on all Xfce workspaces, but keep it tucked away on that display only. Altogether, the setup should let me use the big screen for the fun stuff but I can still monitor everything in the small screen.
\n
\n\n\nUltimately the goal is to get Valve's Steam client running on NetBSD using their Linux compatibility layer while the focus the past few months with Google Summer of Code 2019 were supporting the necessary DRM ioctls for allowing Linux software running on NetBSD to be able to tap accelerated graphics support.
\n\nStudent developer Surya P spent the summer working on compat_netbsd32 DRM interfaces to allow Direct Rendering Manager using applications running under their Linux compatibility layer.
\n\nThese interfaces have been tested and working as well as updating the "suse131" packages in NetBSD to make use of those interfaces. So the necessary interfaces are now in place for Linux software running on NetBSD to be able to use accelerated graphics though Steam itself isn't yet running on NetBSD with this layer.
\n\nThose curious about this DRM ioctl GSoC project can learn more from the NetBSD blog. NetBSD has also been seeing work this summer on Wayland support and better Wine support to ultimately make this BSD a better desktop operating system and potentially a comparable gaming platform to Linux.
\n
OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more.
\n\n\n\n\nAnother year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery.
\n
\nThe seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life.
\nGone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile.
\nOn my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck.
\n\n\nFreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12.
\n
\n\n\nFreeBSD can be installed as the only OS on your MacBook if desired. What you should have is:
\n
Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots.
\n\n\n\n\n\n\nOne of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later.
\n
\n\n\nThis is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl.
\n
\n\n\nThis is a general package update to the CURRENT release repository based upon TrueOS 19.08.
\n
\nLegacy boot ISO functional again
\nThis update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems.
PACKAGE CHANGES FROM 19.07-U1
\n\n\n\n\nThis is the third general package update to the STABLE release repository based upon TrueOS 12-Stable.
\n
The UNIX Philosophy in 2019, why use package managers, touchpad interrupted, Porting wine to amd64 on NetBSD second evaluation report, Enhancing Syzkaller Support for NetBSD, all about the Pinebook Pro, killing a process and all of its descendants, fast software the best software, and more.
\n\n\n\n\nToday, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973:
\n
\n\n\nValuable research is often hindered or outright prevented by the inability to install software. This need not be the case.
\n\nSince I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed.
\n\nIn many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few.
\n\nDeveloper websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this?
\n\nThe elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science.
\n\nFortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically.
\n
\n\n\nFor two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it.
\n\nIt's been a long journey and it's a technical tale, but here it is.
\n
\n\n\nPresently, Wine on amd64 is in test phase. It seems to work fine with caveats like LD_LIBRARY_PATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity.
\n
\n\n\nAs a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period.
\n\nYou can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancing_syzkaller_support_for_netbsd
\n
\n\n\n"So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available."
\n
\n\n\nKilling processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned:
\n\nUnix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number.
\n\nSending signals to all processes in a session is not trivial with syscalls.
\n\nChild processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children.
\n\nThe answer to the “What happens with orphaned process groups” question is not trivial.
\n
\n\n\nI love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness.
\n\nSoftware that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why.
\n\nBut why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools.
\n\nA typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.)
\n
NetBSD 9.0 release process has started, xargs, a tale of two spellcheckers, Adapting TriforceAFL for NetBSD, Exploiting a no-name freebsd kernel vulnerability, and more.
\n\n\n\n\nIf you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on:
\n\n\n
\n\n- New AArch64 architecture support:\n\n
\n\n
- Symmetric and asymmetrical multiprocessing support (aka big.LITTLE)
\n- Support for running 32-bit binaries
\n- UEFI and ACPI support
\n- Support for SBSA/SBBR (server-class) hardware.
\n- The FDT-ization of many ARM boards:\n\n
\n\n
- the 32-bit GENERIC kernel lists 129 different DTS configurations
\n- the 64-bit GENERIC64 kernel lists 74 different DTS configurations
\n- All supported by a single kernel, without requiring per-board configuration.
\n- Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices.
\n- ZFS has been updated to a modern version and seen many bugfixes.
\n- New hardware-accelerated virtualization via NVMM.
\n- NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default.
\n- NVMe performance improvements
\n- Optional kernel ASLR support, and partial kernel ASLR for the default configuration.
\n- Kernel sanitizers:\n\n
\n\n
- KLEAK, detecting memory leaks
\n- KASAN, detecting memory overruns
\n- KUBSAN, detecting undefined behaviour
\n- These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed.
\n- The removal of outdated networking components such as ISDN and all of its drivers
\n- The installer is now capable of performing GPT UEFI installations.
\n- Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding.
\n- Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU.
\nWe try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find.
\n\n\n
\n- Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/
\n
\n\n\nxargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too.
\n
\nI discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better.
\nxargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else.
\nIt literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem:
\nThis is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept.
\n\n\nThis is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study.
\n
\nThe reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole.
\n\n\nI have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation.
\n
\nFor work done during the first coding period, check out this post.
Benedict’s Gear:
\n\n\nGlocalMe G3 Mobile Travel HotSpot and Powerbank
\n
\nMogics Power Bagel
\nCharby Sense Power Cable
Allan’s Gear:
\n\n\nHuawei E5770s-320 4G LTE 150 Mbps Mobile WiFi Pro
\n
\nAOW Global Data SIM Card for On-Demand 4G LTE Mobile Data in Over 90 Countries
\nAll my devices charge from USB-C, so that is great
\nMore USB thumb drives than strictly necessary
\nMy Lenovo X270 laptop running FreeBSD 13-current
\nMy 2016 Macbook Pro (a prize from the raffle at vBSDCon 2017) that I use for email and video conferencing to preserve battery on my FreeBSD machine for work
OPNsense 19.7.1 is out, ZFS on Linux still has annoying issues with ARC size, Hammer2 is now default, NetBSD audio – an application perspective, new FreeNAS Mini, and more.
\n\n\n\n\nWe do not wish to keep you from enjoying your summer time, but this
\n\n
\nis a recommended security update enriched with reliability fixes for the
\nnew 19.7 series. Of special note are performance improvements as well
\nas a fix for a longstanding NAT before IPsec limitation.Full patch notes:
\n
\n\n\nStay safe and hydrated, Your OPNsense team
\n
One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there.
\n\n\nLinux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.)
\n\nThis is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case).
\n\nThe net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all).
\n
commit a49112761c919d42d405ec10252eb0553662c824\nAuthor: Matthew Dillon <dillon at apollo.backplane.com>\nDate: Mon Jun 10 17:53:46 2019 -0700\n\n installer - Default to HAMMER2\n\n * Change the installer default from HAMMER1 to HAMMER2.\n\n * Adjust the nrelease build to print the location of the image files\n when it finishes.\n\nSummary of changes:\n nrelease/Makefile | 2 +-\n usr.sbin/installer/dfuibe_installer/flow.c | 20 ++++++++++----------\n 2 files changed, 11 insertions(+), 11 deletions(-)\n\nhttp://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/a49112761c919d42d405ec10252eb0553662c824\n
\n\n\n\n\nNetBSD audio – an application perspective ... or, "doing it natively, because we can"
\n
audio options for NetBSD in pkgsrc
\n\nMany many abstraction layers available:
\n\nAdvantages of using NetBSD audio directly
\n\n[nia note: SDL2 seems very sensitive to the blk_ms sysctl being high or low, with other implementations there seems to be a less noticable difference. I don't know why.]
\n\n\nTwo new FreeNAS Mini systems join the very popular FreeNAS Mini and Mini XL:
\n\nFreeNAS Mini XL+: This powerful 10 Bay platform (8x 3.5” and 1x 2.5” hot-swap, 1x 2.5” internal) includes the latest, compact server technology and provides dual 10GbE ports, 8 CPU cores and 32 GB RAM for high performance workgroups. The Mini XL+ scales beyond 100TB and is ideal for very demanding applications, including hosting virtual machines and multimedia editing. Starting at $1499, the Mini XL+ configured with cache SSD and 80 TB capacity is $4299, and consumes about 100 Watts.
\n\nFreeNAS Mini E: This cost-effective 4 Bay platform provides the resources required for SOHO use with quad GbE ports and 8 GB of RAM. The Mini E is ideal for file sharing, streaming and transcoding video at 1080p. Starting at $749, the Mini E configured with 8 TB capacity is $999, and consumes about 36 Watts.
\n
DragonFlyBSD Project Update - colo upgrade, future trends, resuming ZFS send, realtime bandwidth terminal graph visualization, fixing telnet fixes, a chapter from the FBI’s history with OpenBSD and an OpenSSH vuln, and more.
\n\n\n\n\nFor the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone.
\n\nThe goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system.
\n\nCurrently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while.
\n\nWell, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get.
\n\nThe new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. > This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time.
\n\nFuture trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going.
\n
\n\n\nOne of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot?
\n\nFor a very long time, you didn't have any options - you had to send a snapshot from the beginning. Now, this limitation was already bad enough. However, another downside of this approach was that all the data which you already send was thrown away. Therefore, ZFS had to go over all this data and remove them from the dataset. Imagine the terabytes of data which you sent via the network was thrown away because as you were sending the last few bytes, the network went off.
\n\nIn this short post, I don't want to go over the whole ZFS snapshot infrastructure (if you think that such a post would be useful, please leave a comment). Now, to get back to the point, this infrastructure is used to clone the datasets. Some time ago a new feature called “Resuming ZFS send” was introduced. That means that if there was some problem with transmitting the dataset from one point to another you could resume it or throw them away. But the point is, that yes, you finally have a choice.
\n
\n\n\nIf for some reasons you want to visualize your bandwidth traffic on an interface (in or out) in a terminal with a nice graph, here is a small script to do so, involving ttyplot, a nice software making graphics in a terminal.
\n\nThe following will works on OpenBSD. You can install ttyplot by pkg_add ttyplot as root, ttyplot package appeared since OpenBSD 6.5.
\n
\n\n\nThere’s a FreeBSD commit to telnet. fix a couple of snprintf() buffer overflows. It’s received a bit of attention for various reasons, telnet in 2019?, etc. I thought I’d take a look. Here’s a few random observations.
\n\n\n
\n- \n
The first line is indented with spaces while the others use tabs.
- \n
The correct type for string length is size_t not unsigned int.
- \n
sizeof(char) is always one. There’s no need to multiply by it.
- \n
If you do need to multiply by a size, this is an unsafe pattern. Use calloc or something similar. (OpenBSD provides reallocarray to avoid zeroing cost of calloc.)
- \n
Return value of malloc doesn’t need to be cast. In fact, should not be, lest you disguise a warning.
- \n
Return value of malloc is not checked for NULL.
- \n
No reason to cast cp to char * when passing to snprintf. It already is that type. And if it weren’t, what are you doing?
- \n
The whole operation could be simplified by using asprintf.
- \n
Although unlikely (probably impossible here, but more generally), adding the two source lengths together can overflow, resulting in truncation with an unchecked snprintf call. asprintf avoids this failure case.
\n\n\n\n\nEarlier this year I FOIAed the FBI for details on allegations of backdoor installed in the IPSEC stack in 2010, originally discussed by OpenBSD devs (https://marc.info/?l=openbsd-tech&m=129236621626462 …) Today, I got an interesting but unexpected responsive record:
\n
Replacing a (silently) failing disk in a ZFS pool, OPNsense 19.7 RC1 released, implementing DRM ioctl support for NetBSD, High quality/low latency VOIP server with umurmur/Mumble on OpenBSD, the PDP-7 where Unix began, LLDB watchpoints, and more.
\n\n\n\n\nMaybe I can’t read, but I have the feeling that official documentations explain every single corner case for a given tool, except the one you will actually need. My today’s struggle: replacing a disk within a FreeBSD ZFS pool.
\n
\nWhat? there’s a shitton of docs on this topic! Are you stupid?
\nI don’t know, maybe. Yet none covered the process in a simple, straight and complete manner.
\n\n\nHi there,
\n
\nFor four and a half years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
\nWe thank all of you for helping test, shape and contribute to the project! We know it would not be the same without you.
\nDownload links, an installation guide[1] and the checksums for the images can be found below as well.
\n\n\nIoctls are input/output control system calls and DRM stands for direct rendering manager The DRM layer provides several services to graphics drivers, many of them driven by the application interfaces it provides through libdrm, the library that wraps most of the DRM ioctls. These include vblank event handling, memory management, output management, framebuffer management, command submission & fencing, suspend/resume support, and DMA services.
\n
\n\n\nNetBSD was able to make native DRM ioctl calls with hardware rendering once xorg and proper mesa packages where installed. We used the glxinfo and glxgears applications to test this out.
\n
\n\n\nDiscord users keep telling about their so called discord server, which is not dedicated to them at all. And Discord has a very bad quality and a lot of voice distorsion.
\n
\nWhy not run your very own mumble server with high voice quality and low latency and privacy respect? This is very easy to setup on OpenBSD!
\nMumble is an open source voip client, it has a client named Mumble (available on various operating system) and at least Android, the server part is murmur but there is a lightweight server named umurmur. People authentication is done through certificate generated locally and automatically accepted on a server, and the certificate get associated with a nickname. Nobody can pick the same nickname as another person if it’s not the same certificate.
\n\n\nFrom time to time, I like to review my knowledge in a certain area, even when I feel like I know a lot about it already. I go back to the basics and read tutorials, manuals, books or watch interesting videos.
\n
\nI’ve been using macOS for a couple of years now, previously being a linux user for some (relatively short) time. Both these operating systems have a common ancestor — Unix. While I’m definitely not an expert, I feel quite comfortable using linux & macOS — I understand the concepts behind the system architecture, know a lot of command line tools & navigate through the shell without a hassle. So-called unix philosophy is also close to my heart. I always feel like there’s more I could squeeze out of it.
\nRecently, I found that book titled “Unix for dummies, 5th edition” which was published back in… 2004. Feels literally like AGES in the computer-related world. However, it was a great shot — the book starts with the basics, providing some brief history of Unix and how it came to life. It talks a lot about the structure of the system and where certain pieces fit (eg. “standard” set of tools), and how to understand permissions and work with files & directories. There’s even a whole chapter about shell-based text editors like Vi and Emacs! Despite the fact that I am familiar with most of these, I could still find some interesting pieces & tools that I either knew existed (but never had a chance to use), or even haven’t ever heard of. And almost all of these are still valid in the modern “incarnations” of Unix’s descendants: Linux and macOS.
\nThe book also talks about networking, surfing the web & working with email. It’s cute to see pictures of those old browsers rendering “ancient” Internet websites, but hey — this is how it looked like no more than fifteen years ago!
\nI can really recommend this book to anyone working on modern macOS or Linux — you will certainly find some interesting pieces. Especially if you like to go back to the roots from time to time as I do!
\n\n\nIn preparation for a talk on Seventh Edition Unix this fall, I stumbled upon a service list from DEC for all known PDP-7 machines. From that list, and other sources, I believe that PDP-7 serial number 34 was the original Unix machine.
\n
\nV0 Unix could run on only one of the PDP-7s. Of the 99 PDP-7s produced, only two had disks. Serial number 14 had an RA01 listed, presumably a disk, though of a different type. In addition to the PDP-7 being obsolete in 1970, no other PDP-7 could run Unix, limiting its appeal outside of Bell Labs. By porting Unix to the PDP-11 in 1970, the group ensured Unix would live on into the future. The PDP-9 and PDP-15 were both upgrades of the PDP-7, so to be fair, PDP-7 Unix did have a natural upgrade path (the PDP-11 out sold the 18 bit systems though ~600,000 to ~1000). Ken Thompson reports in a private email that there were 2 PDP-9s and 1 PDP-15 at Bell Labs that could run a version of the PDP-7 Unix, though those machines were viewed as born obsolete.
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n
\nIn February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types and fix compat32 issues. You can read more about that in my May 2019 report.
\nIn June, I have finally finished the remaining ptrace() work for xstate and got it merged both on NetBSD and LLDB end (meaning it's going to make it into NetBSD 9). I have also worked on debug register support in LLDB, effectively fixing watchpoint support. Once again I had to fight some upstream regressions.
FreeBSD 11.3 has been released, OpenBSD workstation, write your own fuzzer for the NetBSD kernel, Exploiting FreeBSD-SA-19:02.fd, streaming to twitch using OpenBSD, 3 different ways of dumping hex contents of a file, and more.
\n\n\n\n\nThe FreeBSD Release Engineering Team is pleased to announce the availability of FreeBSD 11.3-RELEASE. This is the fourth release of the stable/11 branch.
\n
\n\n\nWhy OpenBSD? Simply because it is the best tool for the job for me for my new-to-me Lenovo Thinkpad T420. Additionally, I do care about security and non-bloat in my personal operating systems (business needs can have different priorities, to be clear).
\n\nI will try to detail what my reasons are for going with OpenBSD (instead of GNU/Linux, NetBSD, or FreeBSD of which I’m comfortable using without issue), challenges and frustrations I’ve encountered, and what my opinions are along the way.
\n\nDisclaimer: in this post, I’m speaking about what is my opinion, and I’m not trying to convince you to use OpenBSD or anything else. I don’t truly care, but wanted to share in case it could be useful to you. I do hope you give OpenBSD a shot as your workstation, especially if it has been a while.
\n
\n\n\nI’m not new to OpenBSD, to be clear. I’ve been using it off and on for over 20 years. The biggest time in my life was the early 2000s (I was even the Python port maintainer for a bit), where I not only used it for my workstation, but also for production servers and network devices.
\n\nI just haven’t used it as a workstation (outside of a virtual machine) in over 10 years, but have used it for servers. Workstation needs, especially for a primary workstation, are greatly different and the small things end up mattering most.
\n
\n\n\nThe easy way to describe fuzzing is to compare it to the process of unit testing a program, but with different input. This input can be random, or it can be generated in some way that makes it unexpected form standard execution perspective.
\n\nThe simplest 'fuzzer' can be written in few lines of bash, by getting N bytes from /dev/rand, and putting them to the program as a parameter.
\n
\n\n\nWhat can be done to make fuzzing more effective? If we think about fuzzing as a process, where we place data into the input of the program (which is a black box), and we can only interact via input, not much more can be done.
\n\nHowever, programs usually process different inputs at different speeds, which can give us some insight into the program's behavior. During fuzzing, we are trying to crash the program, thus we need additional probes to observe the program's behaviour.
\n\nAdditional knowledge about program state can be exploited as a feedback loop for generating new input vectors. Knowledge about the program itself and the structure of input data can also be considered. As an example, if the input data is in the form of HTML, changing characters inside the body will probably cause less problems for the parser than experimenting with headers and HTML tags.
\n\nFor open source programs, we can read the source code to know what input takes which execution path. Nonetheless, this might be very time consuming, and it would be much more helpful if this can be automated. As it turns out, this process can be improved by tracing coverage of the execution
\n
\n\n\nYou can submit your proposal at https://easychair.org/conferences/?conf=vbsdcon2019
\n\nThe talks will have a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.
\n\nIf you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal.
\n\nPossible topics include: How we manage a giant installation with respect to handling spam, snd/or sysadmin, and/or networking, Cool new stuff in BSD, Tell us about your project which runs on BSD.
\n\nBoth users and developers are encouraged to share their experiences.
\n
\n\n\nIn February 2019 the FreeBSD project issued an advisory about a possible vulnerability in the handling of file descriptors. UNIX-like systems such as FreeBSD allow to send file descriptors to other processes via UNIX-domain sockets. This can for example be used to pass file access privileges to the receiving process.
\n\nInside the kernel, file descriptors are used to indirectly reference a C struct which stores the relevant information about the file object. This could for instance include a reference to a vnode which describes the file for the file system, the file type, or the access privileges.
\n\nWhat really happens if a UNIX-domain socket is used to send a file descriptor to another process is that for the receiving process, inside the kernel a reference to this struct is created. As the new file descriptor is a reference to the same file object, all information is inherited. For instance, this can allow to give another process write access to a file on the drive even if the process owner is normally not able to open the file writable.
\n\nThe advisory describes that FreeBSD 12.0 introduced a bug in this mechanism. As the file descriptor information is sent via a socket, the sender and the receiver have to allocate buffers for the procedure. If the receiving buffer is not large enough, the FreeBSD kernel attempts to close the received file descriptors to prevent a leak of these to the sender. However, while the responsible function closes the file descriptor, it fails to release the reference from the file descriptor to the file object. This could cause the reference counter to wrap.
\n\nThe advisory further states that the impact of this bug is possibly a local privilege escalation to gain root privileges or a jail escape. However, no proof-of-concept was provided by the advisory authors.
\n
\n\n\nThe privilege escalation is now a piece of cake thanks to a technique used by kingcope, who published a FreeBSD root exploit in 2005, which writes to the file /etc/libmap.conf. This configuration file can be used to hook the loading of dynamic libraries if a program is started. The exploit therefore creates a dynamic library, which copies /bin/sh to another file and sets the suid-bit for the copy. The hooked library is libutil, which is for instance called by su. Therefore, a call to su by the user will afterwards result in a suid copy of /bin/sh.
\n
\n\n\nIf you ever wanted to make a twitch stream from your OpenBSD system, this is now possible, thanks to OpenBSD developer thfr@ who made a wrapper named fauxstream using ffmpeg with relevant parameters.
\n\nThe setup is quite easy, it only requires a few steps and searching on Twitch website two informations, hopefully, to ease the process, I found the links for you.
\n\nYou will need to make an account on twitch, get your api key (a long string of characters) which should stay secret because it allow anyone having it to stream on your account.
\n
Am5x86 based retro UNIX build log, setting up services in a FreeNAS Jail, first taste of DragonflyBSD, streaming Netflix on NetBSD, NetBSD on the last G4 Mac mini, Hammer vs Hammer2, and more.
\n\n\n\n\nI have recently acquired an Am5x86 computer, in a surprisingly good condition. This is an ongoing project, check this page often for updates!
\n\nI began by connecting a front panel. The panel came from a different chassis and is slightly too wide, so I had to attach it with a couple of zip-ties. However, that makes it stick out from the PC front at an angle, allowing easy access when the computer sits at the floor - and thats where it is most of the time. It's not that bad, to be honest, and its way easier to access than it would be, if mounted vertically
\n\nThere is a mains switch on the front panel because the computer uses an older style power supply. Those power supplies instead of relying on a PSON signal, like modern ATX supplies, run a 4 wire cable to a mains switch. The cable carries live and neutral both ways, and the switch keys in or out the power. The system powers on as soon as the switch is enabled.
\n\nOriginally there was no graphics card in it. Since a PC will not boot with out a GPU, I had to find one. The mainboard only has PCI and ISA slots, and all the GPUs I had were AGP. Fortunately, I bought a PCI GPU hoping it would solve my issue...
\n\nHowever the GPU turned out to be faulty. It took me some time to repair it. I had to repair a broken trace leading to one of the EEPROM pins, and replace a contact in the EEPROM's socket. Then I replaced all the electrolytic capacitors on it, and that fixed it for good.
\n\nHaving used up only one of the three PCI slots, I populated the remaining pair with two ethernet cards. I still have a bunch of ISA slots available, but I have nothing to install there. Yet.
\n
\n\n\nThis piece demonstrates the setup of a server service in a FreeNAS jail and how to share files with a jail using Apache 2.4 as an example. Jails are powerful, self-contained FreeBSD environments with separate network settings, package management, and access to thousands of FreeBSD application packages. Popular packages such as Apache, NGINX, LigHTTPD, MySQL, and PHP can be found and installed with the pkg search and pkg install commands.
\n\nThis example shows creating a jail, installing an Apache web server, and setting up a simple web page.
\n\nNOTE: Do not directly attach FreeNAS to an external network (WAN). Use port forwarding, proper firewalls and DDoS protections when using FreeNAS for external web sites. This example demonstrates expanding the functionality of FreeNAS in an isolated LAN environment.
\n
\n\n\nLast week, I needed to pick a BSD Operating System which supports NUMA to do some testing, so I decided to give Dragonfly BSD a shot. Dragonfly BSDonly can run on X86_64 architecture, which reminds me of Arch Linux, and after some tweaking, I feel Dragonfly BSD may be a “developer-friendly” Operating System, at least for me.
\n\nI mainly use Dragonfly BSD as a server, so I don’t care whether GUI is fancy or not. But I have high requirements of developer tools, i.e., compiler and debugger. The default compiler of Dragonfly BSD is gcc 8.3, and I can also install clang 8.0.0 from package. This means I can test state-of-the-art features of compilers, and it is really important for me. gdb‘s version is 7.6.1, a little lag behind, but still OK.
\n\nFurthermore, the upgradation of Dragonfly BSD is pretty simple and straightforward. I followed document to upgrade my Operating System to 5.6.0 this morning, just copied and pasted, no single error, booted successfully.
\n
\n\n\nHere's a step-by-step guide that allows streaming Netflix media on NetBSD using a intel-haxm accelerated QEMU vm.
\n\nHeads-up! Sound doesn't work, but everything else is fine. Please read the rest of this thread for a solution to this!!
\n
\n\n\nI’m about halfway through the new edition of Sudo Mastery. Assuming nothing terrible happens, should have a complete first draft in four to six weeks. Enough stuff has changed in sudo that I need to carefully double-check every single feature. (I’m also horrified by the painfully obsolete versions of sudo shipped in the latest versions of CentOS and Debian, but people running those operating systems are already accustomed to their creaky obsolescence.)
\n\nBut the reason for this blog post? I have Eddie Sharam’s glorious cover art. My Patronizers saw it last month, so now the rest of you get a turn.
\n
\n\n\nI'm a big fan of NetBSD. I've run it since 2000 on a Mac IIci (of course it's still running it) and I ran it for several years on a Power Mac 7300 with a G3 card which was the second incarnation of the Floodgap gopher server. Today I also still run it on a MIPS-based Cobalt RaQ 2 and an HP Jornada 690. I think NetBSD is a better match for smaller or underpowered systems than current-day Linux, and is fairly easy to harden and keep secure even though none of these systems are exposed to the outside world.
\n\nRecently I had a need to set up a bridge system that would be fast enough to connect two networks and I happened to have two of the "secret" last-of-the-line 1.5GHz G4 Mac minis sitting on the shelf doing nothing. Yes, they're probably outclassed by later Raspberry Pi models, but I don't have to buy anything and I like putting old hardware to good use.
\n
\n\n\nWith the newly released DragonFlyBSD 5.6 there are improvements to its original HAMMER2 file-system to the extent that it's now selected by its installer as the default file-system choice for new installations. Curious how the performance now compares between HAMMER and HAMMER2, here are some initial benchmarks on an NVMe solid-state drive using DragonFlyBSD 5.6.0.
\n\nWith a 120GB Toshiba NVMe SSD on an Intel Core i7 8700K system, I ran some benchmarks of DragonFlyBSD 5.6.0 freshly installed with HAMMER2 and then again when returning to the original HAMMER file-system that remains available via its installer. No other changes were made to the setup during testing.
\n\nAnd then for the more synthetic workloads it was just a mix. But overall HAMMER2 was performing well during the initial testing and great to see it continuing to offer noticeable leads in real-world workloads compared to the aging HAMMER file-system. HAMMER2 also offers better clustering, online deduplication, snapshots, compression, encryption, and many other modern file-system features.
\n
Website protection with OPNsense, FreeBSD Support Pull Request for ZFS-on-Linux, How much has Unix changed, Porting Wine to amd64 on NetBSD, FreeBSD Enterprise 1 PB Storage, the death watch for X11 has started, and more.
\n\n\n\n\nThe OPNsense security platform can help you to protect your network and your webservers with the nginx plugin addition.
\n\n
\nIn old days, install an open source firewall was a very trick task, but today it can be done with few clicks (or key strokes). In this article I'll not describe the detailed OPNsense installation process, but you can watch this video that was extracted from my OPNsense course available in Udemy. The video is in portuguese language, but with the translation CC Youtube feature you may be able to follow it without problems (if you don't are a portuguese speaker ofcourse) :-)\n
\n- See the article for the rest of the writeup
\n
\n\n\nUNIX-like systems have dominated computing for decades, and with the rise of the internet and mobile devices their reach has become even larger. True, most systems now use more modern OSs like Linux, but how much has the UNIX-like landscape changed since the early days?
\n
\nSo, my question was this: how close is a modern *NIX userland to some of the earliest UNIX releases? To do this I'm going to compare a few key points of a modern Linux system with the earliest UNIX documentation I can get my hands on. The doc I am going to be covering(https://www.tuhs.org/Archive/Distributions/Research/Dennis_v1/UNIX_ProgrammersManual_Nov71.pdf) is from November 1971, predating v1 of the system.
\nI think the best place to start this comparison is to look at one of the highest-profile parts of the OS, that being the file system. Under the hood modern EXT file systems are completely different from the early UNIX file systems. However, they are still presented in basically the same way, as a heirerarchicat structure of directories with device files. So paths still look identical, and navigating the file system still functions the same. Often used commands likels
,cp
,mv
,du
, anddf
function the same. So aremount
andumount
. But, there are some key differences. For instance,cd
didn't exist, yet insteadchdir
filled its place. Also,chmod
is somewhat different. Instead of the usual 3-digit octal codes for permissions, this older version only uses 2 digits. Really, that difference is due to the underlying file system using a different permission set than modern systems. For the most part, all the file handling is actually pretty close to a Linux system from 2019.
\n\n\nI have been working on porting Wine to amd64 on NetBSD as a GSoC 2019 project. Wine is a compatibility layer which allows running Microsoft Windows applications on POSIX-complaint operating systems. This report provides an overview of the progress of the project during the first coding period.
\n\n
\nInitially, when I started working on getting Wine-4.4 to build and run on NetBSD i386 the primary issue that I faced was Wine displaying black windows instead of UI, and this applied to any graphical program I tried running with Wine.
\nI suspected it , as it is related to graphics, to be an issue with the graphics driver or Xorg. Subsequently, I tried building modular Xorg, and I tried running Wine on it only to realize that Xorg being modular didn't affect it in the least. After having tried a couple of configurations, I realized that trying to hazard out every other probability is going to take an awful lot of time that I didn't have. This motivated me to bisect the repo using git, and find the first version of Wine which failed on NetBSD.\n
\n- See the article for the rest of the writeup
\n
\n\n\nToday FreeBSD operating system turns 26 years old. 19 June is an International FreeBSD Day. This is why I got something special today :). How about using FreeBSD as an Enterprise Storage solution on real hardware? This where FreeBSD shines with all its storage features ZFS included.
\n
\nToday I will show you how I have built so called Enterprise Storage based on FreeBSD system along with more then 1 PB (Petabyte) of raw capacity.
\nThis project is different. How much storage space can you squeeze from a single 4U system? It turns out a lot! Definitely more then 1 PB (1024 TB) of raw storage space.
\n\n\nOnce we are done with this we expect X.org to go into hard maintenance mode fairly quickly. The reality is that X.org is basically maintained by us and thus once we stop paying attention to it there is unlikely to be any major new releases coming out and there might even be some bitrot setting in over time. We will keep an eye on it as we will want to ensure X.org stays supportable until the end of the RHEL8 lifecycle at a minimum, but let this be a friendly notice for everyone who rely the work we do maintaining the Linux graphics stack, get onto Wayland, that is where the future is.
\n
\nI have no idea how true this is about X.org X server maintenance, either now or in the future, but I definitely think it's a sign that developers have started saying this. If Gnome developers feel that X.org is going to be in hard maintenance mode almost immediately, they're probably pretty likely to also put the Gnome code that deals with X into hard maintenance mode. And public Gnome statements about this (and public action or lack of it) provide implicit support for KDE and any other desktop to move in this direction if they want to (and probably create some pressure to do so). I've known that Wayland was the future for some time, but I would still like it to not arrive any time soon.
DragonflyBSD 5.6 is out, OpenBSD Vulkan Support, bad utmp implementations in glibc and FreeBSD, OpenSSH protects itself against Side Channel attacks, ZFS vs OpenZFS, and more.
\n\nBig-ticket items
Improved VM
\n\nDRM
\n\nHAMMER2
\n\n\n\n\nSomewhat surprisingly, OpenBSD has added the Vulkan library and ICD loader support as their newest port.
\n\n
\nThis new graphics/vulkan-loader port provides the generic Vulkan library and ICD support that is the common code for Vulkan implementations on the system. This doesn't enable any Vulkan hardware drivers or provide something new not available elsewhere, but is rare seeing Vulkan work among the BSDs. There is also in ports the related components like the SPIR-V headers and tools, glsllang, and the Vulkan tools and validation layers.
\nThis is of limited usefulness, at least for the time being considering OpenBSD like the other BSDs lag behind in their DRM kernel driver support that is ported over from the mainline Linux kernel tree but generally years behind the kernel upstream. Particularly with Vulkan, newer kernel releases are needed for some Vulkan features as well as achieving decent performance. The Vulkan drivers of relevance are the open-source Intel ANV Vulkan driver and Radeon RADV drivers, both of which are in Mesa though we haven't seen any testing results to know how well they would work if at all currently on OpenBSD, but they're at least in Mesa and obviously open-source.\n
\n- A note: The BSDs are no longer that far behind.
\n- FreeBSD 12.0 uses DRM from Linux 4.16 (April 2018), and the drm-devel port is based on Linux 5.0 (March 2019)
\n- OpenBSD -current as of April 2019 uses DRM from Linux 4.19.34\n***
\n
\n\n\nI recently released another version – 0.5.0 – of Dinit, the service manager / init system. There were a number of minor improvements, including to the build system (just running “make” or “gmake” should be enough on any of the systems which have a pre-defined configuration, no need to edit mconfig by hand), but the main features of the release were S6-compatible readiness notification, and support for updating the utmp database.
\n\n
\nIn other words, utmp is a record of who is currently logged in to the system (another file, “wtmp”, records all logins and logouts, as well as, potentially, certain system events such as reboots and time updates). This is a hint at the main motivation for having utmp support in Dinit – I wanted the “who” command to correctly report current logins (and I wanted boot time to be correctly recorded in the wtmp file).
\nI wondered: If the files consist of fixed-sized records, and are readable by regular users, how is consistency maintained? That is – how can a process ensure that, when it updates the database, it doesn’t conflict with another process also attempting to update the database at the same time? Similarly, how can a process reading an entry from the database be sure that it receives a consistent, full record and not a record which has been partially updated? (after all, POSIX allows that a write(2) call can return without having written all the requested bytes, and I’m not aware of Linux or any of the *BSDs documenting that this cannot happen for regular files). Clearly, some kind of locking is needed; a process that wants to write to or read from the database locks it first, performs its operation, and then unlocks the database. Once again, this happens under the hood, in the implementation of the getutent/pututline functions or their equivalents.
\nThen I wondered: if a user process is able to lock the utmp file, and this prevents updates, what’s to stop a user process from manually acquiring and then holding such a lock for a long – even practically infinite – duration? This would prevent the database from being updated, and would perhaps even prevent logins/logouts from completing. Unfortunately, the answer is – nothing; and yes, it is possible on different systems to prevent the database from being correctly updated or even to prevent all other users – including root – from logging in to the system.\n
\n- A good find
\n- On FreeBSD, even though write(2) can be asynchronous, once the write syscall returns, the data is in the buffer cache (or ARC), and any future read(2) will see that new data even if it has not yet been written to disk.\n***
\n
\n\n\nLast week, Damien Miller, a Google security researcher, and one of the popular OpenSSH and OpenBSD developers announced an update to the existing OpenSSH code that can help protect against the side-channel attacks that leak sensitive data from computer’s memory. This protection, Miller says, will protect the private keys residing in the RAM against Spectre, Meltdown, Rowhammer, and the latest RAMBleed attack.
\n
\nSSH private keys can be used by malicious threat actors to connect to remote servers without the need of a password. According to CSO, “The approach used by OpenSSH could be copied by other software projects to protect their own keys and secrets in memory”.
\nHowever, if the attacker is successful in extracting the data from a computer or server’s RAM, they will only obtain an encrypted version of an SSH private key, rather than the cleartext version.
\nIn an email to OpenBSD, Miller writes, “this change encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large ‘prekey’ consisting of random data (currently 16KB).”
\n\n\nYou’ve probably heard us say a mix of “ZFS” and “OpenZFS” and an explanation is long-overdue.
\n\n
\nFrom its inception, “ZFS” has referred to the “Zettabyte File System” developed at Sun Microsystems and published under the CDDL Open Source license in 2005 as part of the OpenSolaris operating system. ZFS was revolutionary for completely decoupling the file system from specialized storage hardware and even a specific computer platform. The portable nature and advanced features of ZFS led FreeBSD, Linux, and even Apple developers to start porting ZFS to their operating systems and by 2008, FreeBSD shipped with ZFS in the 7.0 release. For the first time, ZFS empowered users of any budget with enterprise-class scalability and data integrity and management features like checksumming, compression and snapshotting, and those features remain unrivaled at any price to this day. On any ZFS platform, administrators use the zpool and zfs utilities to configure and manage their storage devices and file systems respectively. Both commands employ a user-friendly syntax such as‘zfs create mypool/mydataset’ and I welcome you to watch the appropriately-titled webinar “Why we love ZFS & you should too” or try a completely-graphical ZFS experience with FreeNAS.
\nOracle has steadily continued to develop its own proprietary branch of ZFS and Matt Ahrens points out that over 50% of the original OpenSolaris ZFS code has been replaced in OpenZFS with community contributions. This means that there are, sadly, two politically and technologically-incompatible branches of “ZFS” but fortunately, OpenZFS is orders of magnitude more popular thanks to its open nature. The two projects should be referred to as “Oracle ZFS” and “OpenZFS” to distinguish them as development efforts, but the user still types the ‘zfs’ command, which on FreeBSD relies on the ‘zfs.ko’ kernel module. My impression is that the terms of the CDDL license under which the OpenZFS branch of ZFS is published protects its users from any patent and trademark risks. Hopefully, this all helps you distinguish the OpenZFS project from the ZFS technology.\n
\n- There was further discussion of how the ZFSOnLinux repo will become the OpenZFS repo in the future once it also contains the bits to build on FreeBSD as well during the June 25th ZFS Leadership Meeting. The videos for all of the meetings are available here\n***
\n
OpenZFS-kmod port available, using blacklistd with NPF as fail2ban replacement, ZFS raidz expansion alpha preview 1, audio VU-meter increases CO2 footprint rant, XSAVE and compat32 kernel work for LLDB, where icons for modern X applications come from, and more.
\n\n\n\n\nblacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy.
\n
\nThe interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf
\nNow, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use.
\nUnfortunately (dont' ask me why :P) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen
\n\n\nA couple months ago I noticed that the monitor on my workstation never power off anymore. Screensaver would go on, but DPMs (to do the poweroff) never kicked in.
\n\n
\nI grovels the output of various tools that display DPMS settings, which as usual in Xorg were useless. Everybody said DPMS is on with a timeout. I even wrote my own C program to use every available Xlib API call and even the xscreensaver library calls. (should make it available) No go, everybody says that DPMs is on, enabled and set on a timeout. Didn’t matter whether I let xscreeensaver do the job or just the X11 server.
\nAfter a while I noticed that DPMS actually worked between starting my X11 server and starting all my clients. I have a minimal .xinitrc and start the actual session from a script, that is how I could notice. If I used a regular desktop login I wouldn’t have noticed. A server state bug was much more likely than a client bug.\n
\n- See the article for the rest...
\n
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
\n
\nIn February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types. You can read more about that in my Apr 2019 report.
\nIn May, I was primarily continuing the work on new ptrace interface. Besides that, I've found and fixed a bug in ptrace() compat32 code, pushed LLVM buildbot to ‘green’ status and found some upstream LLVM regressions. More below.
\n\n\nIf you have a traditional window manager like fvwm, one of the things it can do is iconify X windows so that they turn into icons on the root window (which would often be called the 'desktop'). Even modern desktop environments that don't iconify programs to the root window (or their desktop) may have per-program icons for running programs in their dock or taskbar. If your window manager or desktop environment can do this, you might reasonably wonder where those icons come from by default.
\n
\nAlthough I don't know how it was done in the early days of X, the modern standard for this is part of the Extended Window Manager Hints. In EWMH, applications give the window manager a number of possible icons, generally in different sizes, as ARGB bitmaps (instead of, say, SVG format). The window manager or desktop environment can then pick whichever icon size it likes best, taking into account things like the display resolution and so on, and display it however it wants to (in its original size or scaled up or down).
\nHow this is communicated in specific is through the only good interprocess communication method that X supplies, namely X properties. In the specific case of icons, the _NET_WM_ICON property is what is used, and xprop can display the size information and an ASCII art summary of what each icon looks like. It's also possible to use some additional magic to read out the raw data from _NET_WM_ICON in a useful format; see, for example, this Stackoverflow question and its answers.
DragonFlyBSD's kernel optimizations pay off, differences between OpenBSD and Linux, NetBSD 2019 Google Summer of Code project list, Reducing that contention, fnaify 1.3 released, vmctl(8): CLI syntax changes, and things that Linux distributions should not do when packaging.
\n\n\n\n\nDragonFlyBSD lead developer Matthew Dillon has been working on a big VM rework in the name of performance and other kernel improvements recently. Here is a look at how those DragonFlyBSD 5.5-DEVELOPMENT improvements are paying off compared to DragonFlyBSD 5.4 as well as FreeBSD 12 and five Linux distribution releases. With Dillon using an AMD Ryzen Threadripper system, we used that too for this round of BSD vs. Linux performance benchmarks.
\n\n
\nThe work by Dillon on the VM overhaul and other changes (including more HAMMER2 file-system work) will ultimately culminate with the DragonFlyBSD 5.6 release (well, unless he opts for DragonFlyBSD 6.0 or so). These are benchmarks of the latest DragonFlyBSD 5.5-DEVELOPMENT daily ISO as of this week benchmarked across DragonFlyBSD 5.4.3 stable, FreeBSD 12.0, Ubuntu 19.04, Red Hat Enterprise Linux 8.0, Debian 9.9, Debian Buster, and CentOS 7 1810 as a wide variety of reference points both from newer and older Linux distributions. (As for no Clear Linux reference point for a speedy reference point, it currently has a regression with AMD + Samsung NVMe SSD support on some hardware, including this box, prohibiting the drive from coming up due to a presumed power management issue that is still being resolved.)
\nWith Matthew Dillon doing much of his development on an AMD Ryzen Threadripper system after he last year proclaimed the greatness of these AMD HEDT CPUs, for this round of testing I also used a Ryzen Threadripper 2990WX with 32 cores / 64 threads. Tests of other AMD/Intel hardware with DragonFlyBSD will come as the next stable release is near and all of the kernel work has settled down. For now it's mostly entertaining our own curiosity how well these DragonFlyBSD optimizations are paying off and how it's increasing the competition against FreeBSD 12 and Linux distributions.
\n
\n\n\nMaybe you have been reading recently about the release of OpenBSD 6.5 and wonder, "What are the differences between Linux and OpenBSD?"
\n
\nI've also been there at some point in the past and these are my conclusions.
\nThey also apply, to some extent, to other BSDs. However, an important disclaimer applies to this article.
\nThis list is aimed at people who are used to Linux and are curious about OpenBSD. It is written to highlight the most important changes from their perspective, not the absolute most important changes from a technical standpoint.
\nPlease bear with me.
\n\n\nWe are very happy to announce The NetBSD Foundation Google Summer of Code 2019 projects:
\n
\n\n\nThe communiting bonding period - where students get in touch with mentors and community - started yesterday. The coding period will start from May 27 until August 19.
\n
\nPlease welcome all our students and a big good luck to students and mentors! A big thank to Google and The NetBSD Foundation organization mentors and administrators! Looking forward to a great Google Summer of Code!
\n\n\nThe opening keynote at EuroBSDCon 2016 predicted the future 10 years of BSDs. Amongst all the funny previsions, gnn@FreeBSD said that by 2026 OpenBSD will have its first implementation of SMP. Almost 3 years after this talk, that sounds like a plausible forecast... Why? Where are we? What can we do? Let's dive into the issue!
\n
\n\n\nMost of OpenBSD's kernel still runs under a single lock, ze KERNEL_LOCK(). That includes most of the syscalls, most of the interrupt handlers and most of the fault handlers. Most of them, not all of them. Meaning we have collected & fixed bugs while setting up infrastructures and examples. Now this lock remains the principal responsible for the spin % you can observe in top(1) and systat(1).
\n
\nI believe that we opted for a difficult hike when we decided to start removing this lock from the bottom. As a result many SCSI & Network interrupt handlers as well as all Audio & USB ones can be executed without big lock. On the other hand very few syscalls are already or almost ready to be unlocked, as we incorrectly say. This explains why basic primitives like tsleep(9), csignal() and selwakeup() are only receiving attention now that the top of the Network Stack is running (mostly) without big lock.
\n\n\nIn the past years, most of our efforts have been invested into the Network Stack. As I already mentioned it should be ready to be parallelized. However think we should now concentrate on removing the KERNEL_LOCK(), even if the code paths aren't performance critical.
\n
\n\n\nThis release finally addresses some of the problems that prevent simple running of several games.
\n\n
\nThis happens for example when an old FNA.dll library comes with the games that doesn't match the API of our native libraries like SDL2, OpenAL, or MojoShader anymore. Some of those cases can be fixed by simply dropping in a newer FNA.dll. fnaify now asks if FNA 17.12 should be automatically added if a known incompatible FNA version is found. You simply answer yes or no.Another blocker happens when the game expects to check the SteamAPI - either from a running Steam process, or a bundled steam_api library. OpenBSD 6.5-current now has steamworks-nosteam in ports, a stub library for Steamworks.NET that prevents games from crashing simply because an API function isn't found. The repo is here. fnaify now finds this library in /usr/local/share/steamstubs and uses it instead of the bundled (full) Steamworks.NET.dll.
\n
\nThis may help with any games that use this layer to interact with the SteamAPI, mostly those that can only be obtained via Steam.
\n\n\nThe order of the arguments in the create, start, and stop commands of vmctl(8) has been changed to match a commonly expected style. Manual usage or scripting with vmctl must be adjusted to use the new syntax.
\n
\nFor example, the old syntax looked like this:
# vmctl create disk.qcow2 -s 50G
\n\n\nThe new syntax specifies the command options before the argument:
\n
# vmctl create -s 50G disk.qcow2
\n\n\nRight now I am a bit unhappy at Fedora for a specific packaging situation, so let me tell you a little story of what I, as a system administrator, would really like distributions to not do.
\n\n
\nFor reasons beyond the scope of this blog entry, I run a Prometheus and Grafana setup on both my home and office Fedora Linux machines (among other things, it gives me a place to test out various things involving them). When I set this up, I used the official upstream versions of both, because I needed to match what we are running (or would soon be).
\nRecently, Fedora decided to package Grafana themselves (as a RPM), and they called this RPM package 'grafana'. Since the two different packages are different versions of the same thing as far as package management tools are concerned, Fedora basically took over the 'grafana' package name from Grafana. This caused my systems to offer to upgrade me from the Grafana.com 'grafana-6.1.5-1' package to the Fedora 'grafana-6.1.6-1.fc29' one, which I actually did after taking reasonable steps to make sure that the Fedora version of 6.1.6 was compatible with the file layouts and so on from the Grafana version of 6.1.5.
\nWhy is this a problem? It's simple. If you're going to take over a package name from the upstream, you should keep up with the upstream releases. If you take over a package name and don't keep up to date or keep up to date only sporadically, you cause all sorts of heartburn for system administrators who use the package. The least annoying future of this situation is that Fedora has abandoned Grafana at 6.1.6 and I am going to 'upgrade' it with the upstream 6.2.1, which will hopefully be a transparent replacement and not blow up in my face. The most annoying future is that Fedora and Grafana keep ping-ponging versions back and forth, which will make 'dnf upgrade' into a minefield (because it will frequently try to give me a 'grafana' upgrade that I don't want and that would be dangerous to accept). And of course this situation turns Fedora version upgrades into their own minefield, since now I risk an upgrade to Fedora 30 actually reverting the 'grafana' package version on me.
\n
GPU passthrough on bhyve, confusion with used/free disk space on ZFS, OmniOS Community Edition, pfSense 2.4.4 Release p3, NetBSD 8.1 RC1, FreeNAS as your Server OS, and more.
\n\n\n\n\nNormally we cover news focused on KVM and sometimes Xen, but something very special has happened with their younger cousin in the BSD world, Bhyve.\n For those that don’t know, Bhyve (pronounced bee-hive) is the native hypervisor in FreeBSD. It has many powerful features, but one that’s been a pain point for some years now is VGA passthrough. Consumer GPUs have not been useable until very recently despite limited success with enterprise cards.\n However, Twitter user Michael Yuji found a workaround that enables passing through a consumer card to any *nix system configured to use X11:
\n
\n\n\nAll you have to do is add a line pointing the X server to the Bus ID of the passed card and the VM will boot, with acceleration and everything. He theorizes that this may not be possible on windows because of the way it looks for display devices, but it’s a solid start.\n As soon as development surrounding VGA passthrough matures on Bhyve, it will become a very attractive alternative to more common tools like Hyper-V and Qemu, because it makes many powerful features available in the host system like jails, boot environments, BSD networking, and tight ZFS integration. For example, you could potentially run your Router, NAS, preferred workstation OS and any number of other things in one box, and only have to spin up a single VM because of the flexibility afforded by jails over Linux-based containers.\n The user who found this workaround also announced they’d be writing it up at some point, so stay tuned for details on the process.\n It’s been slow going on Bhyve passthrough development for a while, but this new revelation is encouraging. We’ll be closely monitoring the situation and report on any other happenings.
\n \n
\n
\n\n\nI use ZFS extensively. ZFS is my favorite file system. I write articles and give lectures about it. I work with it every day. In traditional file systems we use df(1) to determine free space on partitions. We can also use du(1) to count the size of the files in the directory. But it’s different on ZFS and this is the most confusing thing EVER. I always forget which tool reports what disk space usage! Every time somebody asks me, I need to google it. For this reason I decided to document it here - for myself - because if I can’t remember it at least I will not need to google it, as it will be on my blog, but maybe you will also benefit from this blog post if you have the same problem or you are starting your journey with ZFS.
\n \nThe understanding of how ZFS is uses space and how to determine which value means what is a crucial thing. I hope thanks to this article I will finally remember it!
\n
\n\n\nThe OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030.\n OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details.\n This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details.\n If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28.\n The OmniOS team and the illumos community have been very active in creating new features and improving existing ones over the last 6 months.
\n
\n\n\nWe are pleased to announce the release of pfSense® software version 2.4.4-p3, now available for new installations and upgrades!\n pfSense software version 2.4.4-p3 is a maintenance release, bringing a number of security enhancements as well as a handful of fixes for issues present in the 2.4.4-p2 release.\n pfSense 2.4.4-RELEASE-p3 updates and installation images are available now!\n To see a complete list of changes and find more detail, see the Release Notes.\n We had hoped to bring you this release a few days earlier, but given the announcement last Tuesday of the Intel Microarchitectural Data Sampling (MDS) issue, we did not have sufficient time to fully incorporate those corrections and properly test for release on Thursday. We felt that it was worth delaying for a few days, rather than making multiple releases within a week.
\n
\n\n\nDue to the significant nature of the changes in 2.4.4 and later, \n warnings and error messages, particularly from PHP and package updates, are likely to occur during the upgrade process. In nearly all cases these errors are a harmless side effect of the changes between FreeBSD 11.1 and 11.2 and between PHP 5.6 and PHP 7.2.\n Always take a backup of the firewall configuration prior to any major change to the firewall, such as an upgrade.\n Do not update packages before upgrading pfSense! Either remove all packages or do not update packages before running the upgrade.\n The upgrade will take several minutes to complete. The exact time varies based on download speed, hardware speed, and other factors such installed packages. Be patient during the upgrade and allow the firewall enough time to complete the entire process. After the update packages finish downloading it could take 10-20 minutes or more until the upgrade process ends. The firewall may reboot several times during the upgrade process. Monitor the upgrade from the firewall console for the most accurate view.
\n
\n\n\nThe NetBSD Project is pleased to announce NetBSD 8.1, the first update of the NetBSD 8 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.
\n \nSome highlights of the 8.1 release are:
\n
\n\n\nWhat if you could have a server OS that had built in RAID, NAS and SAN functionality, and could manage packages, containers and VMs in a GUI? What if that server OS was also free to download and install? Wouldn’t that be kind of awesome? Wouldn’t that be FreeNAS?\n FreeNAS is the world’s number one, open source storage OS, but it also comes equipped with all the jails, plugins, and VMs you need to run additional server-level services for things like email and web site hosting. File, Block, and even Object storage is all built-in and can be enabled with a few clicks. The ZFS file system scales to more drives than you could ever buy, with no limits for dataset sizes, snapshots, and restores.\n FreeNAS is also 100% FreeBSD. This is the OS used in the Netflix CDN, your PS4, and the basis for iOS. Set up a jail and get started downloading packages like Apache or NGINX for web hosting or Postfix for email service.\n Just released, our new TrueCommand management platform also streamlines alerts and enables multi-system monitoring.
\n
FreeBSD 11.3-beta 1 is out, BSDCan 2019 recap, OpenIndiana 2019.04 is out, Overview of ZFS Pools in FreeNAS, why open source firmware is important for security, a new Opnsense release, wireguard on OpenBSD, and more.
\n\n\n\n\nWe have released a new OpenIndiana Hipster snapshot 2019.04. The noticeable changes:
\n
Firefox was updated to 60.6.3 ESR
Virtualbox packages were added (including guest additions)
Mate was updated to 1.22
IPS has received updates from OmniOS CE and Oracle IPS repos, including automatic boot environment naming
Some OI-specific applications have been ported from Python 2.7/GTK 2 to Python 3.5/GTK 3
Quick Demo Video: https://www.youtube.com/watch?v=tQ0-fo3XNrg
\n\n\nFreeNAS uses the OpenZFS (ZFS) file system, which handles both disk and volume management. ZFS offers RAID options mirror, stripe, and its own parity distribution called RAIDZ that functions like RAID5 on hardware RAID. The file system is extremely flexible and secure, with various drive combinations, checksums, snapshots, and replication all possible. For a deeper dive on ZFS technology, read the ZFS Primer section of the FreeNAS documentation.
\n \nSUGGEST LAYOUT attempts to balance usable capacity and redundancy by automatically choosing an ideal vdev layout for the number of available disks.
\n
\n\n\nThe goal of the root of trust should be to verify that the software installed in every component of the hardware is the software that was intended. This way you can know without a doubt and verify if hardware has been hacked. Since we have very little to no visibility into the code running in a lot of places in our hardware it is hard to do this. How do we really know that the firmware in a component is not vulnerable or that is doesn’t have any backdoors? Well we can’t. Not unless it was all open source.\n Every cloud and vendor seems to have their own way of doing a root of trust. Microsoft has Cerberus, Google has Titan, and Amazon has Nitro. These seem to assume an explicit amount of trust in the proprietary code (the code we cannot see). This leaves me with not a great feeling. Wouldn’t it be better to be able to use all open source code? Then we could verify without a doubt that the code you can read and build yourself is the same code running on hardware for all the various places we have firmware. We could then verify that a machine was in a correct state without a doubt of it being vulnerable or with a backdoor.\n It makes me wonder what the smaller cloud providers like DigitalOcean or Packet have for a root of trust. Often times we only hear of these projects from the big three or five.
\n
\n\n\nThis update addresses several privilege escalation issues in the access control implementation and new memory disclosure issues in Intel CPUs. We would like to thank Arnaud Cordier and Bill Marquette for the top-notch reports and coordination.
\n
Here are the full patch notes:
system: address CVE-2019-11816 privilege escalation bugs[1] (reported by Arnaud Cordier)
system: /etc/hosts generation without interfacehasgateway()
system: show correct timestamp in config restore save message (contributed by nhirokinet)
system: list the commands for the pluginctl utility when n+ argument is given
system: introduce and use userIsAdmin() helper function instead of checking for 'page-all' privilege directly
system: use absolute path in widget ACLs (reported by Netgate)
system: RRD-related cleanups for less code exposure
interfaces: add EN DUID Generation using OPNsense PEN (contributed by Team Rebellion)
interfaces: replace legacygetallinterface_addresses() usage
firewall: fix port validation in aliases with leading / trailing spaces
firewall: fix outbound NAT translation display in overview page
firewall: prevent CARP outgoing packets from using the configured gateway
firewall: use CARP net.inet.carp.demotion to control current demotion in status page
firewall: stop live log poller on error result
dhcpd: change rule priority to 1 to avoid bogon clash
dnsmasq: only admins may edit custom options field
firmware: use insecure mode for base and kernel sets when package fingerprints are disabled
firmware: add optional device support for base and kernel sets
firmware: add Hostcentral mirror (HTTP, Melbourne, Australia)
ipsec: always reset rightallowany to default when writing configuration
lang: say \"hola\" to Spanish as the newest available GUI language
lang: updates for Chinese, Czech, Japanese, German, French, Russian and Portuguese
network time: only admins may edit custom options field
openvpn: call openvpnrefreshcrls() indirectly via plugin_configure() for less code exposure
openvpn: only admins may edit custom options field to prevent privilege escalation (reported by Bill Marquette)
openvpn: remove custom options field from wizard
unbound: only admins may edit custom options field
wizard: translate typehint as well
plugins: os-freeradius 1.9.3 fixes string interpolation in LDAP filters (contributed by theq86)
plugins: os-nginx 1.12[2]
plugins: os-theme-cicada 1.17 (contributed by Team Rebellion)
plugins: os-theme-tukan 1.17 (contributed by Team Rebellion)
src: timezone database information update[3]
src: install(1) broken with partially matching relative paths[4]
src: microarchitectural Data Sampling (MDS) mitigation[5]
ports: carootnss 3.44
ports: php 7.2.18[6]
ports: sqlite 3.28.0[7]
ports: strongswan custom XAuth generic patch removed
\n\n\nEarlier this week I imported a port for WireGuard into the OpenBSD ports tree. At the moment we have the userland daemon and the tools available. The in-kernel implementation is only available for Linux. At the time of writing there are packages available for -current.\n Jason A. Donenfeld (WireGuard author) has worked to support OpenBSD in WireGuard and as such his post on ports@ last year got me interested in WireGuard, since then others have toyed with WireGuard on OpenBSD before and as such I've used Ted's article as a reference. Note however that some of the options mentioned there are no longer valid. Also, I'll be using two OpenBSD peers here.\n The setup will be as follows: two OpenBSD peers, of which we'll dub wg1 the server and wg2 the client. The WireGuard service on wg1 is listening on 100.64.4.3:51820.
\n
\n\n\nWireGuard (cl)aims to be easier to setup and faster than OpenVPN and while I haven't been able to verify the latter, the first is certainly true...once you've figured it out. Most documentation out there is for Linux so I had to figure out the wireguardgo service and the tun parameters. But all in all, sure, it's easier. Especially the client configuration on iOS which I didn't cover here because it's essentially pkgadd libqrencode ; cat client.conf | qrencode -t ansiutf8, scan the code with the WireGuard app and you're good to go. What is particularly neat is that WireGuard on iOS supports Always-on.
\n
Running AIX on QEMU on Linux on Windows, your NAS fleet with TrueCommand, Unleashed 1.3 is available, LLDB: CPU register inspection support extension, V7 Unix programs often not written as expected, and more.
\n\n\n\n\nYES it’s real!\n I’m using the Linux subsystem on Windows, as it’s easier to build this Qemu tree from source. I’m using Debian, but these steps will work on other systems that use Debian as a base.\n first thing first, you need to get your system with the needed pre-requisites to compile\n Great with those in place, now clone Artyom Tarasenko’s source repository\n Since the frame buffer apparently isn’t quite working just yet, I configure for something more like a text mode build.\n Now for me, GCC 7 didn’t build the source cleanly. I had to make a change to the file config-host.mak and remove all references to -Werror. Also I removed the sound hooks, as we won’t need them.\n Now you can build Qemu.\n Okay, all being well you now have a Qemu. Now following the steps from Artyom Tarasenko’s blog post, we can get started on the install!
\n
\n\n\nHundreds of thousands of FreeNAS and TrueNAS systems are deployed around the world, with many sites having dozens of systems. Managing multiple systems individually can be time-consuming. iXsystems has responded to the challenge by creating a “single pane of glass” application to simplify the scaling of data, drive management, and administration of iXsystems NAS platforms. We are proud to introduce TrueCommand.\n TrueCommand is a ZFS-aware management application that manages TrueNAS and FreeNAS systems. \n The public Beta of TrueCommand is available for download now. TrueCommand can be used with small iXsystems NAS fleets for free. Licenses can be purchased for large-scale deployments and enterprise support.\n TrueCommand expands on the ease of use and power of TrueNAS and FreeNAS systems with multi-system management and reporting.
\n
\n\n\nThis is the fourth release of Unleashed - an operating system fork of illumos. For more information about Unleashed itself and the download links, see our website.\n As one might expect, this release removes a few things.\n The most notable being the removal of ksh93 along with all its libs.\n As far as libc interfaces are concerned, a number of non-standard functions were removed. In general, they have been replaced by the standards-compliant versions. (getgrentr, fgetgrentr, getgrgidr, getgrnamr, ttynamer, getloginr, shmdt, sigwait, gethostname, putmsg, putpmsg, and getaddrinfo)\n Additionally, wordexp and wordfree have been removed from libc. Even though they are technically required by POSIX, software doesn't seem to use them. Because of the fragile implementation (shelling out), we took the OpenBSD approach and just removed them.\n The default compilation environment now includes XOPENSOURCE=700 and EXTENSIONS. Additionally, all applications now use 64-bit file offsets, making use of LARGEFILESOURCE, LARGEFILE64SOURCE, and FILEOFFSET_BITS unnecessary.\n Last but not least, nightly.sh is no more. In short, to build one simply runs 'make'. (See README for detailed build instructions.)
\n
\n\n\nWhy did we decide to fork illumos? After all, there are already many illumos distributions available to choose from. We felt we could do better than any of them by taking a more aggressive stance toward compatibility and reducing cruft from code and community interactions alike.
\n
\n\n\nUpstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.\n In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and updating NetBSD distribution to LLVM 8 (which is still stalled by unresolved regressions in inline assembly syntax). You can read more about that in my Mar 2019 report.\n In April, my main focus was on fixing and enhancing the support for reading and writing CPU registers. In this report, I'd like to shortly summarize what I have done, what I have learned in the process and what I still need to do.
\n
\n\n\nMy work continues with the two milestones from last month, plus a third that's closely related:\n Add support for FPU registers support for NetBSD/i386 and NetBSD/amd64.\n Support XSAVE, XSAVEOPT, ... registers in core(5) files on NetBSD/amd64.\n Add support for Debug Registers support for NetBSD/i386 and NetBSD/amd64.\n The most important point right now is deciding on the format for passing the remaining registers, and implementing the missing ptrace interface kernel-side. The support for core files should follow using the same format then.\n Userland-side, I will work on adding matching ATF tests for ptrace features and implement LLDB side of support for the new ptrace interface and core file notes. Afterwards, I will start working on improving support for the same things on 32-bit (i386) executables.
\n
\n\n\nYesterday I wrote that V7 ed read its terminal input in cooked mode a line at a time, which was an efficient, low-CPU design that was important on V7's small and low-power hardware. Then in comments, frankg pointed out that I was wrong about part of that, namely about how ed read its input.
\n
\n\n\nReading this section of the source code for ed taught me that it has an interesting, undocumented, and entirely characteristic little behavior. Officially, ed commands that have you enter new text have that new text terminate by a . on a line by itself:
\n \nIn other words, it turns a single line with '.' into an EOF. The consequence of this is that if you type a real EOF at the start of a line, you get the same result, thus saving you one character (you use Control-D instead of '.' plus newline). This is very V7 Unix behavior, including the lack of documentation.
\n \nThis is also a natural behavior in one sense. A proper program has to react to EOF here in some way, and it might as well do so by ending the input mode. It's also natural to go on to try reading from the terminal again for subsequent commands; if this was a real and persistent EOF, for example because the pty closed, you'll just get EOF again and eventually quit. V7 ed is slightly unusual here in that it deliberately converts '.' by itself to EOF, instead of signaling this in a different way, but in a way that's also the simplest approach; if you have to have some signal for each case and you're going to treat them the same, you might as well have the same signal for both cases.
\n \nModern versions of ed appear to faithfully reimplement this convenient behavior, although they don't appear to document it. I haven't checked OpenBSD, but both FreeBSD ed and GNU ed work like this in a quick test. I haven't checked their source code to see if they implement it the same way.
\n \n
\n
36 year old UFS bug fixed, a BSD for the road, automatic upgrades with OpenBSD, DTrace ext2fs support in FreeBSD, Dedicated SSH tunnel user, upgrading VMM VMs to OpenBSD 6.5, and more.
\n\n\n\n\nThis update eliminates a kernel stack disclosure bug in UFS/FFS directory entries that is caused by uninitialized directory entry padding written to the disk.
\n \n\n
\n \n- When the directory entry is written to disk, it is written as a full 32bit entry, and the unused bytes were not initialized, so could possibly contain sensitive data from the kernel stack\n It can be viewed by any user with read access to that directory. Up to 3 bytes of kernel stack are disclosed per file entry, depending on the the amount of padding the kernel needs to pad out the entry to a 32 bit boundary. The offset in the kernel stack that is disclosed is a function of the filename size. Furthermore, if the user can create files in a directory, this 3 byte window can be expanded 3 bytes at a time to a 254 byte window with 75% of the data in that window exposed. The additional exposure is done by removing the entry, creating a new entry with a 4-byte longer name, extracting 3 more bytes by reading the directory, and repeating until a 252 byte name is created.\n This exploit works in part because the area of the kernel stack that is being disclosed is in an area that typically doesn't change that often (perhaps a few times a second on a lightly loaded system), and these file creates and unlinks themselves don't overwrite the area of kernel stack being disclosed.\n It appears that this bug originated with the creation of the Fast File System in 4.1b-BSD (Circa 1982, more than 36 years ago!), and is likely present in every Unix or Unix-like system that uses UFS/FFS. Amazingly, nobody noticed until now.\n This update also adds the -z flag to fsck_ffs to have it scrub the leaked information in the name padding of existing directories. It only needs to be run once on each UFS/FFS filesystem after a patched kernel is installed and running.\n Submitted by: David G. Lawrence dg@dglawrence.com
\n \n- So a patched kernel will no longer leak this data, and running the
\n \nfsck_ffs -z
command will erase any leaked data that may exist on your system- OpenBSD commit with additional detail on mitigations\n The impact on OpenBSD is very limited:\n 1 - such stack bytes can be found in raw-device reads, from group operator. If you can read the raw disks you can undertake other more powerful actions.\n 2 - read(2) upon directory fd was disabled July 1997 because I didn't like how grep * would display garbage and mess up the tty, and applying vis(3) for just directory reads seemed silly. read(2) was changed to return 0 (EOF). Sep 2016 this was further changed to EISDIR, so you still cannot see the bad bytes.\n 3 - In 2013 when guenther adapted the getdents(2) directory-reading system call to 64-bit ino_t, the userland data format changed to 8-byte-alignment, making it incompatible with the 4-byte-alignment UFS on-disk format. As a result of code refactoring the bad bytes were not copied to userland. Bad bytes will remain in old directories on old filesystems, but nothing makes those bytes user visible.\n There will be no errata or syspatch issued. I urge other systems which do expose the information to userland to issue errata quickly, since this is a 254 byte infoleak of the stack which is great for ROP-chain building to attack some other bug. Especially if the kernel has no layout/link-order randomization ...
\n
\n
\n\n\nAs regular It’s FOSS readers should know, I like diving into the world of BSDs. Recently, I came across an interesting BSD that is designed to live on a thumb drive. Let’s take a look at NomadBSD.\n NomadBSD is different than most available BSDs. NomadBSD is a live system based on FreeBSD. It comes with automatic hardware detection and an initial config tool. NomadBSD is designed to “be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD’s hardware compatibility.”\n This German BSD comes with an OpenBox-based desktop with the Plank application dock. NomadBSD makes use of the DSB project. DSB stands for “Desktop Suite (for) (Free)BSD” and consists of a collection of programs designed to create a simple and working environment without needing a ton of dependencies to use one tool. DSB is created by Marcel Kaiser one of the lead devs of NomadBSD.\n Just like the original BSD projects, you can contact the NomadBSD developers via a mailing list.
\n
\n\n\nNomadBSD recently released version 1.2 on April 21, 2019. This means that NomadBSD is now based on FreeBSD 12.0-p3. TRIM is now enabled by default. One of the biggest changes is that the initial command-line setup was replaced with a Qt graphical interface. They also added a Qt5 tool to install NomadBSD to your hard drive. A number of fixes were included to improve graphics support. They also added support for creating 32-bit images.
\n
\n\n\nI first discovered NomadBSD back in January when they released 1.2-RC1. At the time, I had been unable to install Project Trident on my laptop and was very frustrated with BSDs. I downloaded NomadBSD and tried it out. I initially ran into issues reaching the desktop, but RC2 fixed that issue. However, I was unable to get on the internet, even though I had an Ethernet cable plugged in. Luckily, I found the wifi manager in the menu and was able to connect to my wifi.\n Overall, my experience with NomadBSD was pleasant. Once I figured out a few things, I was good to go. I hope that NomadBSD is the first of a new generation of BSDs that focus on mobility and ease of use. BSD has conquered the server world, it’s about time they figured out how to be more user-friendly.
\n \n
\n
upgrade](https://www.tumfatig.net/20190426/openbsd-automatic-upgrade/)
\n\n\n\n\nOpenBSD 6.5 advertises for an installer improvement: rdsetroot(8) (a build-time tool) is now available for general use. Used in combination with autoinstall.8, it is now really easy to do automatic upgrades of your OpenBSD instances.\n I first manually upgraded my OpenBSD sandbox to 6.5. Once that was done, I could use the stock rdsetroot(8) tool. The plan is quite simple: write an unattended installation response file, insert it to a bsd.rd 6.5 installation image and reboot my other OpenBSD instances using that image.
\n
\n\n\nThere must be a way to run onetime commands (in the manner of fw_update) to automatically run sysmerge and packages upgrades. As for now, I’d rather do it manually.\n This worked like a charm on two Synology KVM instances using a single sd0 disk, on my Thinkpad X260 using Encrypted root with Keydisk and on a Vultr instance using Encrypted root with passphrase. And BTW, the upgrade on the X260 used the (iwn0) wireless connection.\n I just read that florian@ has released the sysupgrade(8) utility which should be released with OpenBSD 6.6. That will make upgrades even easier! Until then, happy upgrading.
\n
Which logs were replaced by dtrace-probes:
\n\nThe only debug macro, which was leaved is EXT2FSPRINTEXTENTS.
It is impossible to replace it by dtrace-probes, because the additional logic is required to walk thru file extents.
The user still be able to see mount errors in the dmesg in case of:
\n\n\n\n\nI use ssh tunneling A LOT, for everything. Yesterday, I removed the public access of my IMAP server, it’s now only available through ssh tunneling to access the daemon listening on localhost. I have plenty of daemons listening only on localhost that I can only reach through a ssh tunnel. If you don’t want to bother with ssh and redirect ports you need, you can also make a VPN (using ssh, openvpn, iked, tinc…) between your system and your server. I tend to avoid setting up VPN for the current use case as it requires more work and more maintenance than running ssh server and a ssh client.\n The last change, for my IMAP server, added an issue. I want my phone to access the IMAP server but I don’t want to connect to my main account from my phone for security reasons. So, I need a dedicated user that will only be allowed to forward ports.\n This is done very easily on OpenBSD.\n The steps are: 1. generate ssh keys for the new user 2. add an user with no password 3. allow public key for port forwarding\n Obviously, you must allow users (or only this one) to make port forwarding in your sshd_config.
\n \n
\n
\n\n\nWe're running dedicated vmm(4)/vmd(8) servers to host opinionated VMs.\n OpenBSD 6.5 is released! There are two ways you can upgrade your VM.\n Either do a manual upgrade or leverage autoinstall(8). You can take care of it via the console with vmctl(8).
\n
\n\n\nTo get connected to the console you need to have access to the host your VM is running on. The same username and public SSH key, as provided for the VM, are used to create a local user on the host.\n When this is done you can use vmctl(8) to manage your VM. The options you have are:
\n
```$ vmctl start id [-c]```\n
\n\n$ vmctl stop id [-fw]```
\n\n```-w Wait until the VM has been terminated.```\n
\n\n-c Automatically connect to the VM console.```
\n\nFreeBSD ZFS vs. ZoL performance, Dragonfly 5.4.2 has been release, containing web services with iocell, Solaris 11.4 SRU8, Problem with SSH Agent forwarding, OpenBSD 6.4 to 6.5 upgrade guide, and more.
\n\n\n\n\nWith iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS.\n Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices.\n FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out.\n Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape.
\n
\n\n\nHere's the tag commit, for what has changed from 5.4.1 to 5.4.2\n The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely. I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon. This version includes Matt's fix for the HAMMER2 corruption bug he identified recently.\n If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work.
\n
> cd /usr/src\n> git pull\n> make buildworld.\n> make buildkernel.\n> make installkernel.\n> make installworld\n> make upgrade\n
\n\n\n\n\nAfter your next reboot, you can optionally update your rescue system:
\n
> cd /usr/src\n> make initrd\n
\n\n\n\n\nAs always, make sure your packages are up to date:
\n
> pkg update\n> pkg upgrade\n
\n\n\n\n\nI'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛 \n If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U
\n
\n\n\nThere are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell.
\n
\n\n\nOnce you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets.
\n \nAs you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases.
\n
\n\n\nToday we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1.
\n \n\n
\n- This SRU introduces the following enhancements:\n \n \n
\n\n
\n- Integration of 28060039 introduced an issue where any firmware update/query commands will log eereports and repeated execution of such commands led to faulty/degraded NIC. The issue has been addressed in this SRU.
\n \n- UCB (libucb, librpcsoc, libdbm, libtermcap, and libcurses) libraries have been reinstated for Oracle Solaris 11.4
\n \n- Re-introduction of the service fc-fabric.
\n \n- ibus has been updated to 1.5.19
\n\n\nAfter hacking the matrix.org website today, the attacker opened a series of GitHub issues mentioning the flaws he discovered. In one of those issues, he mentions that “complete compromise could have been avoided if developers were prohibited from using [SSH agent forwarding].”\n Here’s what man ssh_config has to say about ForwardAgent: \"Agent forwarding should be enabled with caution. Users with the ability to bypass file permissions on the remote host (for the agent’s Unix-domain socket) can access the local agent through the forwarded connection. An attacker cannot obtain key material from the agent, however they can perform operations on the keys that enable them to authenticate using the identities loaded into the agent.\"\"\n Simply put: if your jump box is compromised and you use SSH agent forwarding to connect to another machine through it, then you risk also compromising the target machine!\n Instead, you should use either ProxyCommand or ProxyJump (added in OpenSSH 7.3). That way, ssh will forward the TCP connection to the target host via the jump box and the actual connection will be made on your workstation. If someone on the jump box tries to MITM your connection, then you will be warned by ssh.
\n
\n\n\nStart by performing the pre-upgrade steps. Next, boot from the install kernel, bsd.rd: use bootable install media, or place the 6.5 version of bsd.rd in the root of your filesystem and instruct the boot loader to boot this kernel. Once this kernel is booted, choose the (U)pgrade option and follow the prompts. Apply the configuration changes and remove the old files. Finish up by upgrading the packages: pkg_add -u.\n Alternatively, you can use the manual upgrade process.\n You may wish to check the errata page or upgrade to the stable branch to get any post-release fixes.
\n
OpenBSD 6.5 has been released, mount ZFS datasets anywhere, help test upcoming NetBSD 9 branch, LibreSSL 2.9.1 is available, Bail Bond Denied Edition of FreeBSD Mastery: Jails, and one reason ed(1) was a good editor back in the days in this week’s episode.
\n\n\n\n\nZFS is very flexible about mountpoints, and there are many features available to provide great flexibility.\n When you create zpool maintank, the default mountpoint is /maintank.\n You might be happy with that, but you don’t have to be content. You can do magical things.
\n
\n\n\nFolks,\n once again we are quite late for branching the next NetBSD release (NetBSD 9).\n Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that.\n On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later.\n On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help:
\n \n\n
\n \n- please test -current, on all the various machines you have
\n \n- especially interesting would be test results from uncommon architectures\n or strange combinations (like the sparc userland on sparc64 kernel issue\n I ran in yesterday)\n Please test, report success, and file PRs for failures!\n We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may.\n We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go.\n Our goal is to have an unprecedented short release cycle this time. But..\n we always say that upfront.
\n
\n
\n\n\nWe have released LibreSSL 2.9.1, which will be arriving in the LibreSSL\n directory of your local OpenBSD mirror soon. This is the first stable release\n from the 2.9 series, which is also included with OpenBSD 6.5
\n \nIt includes the following changes and improvements from LibreSSL 2.8.x:
\n
API and Documentation Enhancements
\n\nCompatibility Changes
\n\nTesting and Proactive Security
\n\nInternal Improvements
\n\nPortable Improvements
\n\nBug Fixes
\n\n\n\n\nThe LibreSSL project continues improvement of the codebase to reflect modern,\n safe programming practices. We welcome feedback and improvements from the\n broader community. Thanks to all of the contributors who helped make this\n release possible.
\n \n
\n
\n\n\nI had a brilliant, hideous idea: to produce a charity edition of FreeBSD Mastery: Jails featuring the cover art I would use if I was imprisoned and did not have access to a real cover artist. (Never mind that I wouldn’t be permitted to release books while in jail: we creative sorts scoff at mere legal and cultural details.)\n I originally wanted to produce my own take on the book’s cover art. My first attempt failed spectacularly.\n I downgraded my expectations and tried again. And again. And again.\n I’m pleased to reveal the final cover for FreeBSD Mastery: Jails–Bail Bond Edition!\n This cover represents the very pinnacle of my artistic talents, and is the result of literally hours of effort.\n But, as this book is available only to the winner of charity fund-raisers, purchase of this tome represents moral supremacy. I recommend flaunting it to your family, coworkers, and all those of lesser character.\n Get your copy by winning the BSDCan 2019 charity auction… or any other other auction-type event I deem worthwhile.\n As far as my moral fiber goes: I have learned that art is hard, and that artists are not paid enough.\n And if I am ever imprisoned, I do hope that you’ll contribute to my bail fund. Otherwise, you’ll get more covers like this one.
\n
\n\n\nIt is common to describe ed(1) as being line oriented, as opposed to screen oriented editors like vi. This is completely accurate but it is perhaps not a complete enough description for today, because ed is line oriented in a way that is now uncommon. After all, you could say that your shell is line oriented too, and very few people use shells that work and feel the same way ed does.\n The surface difference between most people's shells and ed is that most people's shells have some version of cursor based interactive editing. The deeper difference is that this requires the shell to run in character by character TTY input mode, also called raw mode. By contrast, ed runs in what Unix usually calls cooked mode, where it reads whole lines from the kernel and the kernel handles things like backspace. All of ed's commands are designed so that they work in this line focused way (including being terminated by the end of the line), and as a whole ed's interface makes this whole line input approach natural. In fact I think ed makes it so natural that it's hard to think of things as being any other way. Ed was designed for line at a time input, not just to not be screen oriented.\n This input mode difference is not very important today, but in the days of V7 and serial terminals it made a real difference. In cooked mode, V7 ran very little code when you entered each character; almost everything was deferred until it could be processed in bulk by the kernel, and then handed to ed all in a single line which ed could also process all at once. A version of ed that tried to work in raw mode would have been much more resource intensive, even if it still operated on single lines at a time.
\n
Introducing funlinkat(), an OpenBSD Router with AT&T U-Verse, using NetBSD on a raspberry pi, ZFS encryption is still under development, Rump kernel servers and clients tutorial, Snort on OpenBSD 6.4, and more.
\n\n\n\n\nOne of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove.\n The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen:
\n \n\n
\n \n- We will remove an entry in the directory with the filename.
\n \n- We will decrease a file reference count (in inode).
\n \n- If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later\n This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself.\n Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time.\n The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking.\n \n \n
\n\n
\n- When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete.\n We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition.
\n \n- Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it.\n In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor\n We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel.\n This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file.\n After some discussion we decided that the only sensible option for FreeBSD would be to create a funlinkat() function. This syscall would only performs additional sanitary checks if we are removing a directory entry which corresponds to the inode stored which refers to the file descriptor.\n int funlinkat(int dfd, const char *path, int fd, int flags);\n The API above will check if the path opened relative to the dfd points to the same vnode. Thanks to that we removed a race condition because all those sanitary checks are performed in the kernel mode while the file system is locked and there is no possibility to change it.\n The fd parameter may be set to the FD_NONE value which will mean that the sanitary check should not be performed and funlinkat will behave just like unlinkat.\n As you can notice I often refer to the unlink syscall but at the end the APIs looks like unlinkat syscall. It is true that the unlink syscall is very old and kind of deprecated. That said I referred to unlink because it’s just simpler. These days unlink simply uses the same code as unlinkat.
\n
\n\n\nI upgraded to AT&T's U-verse Gigabit internet service in 2017 and it came with an Arris BGW-210 as the WiFi AP and router. The BGW-210 is not a terrible device, but I already had my own Airport Extreme APs wired throughout my house and an OpenBSD router configured with various things, so I had no use for this device. It's also a potentially-insecure device that I can't upgrade or fully disable remote control over.\n Fully removing the BGW-210 is not possible as we'll see later, but it is possible to remove it from the routing path. This is how I did it with OpenBSD.
\n \n
\n
\n\n\nDo you have an old Raspberry Pi lying around gathering dust, maybe after a recent Pi upgrade? Are you curious about BSD Unix? If you answered \"yes\" to both of these questions, you'll be pleased to know that the first is the solution to the second, because you can run NetBSD, as far back as the very first release, on a Raspberry Pi.\n BSD is the Berkley Software Distribution of Unix. In fact, it's the only open source Unix with direct lineage back to the original source code written by Dennis Ritchie and Ken Thompson at Bell Labs. Other modern versions are either proprietary (such as AIX and Solaris) or clever re-implementations (such as Minix and GNU/Linux). If you're used to Linux, you'll feel mostly right at home with BSD, but there are plenty of new commands and conventions to discover. If you're still relatively new to open source, trying BSD is a good way to experience a traditional Unix.\n Admittedly, NetBSD isn't an operating system that's perfectly suited for the Pi. It's a minimal install compared to many Linux distributions designed specifically for the Pi, and not all components of recent Pi models are functional under NetBSD yet. However, it's arguably an ideal OS for the older Pi models, since it's lightweight and lovingly maintained. And if nothing else, it's a lot of fun for any die-hard Unix geek to experience another side of the POSIX world.
\n \n
\n
\n\n\nOne of the big upcoming features that a bunch of people are looking forward to in ZFS is natively encrypted filesystems. This is already in the main development tree of ZFS On Linux, will likely propagate to FreeBSD (since FreeBSD ZFS will be based on ZoL), and will make it to Illumos if the Illumos people want to pull it in. People are looking forward to native encryption so much, in fact, that some of them have started using it in ZFS On Linux already, using either the development tip or one of the 0.8.0 release candidate pre-releases (ZoL is up to 0.8.0-rc3 as of now). People either doing this or planning to do this show up on the ZoL mailing list every so often.
\n \n \n \n
\n
\n\n\nThe rump anykernel architecture allows to run highly componentized kernel code configurations in userspace processes. Coupled with the rump sysproxy facility it is possible to run loosely distributed client-server \"mini-operating systems\". Since there is minimum configuration and the bootstrap time is measured in milliseconds, these environments are very cheap to set up, use, and tear down on-demand.\n This document acts as a tutorial on how to configure and use unmodified NetBSD kernel drivers as userspace services with utilities available from the NetBSD base system. As part of this, it presents various use cases. One uses the kernel cryptographic disk driver (cgd) to encrypt a partition. Another one demonstrates how to operate an FFS server for editing the contents of a file system even though your user account does not have privileges to use the host's mount() system call. Additionally, using a userspace TCP/IP server with an unmodified web browser is detailed.
\n \n
\n
\n\n\nAs you may recall from previous posts, I am running an OpenBSD server on an APU2 air-cooled 3 Intel NIC box as my router/firewall for my secure home network. Given that all of my Internet traffic flows through this box, I thought it would be a cool idea to run an Intrusion Detection System (IDS) on it. Snort is the big hog of the open source world so I took a peek in the packages directory on one of the mirrors and lo and behold we have the latest & greatest version of Snort available! Thanks devs!!!\n I did some quick Googling and didn’t find much “modern” howto help out there so, after some trial and error, I have it up and running. I thought I’d give back in a small way and share a quickie howto for other Googlers out there who are looking for guidance. Here’s hoping that my title is good enough “SEO” to get you here!
\n \n
\n
A PI-powered Plan 9 cluster, an SSH tarpit, rdist for when Ansible is too much, falling in love with OpenBSD again, how I created my first FreeBSD port, the Tilde Institute of OpenBSD education and more.
\n\n\n\n\nPlan 9 from Bell Labs comes from the same stable as the UNIX operating system, which of course Linux was designed after, and Apple’s OS X runs on top of a certified UNIX operating system. Just like UNIX, Plan 9 was developed as a research O/S — a vehicle for trying out new concepts — with it building on key UNIX principles and taking the idea of devices are just files even further.\n In this post, we take a quick look at the Plan 9 O/S and some of the notable features, before moving on to the construction of a self-contained 4-node Raspberry Pi cluster that will provide a compact platform for experimentation.
\n \n
\n
\n\n\nI’m a big fan of tarpits: a network service that intentionally inserts delays in its protocol, slowing down clients by forcing them to wait. This arrests the speed at which a bad actor can attack or probe the host system, and it ties up some of the attacker’s resources that might otherwise be spent attacking another host. When done well, a tarpit imposes more cost on the attacker than the defender.\n The Internet is a very hostile place, and anyone who’s ever stood up an Internet-facing IPv4 host has witnessed the immediate and continuous attacks against their server. I’ve maintained such a server for nearly six years now, and more than 99% of my incoming traffic has ill intent. One part of my defenses has been tarpits in various forms.
\n \n
\n
\n\n\nThe post written about rdist(1) on johan.huldtgren.com sparked\n us to write one as well. It's a great, underappreciated, tool. And we wanted to show how we wrapped doas(1) around it.\n There are two services in our infrastructure for which we were looking to keep the configuration in sync and to reload the process when the configuration had indeed changed. There is a pair of nsd(8)/unbound(8) hosts and a pair of hosts running relayd(8)/httpd(8) with carp(4) between them.\n We didn't have a requirement to go full configuration management with tools like Ansible or Salt Stack. And there wasn't any interest in building additional logic on top of rsync or repositories. > Enter rdist(1), rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing.
\n \n
\n
\n\n\nI was checking the other day and was appalled at how long it has been since I posted here. I had been working a job during 2018 that had me traveling 3,600 miles by air every week so that is at least a viable excuse.\n So what is my latest project? I wanted to get something better than the clunky old T500 “freedom laptop” that I could use as my daily driver. Some background here. My first paid gig as a programmer was on SunOS 4 (predecessor to Solaris) and Ultrix (on a DEC MicroVAX). I went from there to a Commodore Amiga (preemptive multitasking in 1985!). I went from there to OS/2 (I know, patron saint of lost causes) and then finally decided to “sell out” and move to Windows as the path of least resistance in the mid 90’s.\n My wife bought me an iPod literally just as they started working with computers other than Macs and I watched with fascination as Apple made the big gamble and moved away from PowerPC chips to Intel. That was the beginning of the Apple Fan Boi years for me. My gateway drug was a G4 MacMini and I managed somehow to get in on the pre-production, developer build of an Intel-based Mac. I was quite happy on the platform until about three years ago.
\n \n
\n
\n\n\nI created my first FreeBSD port recently. I found that FreeBSD didn't have a port for GoCD, which is a continuous integration and continuous deployment (CI/CD) system. This was a great opportunity to learn how to build a FreeBSD port while also contributing back to the community
\n \n
\n
\n\n\nWelcome to tilde.institute! This is an OpenBSD machine whose purpose is to provide a space in the tildeverse for experimentation with and education of the OpenBSD operating system. A variety of editors, shells, and compilers are installed to allow for development in a native OpenBSD environment. OpenBSD's httpd(8) is configured with slowcgi(8) as the fastcgi provider and sqlite3 available. This allows users to experiment with web development using compiled CGI in C, aka the BCHS Stack. In addition to php7.0 and mysql (mariadb) by request, this provides an environment where the development of complex web apps is possible.
\n \n
\n
This week we have a special episode with a Michael W. Lucas interview about his latest jail book that’s been released. We’re talking all things jails, writing, book sponsoring, the upcoming BSDCan 2019 conference, and more.
\n\n###Interview - Michael W. Lucas - mwl@mwl.io / @mwlauthor
\nFreeBSD Mastery: Jails
FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more.
\n\n##Headlines
\n###AsiaBSDcon 2019 recap
\n\nAdventure in DRMland - Or how to write a FreeBSD ARM64 DRM driver by Emmanuel
\n
\nVadot
\n\n\npowerpc64 architecture support in FreeBSD ports by Piotr Kubaj
\n
\nManaging System Images with ZFS by Allan Jude
\nFreeBSD - Improving block I/O compatibility in bhyve by Sergiu Weisz
\nSecurity Fantasies and Realities for the BSDs by George V.
\nNeville-Neil
\nZRouter: Remote update of firmware by Hiroki Mori
\nImproving security of the FreeBSD boot process by Marcin Wojtas
\n\nAdventures in DRMland by Emmanuel Vadot
\n
\nIntel HAXM by Kamil Rytarowski
\nBSD Solutions in Australian NGOs
\nContainer Migration on FreeBSD by Yuhei Takagawa
\nSecurity Fantasies and Realities for the BSDs by George Neville-Neil
\n\n\nZRouter: Remote update of firmware by Hiroki Mori
\n
\nImproving security of the FreeBSD boot process by Marcin Wojtas
###FreeBSD Quarterly Status Report - Fourth Quarter 2018
\n\n\n\n\nSince we are still on this island among many in this vast ocean of the Internet, we write this message in a bottle to inform you of the work we have finished and what lies ahead of us. These deeds that we have wrought with our minds and hands, they are for all to partake of - in the hopes that anyone of their free will, will join us in making improvements. In todays message the following by no means complete or ordered set of improvements and additions will be covered:
\n
\ni386 PAE Pagetables for up to 24GB memory support, Continuous Integration efforts, driver updates to ENA and graphics, ARM enhancements such as RochChip, Marvell 8K, and Broadcom support as well as more DTS files, more Capsicum possibilities, as well as pfsync improvements, and many more things that you can read about for yourselves.
\nAdditionally, we bring news from some islands further down stream, namely the nosh project, HardenedBSD, ClonOS, and the Polish BSD User-Group.
\nWe would, selfishly, encourage those of you who give us the good word to please send in your submissions sooner than just before the deadline, and also encourage anyone willing to share the good word to please read the section on which submissions we’re also interested in having.
###GhostBSD: A Solid Linux-Like Open Source Alternative
\n\n\n\n\nThe subject of this week’s Linux Picks and Pans is a representative of a less well-known computing platform that coexists with Linux as an open source operating system. If you thought that the Linux kernel was the only open source engine for a free OS, think again. BSD (Berkeley Software Distribution) shares many of the same features that make Linux OSes viable alternatives to proprietary computing platforms.
\n
\nGhostBSD is a user-friendly Linux-like desktop operating system based on TrueOS. TrueOS is, in turn, based on FreeBSD’s development branch. TrueOS’ goal is to combine the stability and security of FreeBSD with a preinstalled GNOME, MATE, Xfce, LXDE or Openbox graphical user interface.
\nI stumbled on TrueOS while checking out new desktop environments and features in recent new releases of a few obscure Linux distros. Along the way, I discovered that today’s BSD computing family is not the closed source Unix platform the “BSD” name might suggest.
\nIn last week’s Redcore Linux review, I mentioned that the Lumina desktop environment was under development for an upcoming Redcore Linux release. Lumina is being developed primarily for BSD OSes. That led me to circle back to a review I wrote two years ago on Lumina being developed for Linux.
\nGhostBSD is a pleasant discovery. It has nothing to do with being spooky, either. That goes for both the distro and the open source computing family it exposes.
\nKeep reading to find out what piqued my excitement about Linux-like GhostBSD.
##News Roundup
\n###SPARCbook 3000ST - The coolest 90s laptop
\n\n\nA few weeks back I managed to pick up an incredibly rare laptop in immaculate condition for $50 on Kijiji: a Tadpole Technologies SPARCbook 3000ST from 1997 (it also came with two other working Pentium laptops from the 1990s).
\n
\nSun computers were an expensive desire for many computer geeks in the 1990s, and running UNIX on a SPARC-based laptop was, well, just as cool as it gets. SPARC was an open hardware platform that anyone could make, and Tadpole licensed the Solaris UNIX operating system from Sun for their SPARCbooks. Tadpole essentially made high-end UNIX/VAX workstations on costly, unusual platforms (PowerPC, DEC Alpha, SPARC) but only their SPARCbooks were popular in the high-end UNIX market of the 1990s.
###OpenSSH 8.0 Releasing With Quantum Computing Resistant Keys
\n\n\n\n\n\n\nOpenSSH 7.9 came out with a host of bug fixes last year with few new features, as is to be expected in minor releases. However, recently, Damien Miller has announced that OpenSSH 8.0 is nearly ready to be released. Currently, it’s undergoing testing to ensure compatibility across supported systems.
\n
\n\n\nBetter Security
\n
\nCopying filenames with scp will be more secure in OpenSSH 8.0 due to the fact that copying filenames from a remote to local directory will prompt scp to check if the files sent from the server match your request. Otherwise, an attack server would theoretically be able to intercept the request by serving malicious files in place of the ones originally requested. Knowing this, you’re probably better off never using scp anyway. OpenSSH advises against it:
\n“The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead.”
\n\n\nssh(1): When prompting whether to record a new host key, accept the key fingerprint as a synonym for “yes”. This allows the user to paste a fingerprint obtained out of band at the prompt and have the client do the comparison for you.
\n
###Project Trident : 18.12-U8 Available
\n\n\n\n\nThank you all for your patience! Project Trident has finally finished some significant infrastructure updates over the last 2 weeks, and we are pleased to announce that package update 8 for 18.12-RELEASE is now available.
\n
\nTo switch to the new update, you will need to open the “Configuration” tab in the update manager and switch to the new “Trident-release” package repository. You can also perform this transition via the command line by running: sudo sysup --change-train Trident-release
##Beastie Bits
\n\n##Feedback/Questions
\n\nStorage changing software, what makes Unix special, what you need may be “pipeline +Unix commands”, running a bakery on Emacs and PostgreSQL, the ultimate guide to memorable tech talks, light-weight contexts, and more.
\n\n##Headlines
\n\n###Tracking a storage issue led to software change
\n\n\n\n\nEarly last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure.
\n
\nThis migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD.
\nCurrently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose.
\n\n\nWhile migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units.
\n
\n\n\nEver since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets.
\n
\n\n\nIt is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives.
\n
\nSo what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.)
##News Roundup
\n###What you need may be “pipeline +Unix commands” only
\n\n\n\n\nI came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved.
\n
\nThe IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.
\nBTW, if your data set can be disposed by an awk script, it should not be called “big data”.
###Running a bakery on Emacs and PostgreSQL
\n\n\n\n\nJust over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one.
\n
\nOne of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face of it, this is really simple, just work out how much dough you need, then work out what quantities to mix to make that much dough. Easy. You can do it with a pencil and paper. Or, in traditional bakers’ fashion, by scrawling with your finger on a floured work bench.
\nAnd that’s how I coped for a few weeks early on. But I kept making mistakes, which makes for an inconsistent product (bread is very forgiving, you have to work quite hard to make something that isn’t bread, but consistency matters). I needed to automate.
###The Ultimate Guide To Memorable Tech Talks
\n\n\n\n\nImagine this. You’re a woman in a male-dominated field. English is not your first language. Even though you’re confident in your engineering work, the thought of public speaking and being recorded for the world to see absolutely terrifies you.
\n
\nThat was me, five years ago. Since then, I’ve moved into a successful career in Developer Advocacy and spoken at dozens of technical events in the U.S. and worldwide.
\nI think everyone has the ability to deliver stellar conference talks, which is why I took the time to write this post.
###Light-weight Contexts: An OS Abstraction for Safety and Performance (2016)
\n\n\n\n\nAbstract: “We introduce a new OS abstraction—light-weight con-texts (lwCs)—that provides independent units of protection, privilege, and execution state within a process. A process may include several lwCs, each with possibly different views of memory, file descriptors, and access capabilities. lwCs can be used to efficiently implement roll-back (process can return to a prior recorded state),isolated address spaces (lwCs within the process may have different views of memory, e.g., isolating sensitive data from network-facing components or isolating different user sessions), and privilege separation (in-process reference monitors can arbitrate and control access).
\n
\nlwCs can be implemented efficiently: the overhead of a lwC is proportional to the amount of memory exclusive to the lwC; switching lwCs is quicker than switching kernel threads within the same process. We describe the lwC abstraction and API, and an implementation of lwCs within the FreeBSD 11.0 kernel. Finally, we present an evaluation of common usage patterns, including fast roll-back, session isolation, sensitive data isolation, and in-process reference monitoring, using Apache, nginx, PHP,and OpenSSL.”
##Beastie Bits
\n\n##Feedback/Questions
\n\nCharles - Volunteer work
\nJake - Bhyve Front Ends
\nWe’ve hit that point where we are running low on your questions, so if you have any questions rolling around in your head that you’ve not thought of to ask yet… send them in!
\nFreeBSD on Cavium ThunderX, looking at NetBSD as an OpenBSD user, taking time-stamped notes in vim, OpenBSD 6.5 has been tagged, FreeBSD and NetBSD in GSoC 2019, SecBSD: an UNIX-like OS for Hackers, and more.
\n\n##Headlines
\n###ARM’d and dangerous: FreeBSD on Cavium ThunderX (aarch64)
\n\n\nWhile I don’t remember for how many years I’ve had an interest in CPU architectures that could be an alternative to AMD64, I know pretty well when I started proposing to test 64-bit ARM at work. It was shortly after the disaster named Spectre / Meltdown that I first dug out server-class ARM hardware and asked whether we should get one such server and run some tests with it.
\n
\nWhile the answer wasn’t a clear “no” it also wasn’t exactly “yes”. I tried again a few times over the course of 2018 and each time I presented some more points why I thought it might be a good thing to test this. But still I wasn’t able to get a positive answer. Finally in January 2019 year I got a definitive answer – and it was “yes, go ahead”! The fact that Amazon had just presented their Graviton ARM Processor may have helped the decision.
###Looking at NetBSD from an OpenBSD user perspective
\n\n\n\n\nI use to use NetBSD quite a lot. From 2.0 to 6.99. But for some reasons, I stopped using it about 2012, in favor of OpenBSD. Reading on the new 8 release, I wanted to see if all the things I didn’t like on NetBSD were gone. Here is a personal Pros / Cons list. No Troll, hopefully. Just trying to be objective.
\n
\n\n\nSo that was it. I didn’t spend more than 30 minutes of it. But I didn’t want to spend more time on it. I did stop using NetBSD because of the need to compile each and every packages ; it was in the early days of pkgin. I also didn’t like the way system maintenance was to be done. OpenBSD’s 6-months release seemed far more easy to manage. I still think NetBSD is a great OS. But I believe you have to spent more time on it than you would have to do with OpenBSD.
\n
\nThat said, I’ll keep using my Puffy OS.
##News Roundup
\n###Using Vim to take time-stamped notes
\n\n\n\n\nI frequently find myself needing to take time-stamped notes. Specifically, I’ll be in a call, meeting, or interview and need to take notes that show how long it’s been since the meeting started.
\n
\nMy first thought was that there’s be a plugin to add time stamps, but a quick search didn’t turn anything up. However, I little digging did turn up the fact that vim has the built-in ability to tell time.
\nThis means that writing a bit of vimscript to insert a time stamp is pretty easy. After a bit of fiddling, I came up with something that serves my needs, and I decided it might be useful enough to others to be worth sharing.
###OpenBSD 6.5-beta has been tagged
\n\n\n\n\nIt’s that time of year again; Theo (deraadt@) has just tagged 6.5-beta. A good reminder for us all run an extra test install and see if your favorite port still works as you expect.
\n
CVSROOT: /cvs
\nModule name: src
\nChanges by: deraadt@cvs.openbsd.org 2019/02/26 15:24:41
\n
\nModified files:
\netc/root : root.mail
\nshare/mk : sys.mk
\nsys/conf : newvers.sh
\nsys/sys : ktrace.h param.h
\nusr.bin/signify: signify.1
\nsys/arch/macppc/stand/tbxidata: bsd.tbxi
\n
\nLog message:
\ncrank to 6.5-beta
\n
###The NetBSD Foundation participating in Google Summer of Code 2019
\n\n\n\n\nFor the 4th year in a row and for the 13th time The NetBSD Foundation will participate in Google Summer of Code 2019!
\n
\nIf you are a student and would like to learn more about Google Summer of Code please go to the Google Summer of Code homepage.
\nYou can find a list of projects in Google Summer of Code project proposals in the wiki.
\nDo not hesitate to get in touch with us via #netbsd-code IRC channel on Freenode and via NetBSD mailing lists!
###SecBSD: an UNIX-like OS for Hackers
\n\n\n\n\nSecBSD is an UNIX-like operating system focused on computer security based on OpenBSD. Designed for security testing, hacking and vulnerability assessment, it uses full disk encryption and ProtonVPN + OpenVPN by default.
\n
\nA security BSD enviroment for security researchers, penetration testers, bug hunters and cybersecurity experts. Developed by Dark Intelligence Team for private use and will be public release coming soon.
##Beastie Bits
\n\n##Feedback/Questions
\n\nA kernel of failure, IPv6 fragmentation vulnerability in OpenBSD’s pf, a guide to the terminal, using a Yubikey for SSH public key authentication, FreeBSD desktop series, and more.
\n\n##Headlines
\n\n\n\n\n\n\nToday in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created.
\n
###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter
\n\n\n\n\nPacket Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization.
\n
\nPacket Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris.
\n\n\nNote that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product.
\n
##News Roundup
\n###How I’m still not using GUIs in 2019: A guide to the terminal
\n\n\nTL;DR: Here are my dotfiles. Use them and have fun.
\n
\n\n\nGUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal.
\n
\nIDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better.
\nIn this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows.
\n\n\nWhenever in doubt, read the manual.
\n
###Using a Yubikey as smartcard for SSH public key authentication
\n\n\n\n\nSSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys.
\n
\nYou might have heard of Yubikeys.
\nThese are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard.
\nIn OpenBSD, you can use them for Login (with login_yubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up.
###The 18 Part FreeBSD Desktop Series by Vermaden
\n\n##Beastie Bits
\n\n##Feedback/Questions
\n\nSoftware will never fix Spectre-type bugs, a proof that sed is Turing complete, managed jails using Bastille, new version of netdata, using grep with /dev/null, using GMail with mutt, and more.
\n\n##Headlines
\n###Google: Software is never going to be able to fix Spectre-type bugs
\n\n\nResearchers from Google investigating the scope and impact of the Spectre attack have published a paper asserting that Spectre-like vulnerabilities are likely to be a continued feature of processors and, further, that software-based techniques for protecting against them will impose a high performance cost. And whatever the cost, the researchers continue, the software will be inadequate—some Spectre flaws don’t appear to have any effective software-based defense. As such, Spectre is going to be a continued feature of the computing landscape, with no straightforward resolution.
\n
\nThe discovery and development of the Meltdown and Spectre attacks was undoubtedly the big security story of 2018. First revealed last January, new variants and related discoveries were made throughout the rest of the year. Both attacks rely on discrepancies between the theoretical architectural behavior of a processor—the documented behavior that programmers depend on and write their programs against—and the real behavior of implementations.
\nSpecifically, modern processors all perform speculative execution; they make assumptions about, for example, a value being read from memory or whether an if condition is true or false, and they allow their execution to run ahead based on these assumptions. If the assumptions are correct, the speculated results are kept; if it isn’t, the speculated results are discarded and the processor redoes the calculation. Speculative execution is not an architectural feature of the processor; it’s a feature of implementations, and so it’s supposed to be entirely invisible to running programs. When the processor discards the bad speculation, it should be as if the speculation never even happened.
###A proof that Unix utility sed is Turing complete
\n\n\n\n\nMany people are surprised when they hear that sed is Turing complete. How come a text filtering program is Turing complete, they wonder. Turns out sed is a tiny assembly language that has a comparison operation, a branching operation and a temporary buffer. These operations make sed Turing complete.
\n
\nI first learned about this from Christophe Blaess. His proof is by construction – he wrote a Turing machine in sed (download turing.sed). As any programming language that can implement a Turing machine is Turing complete we must conclude that sed is also Turing complete.
\nChristophe offers his own introduction to Turing machines and a description of how his sed implementation works in his article Implementation of a Turing Machine as a sed Script.
\n\n\nChristophe isn’t the first person to realize that sed is almost a general purpose programming language. People have written tetris, sokoban and many other programs in sed. Take a look at these:
\n
##News Roundup
\n###Bastille helps you quickly create and manage FreeBSD Jails.
\n\n\nBastille helps you quickly create and manage FreeBSD Jails.
\n
\nJails are extremely lightweight containers that provide a full-featured UNIX-like operating system inside. These containers can be used for software development, rapid testing, and secure production Internet services.
\nBastille provides an interface to create, manage and destroy these secure virtualized environments.
\n\n\nNetdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.
\n
\nNetdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).
\nNetdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.
###Using grep with /dev/null, an old Unix trick
\n\n\n\n\nEvery so often I will find myself writing a grep invocation like this:
\n
find .... -exec grep <something> /dev/null '{}' '+'
\n\n\nThe peculiar presence of /dev/null here is an old Unix trick that is designed to force grep to always print out file names, even if your find only matches one file, by always insuring that grep has at least two files as arguments. You can wind up wanting to do the same thing with a direct use of grep if you’re not certain how many files your wildcard may match.
\n
\n\n\nI recently switched to using mutt for email and while setting up mutt to use imap is pretty straightforward, this tutorial will also document some advanced concepts such as encrypting your account password and sending emails from a different From address.
\n
\nThis tutorial assumes that you have some familiarity with using mutt and have installed it with sidebar support (sudo apt-get install mutt-patched for the ubuntu folks) and are comfortable with editing your muttrc.
\nIf you would just like to skip to the end, my mutt configuration file can be found here.
##Beastie Bits
\n\n##Feedback/Questions
\n\nDesign and Implementation of NetBSD’s rc.d system, first impressions of Project Trident 18.12, PXE booting a FreeBSD disk image, middle mouse button pasting, NetBSD gains hardware accelerated virtualization, and more.
\n\n##Headlines
\n###The Design and Implementation of the NetBSD rc.d system
\n\n\nIn this paper I cover the design and implementation of the rc.d system start-up mechanism in NetBSD 1.5, which replaced the monolithic /etc/rc start-up file inherited from 4.4BSD. Topics covered include a history of various UNIX start-up mechanisms (including NetBSD prior to 1.5), design considerations that evolved over six years of discussions, implementation details, an examination of the human issues that occurred during the design and implementation, as well as future directions for the system.
\n
\n\n\nNetBSD recently converted from the traditional 4.4BSD monolithic /etc/rc start-up script to an /etc/rc.d mechanism, where there is a separate script to manage each service or daemon, and these scripts are executed in a specific order at system boot.
\n
\nThis paper covers the motivation, design and implementation of the rc.d system; from the history of what NetBSD had before to the system that NetBSD 1.5 shipped with in December 2000, as well as future directions.
\nThe changes were contentious and generated some of the liveliest discussions about any feature change ever made in NetBSD. Parts of those discussions will be covered to provide insight into some of the design and implementation decisions.
\n\n\nThere is great diversity in the system start-up mechanisms used by various UNIX variants. A few of the more pertinent schemes are detailed below. As NetBSD is derived from 4.4BSD, it follows that a description of the latter’s method is relevant. Solaris’ start-up method is also detailed, as it is the most common System V UNIX variant.
\n
###First impressions of Project Trident 18.12
\n\n\n\n\nProject Trident (hereafter referred to as Trident) is a desktop operating system based on TrueOS. Trident takes the rolling base platform of TrueOS, which is in turn based on FreeBSD’s development branch, and combines it with the Lumina desktop environment.
\n
+Installing
\n\n\n\n\nThe debut release of Trident is available as a 4.1GB download that can be burned to a disc or transferred to a USB thumb drive. Booting from the Trident media brings up a graphical interface and automatically launches the project’s system installer. Down the left side of the display there are buttons we can click to show hardware information and configuration options. These buttons let us know if our wireless card and video card are compatible with Trident and give us a chance to change our preferred language and keyboard layout. At the bottom of the screen we find buttons that will open a terminal or shutdown the computer.
\n
\n\n\nTrident boots to a graphical login screen where we can sign into the Lumina desktop or a minimal Fluxbox session. Lumina, by default, uses Fluxbox as its window manager. The Lumina desktop places its panel along the bottom of the screen and an application menu sits in the bottom-left corner. On the desktop we find icons for opening the software manager, launching the Falkon web browser, running the VLC media player, opening the Control Panel and adjusting the Lumina theme.
\n
\nThe application menu has an unusual and compact layout. The menu shows just a search box and buttons for browsing applications, opening a file manager, accessing desktop settings and signing out. To see what applications are available we can click the Browse Applications entry, which opens a window in the menu where we can scroll through installed programs. This is a bit awkward since the display window is small and only shows a few items at a time.
\nEarly on I found it is possible to swap out the default “Start menu” with an alternative “Application menu” through the Panels configuration tool. This alternative menu offers a classic tree-style application menu. I found the latter menu easier to navigate as it expands to show all the applications in a selected category.
\n\n\nI have a lot of mixed feelings and impressions when it comes to Trident. On the one hand, the operating system has some great technology under the hook. It has cutting edge packages from the FreeBSD ecosystem, we have easy access to ZFS, boot environments, and lots of open source packages. Hardware support, at least on my physical workstation, was solid and the Lumina desktop is flexible.
\n
##News Roundup
\n###PXE booting of a FreeBSD disk image
\n\n\nI had to set up a regression and network performance lab. This lab will be managed by a Jenkins, but the first step is to understand how to boot a FreeBSD disk by PXE. This article explains a simple way of doing it.
\n
\nFor information, all these steps were done using 2 PC Engines APU2 (upgraded with latest BIOS for iPXE support), so it’s a headless (serial port only, this can be IPMI SoL with different hardware) .
\n\n\nBefore explaining all steps and command line, here is the full big picture of the final process.
\n
###Why I like middle mouse button paste in xterm so much
\n\n\n\n\nIn my entry about how touchpads are not mice, I mused that one of the things I should do on my laptop was insure that I had a keyboard binding for paste, since middle mouse button is one of the harder multi-finger gestures to land on a touchpad. Kurt Mosiejczuk recently left a comment there where they said:
\n
\nShift-Insert is a keyboard equivalent for paste that is in default xterm (at least OpenBSD xterm, and putty on Windows too). I use that most of the time now as it seems less… trigger-happy than right click paste.
\nThis sparked some thoughts, because I can’t imagine giving up middle mouse paste if I have a real choice. I had earlier seen shift-insert mentioned in other commentary on my entry and so have tried a bit to use it on my laptop, and it hasn’t really felt great even there; on my desktops, it’s even less appealing (I tried shift-insert out there to confirm that it did work in my set of wacky X resources).
\nIn thinking about why this is, I came to the obvious realization about why all of this is so. I like middle mouse button paste in normal usage because it’s so convenient, because almost all of the time my hand is already on the mouse. And the reason my hand is already on the mouse is because I’ve just used the mouse to shift focus to the window I want to paste into. Even on my laptop, my right hand is usually away from the keyboard as I move the mouse pointer on the touchpad, making shift-Insert at least somewhat awkward.
###NetBSD Gains Hardware Accelerated Virtualization
\n\n\n\n\nNVMM provides hardware-accelerated virtualization support for NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is shipped via libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.
\n
##Beastie Bits
\n\n##Feedback/Questions
\n\nAdding glue to a desktop environment, flashing the BIOS on a PC Engine, revive a Cisco IDS into a capable OpenBSD computer, An OpenBSD WindowMaker desktop, RealTime data compression, the love for pipes, and more.
\n\n##Headlines
\n###Adding Glue To a Desktop Environment
\n\n\nIn this article we will put some light on a lot of tools used in the world of Unix desktop environment customization, particularly regarding wmctrl, wmutils, xev, xtruss, xwininfo, xprop, xdotools, xdo, sxhkd, xbindkeys, speckeysd, xchainkeys, alttab, triggerhappy, gTile, gidmgr, keynav, and more. If those don’t make sense then this article will help. Let’s hope this can open your mind to new possibilities.
\n
\nWith that in mind we can wonder if what’s actually needed from a window manager, presentation and operation, can be split up and complemented with other tools. We can also start thinking laterally, the communication and interaction between the different components of the environment. We have the freedom to do so because the X protocol is transparent and components usually implement many standards for interfacing between windows. It’s like gluing parts together to create a desktop environment.
###Flashing the BIOS on the PC Engines APU4c4
\n\n\n\n\nI absolutely love the PC Engines APU devices. I use them for testing HardenedBSD experimental features in more constrained 64-bit environments and firewalls. Their USB and mSATA ports have a few quirks, and I bumped up against a major quirk that required flashing a different BIOS as a workaround. This article details the hacky way in which I went about doing that.
\n
\nWhat prompted this article is that something in either the CAM or GEOM layer in FreeBSD 11.2 caused the mSATA to hang, preventing file writes. OPNsense 18.7 uses FreeBSD 11.1 whereas the recently-released OPNsense 19.1 uses HardenedBSD 11.2 (based on FreeBSD 11.2). I reached out to PC Engines directly, and they let me know that the issue is a known BIOS issue. Flashing the “legacy” BIOS series would provide me with a working system.
\nIt also just so happens that a new “legacy” BIOS version was just released which turns on ECC mode for the RAM. So, I get a working OPNsense install AND ECC RAM! I’ll have one bird for dinner, the other for dessert.
\nThough I’m using an APU4, these instructions should work for the other APU devices. The BIOS ROM download URLs should be changed to reflect the device you’re targeting along with the BIOS version you wish to deploy.
\nSPECIAL NOTE: There be dragons! I’m primarily writing this article to document the procedure for my own purposes. My memory tends to be pretty faulty these days. So, if something goes wrong, please do not hold me responsible. You’re the one at the keyboard. ;)
\nVERY SPECIAL NOTE: We’ll use the mSATA drive for swap space, just in case. Should the swap space be used, it will destroy whatever is on the disk.
##News Roundup
\n###Revive a Cisco IDS into a capable OpenBSD computer!
\n\n\nEven though Cisco equipment is very capable, it tends to become End-of-Life before you can say “planned obsolescence”. Websites become bigger, bandwidths increase, and as a side effect of those “improvements”, routers, firewalls, and in this case, intrusion prevention systems get old quicker and quicker.
\n
\nApparently, this was also the case for the Cisco IDS-4215 Intrusion Detection Sensor that I was given a few months ago.
\nI’m not too proud to admit that at first, I didn’t care about the machine itself, but rather about the add-on PCI network card with 4 Fast Ethernet interfaces. The sensor has obviously seen better days, as it had a broken front panel and needed some cleaning, but upon a closer inspection under the hood (which is held closed by the 4 screws on top), this IDS consists of an embedded Celeron PC with two onboard Ethernet cards, a 2.5″ IDE hard disk, a CF card, and 2 PCI expansion slots (more on them later). Oh, and don’t forget the nasty server-grade fan, which pushed very little air for the noise it was making.
###An OpenBSD desktop using WindowMaker
\n\n\n\n\nSince I started using *N?X, I’ve regularly used WindowMaker. I’ve always liked the look and feel, the dock system and the dockapps. It may look a bit oldish nowadays. And that’s enough to try to change this. So here it is, a 2019 flavored WindowMaker Desktop, running on OpenBSD 6.4/amd64.
\n
\nThis configuration uses the Nord color-scheme, the Adapta-Nokto-Eta GTK theme and the Moblin Unofficial Icons icon set. I did remove applications icons. I just don’t need them on the bottom of the screen as I heavily use “F11” to pop-up the windows list. To be able to do that and keep the dockapps, I tweaked my ~/GNUstep/Defaults/WMWindowAttributes and created a ~/GNUstep/Library/WindowMaker/Themes/Nord.themed/style.
\nAnd here it is, the NeXT OpenBSD Desktop!
\n\n\nIn a previous episode, we’ve seen that it is possible to create opaque types. However, creation and destruction of such type must be delegated to some dedicated functions, which themselves rely on dynamic allocation mechanisms.
\n
\nSometimes, it can be convenient to bypass the heap, and all its malloc() / free() shenanigans. Pushing a structure onto the stack, or within thread-local storage, are natural capabilities offered by a normal struct. It can be desirable at times.
\nThe previously described opaque type is so secret that it has no size, hence is not suitable for such scenario.
\nFortunately, static opaque types are possible.
\nThe main idea is to create a “shell type”, with a known size and an alignment, able to host the target (private) structure.
\nFor safer maintenance, the shell type and the target structure must be kept in sync, by using typically a static assert. It will ensure that the shell type is always large enough to host the target structure. This check is important to automatically detect future evolution of the target structure.
\n\n\nMy top used shell command is |. This is called a pipe.
\n
\nIn brief, the | allows for the output of one program (on the left) to become the input of another program (on the right). It is a way of connecting two commands together.
\nAccording to doc.cat-v.org/unix/pipes/, the origin of pipes came long before Unix. Pipes can be traced back to this note from Doug McIlroy in 1964
##Beastie Bits
\n\n##BUG Calendar
\n\n##Feedback/Questions
\n\nStrategic thinking to keep FreeBSD relevant, reflecting on the soul of a new machine, 10GbE Benchmarks On Nine Linux Distros and FreeBSD, NetBSD integrating LLVM sanitizers in base, FreeNAS 11.2 distrowatch review, and more.
\n\n##Headlines
\n###Strategic thinking, or what I think what we need to do to keep FreeBSD relevant
\n\n\nSince I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer.
\n
\nSometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay where we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it.
\nThis discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case.
\nFreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end‐users as a work‐horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web‐/mail‐/whatever‐server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution.
\nI have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question.
###Reflecting on The Soul of a New Machine
\n\n\n\n\nLong ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark.
\n
\nSince reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation.
\nExcited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are singular: the saga of engineers debugging a nasty I-cache data corruption issue; the young engineer who implements the simulator in an impossibly short amount of time because no one wanted to tell him that he was being impossibly ambitious; the engineer who, frustrated with a nanosecond-scale timing problem in the ALU that he designed, moved to a commune in Vermont, claiming a desire to deal with “no unit of time shorter than a season”. But by limiting myself to these passages, I was succumbing to the selection bias of my much younger self; re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate — from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch — are now deeply evocative of concrete episodes of my own career.
##News Roundup
\n\n###Out-Of-The-Box 10GbE Network Benchmarks On Nine Linux Distributions Plus FreeBSD 12
\n\n\n\n\nLast week I started running some fresh 10GbE Linux networking performance benchmarks across a few different Linux distributions. That testing has now been extended to cover nine Linux distributions plus FreeBSD 12.0 to compare the out-of-the-box networking performance.
\n
\nTested this round alongside FreeBSD 12.0 was Antergos 19.1, CentOS 7, Clear Linux, Debian 9.6, Fedora Server 29, openSUSE Leap 15.0, openSUSE Tumbleweed, Ubuntu 18.04.1 LTS, and Ubuntu 18.10.
\nAll of the tests were done with a Tyan S7106 1U server featuring two Intel Xeon Gold 6138 CPUs, 96GB of DDR4 system memory, and Samsung 970 EVO SSD. For the 10GbE connectivity on this server was an add-in HP NC523SFP PCIe adapter providing two 10Gb SPF+ ports using a QLogic 8214 controller.
\nOriginally the plan as well was to include Windows Server 2016/2019. Unfortunately the QLogic driver download site was malfunctioning since Cavium’s acquisition of the company and the other Windows Server 2016 driver options not panning out and there not being a Windows Server 2019 option. So sadly that Windows testing was thwarted so I since started testing over with a Mellanox Connectx-2 10GbE NIC, which is well supported on Windows Server and so that testing is ongoing for the next article of Windows vs. Linux 10 Gigabit network performance plus some “tuned” Linux networking results too.
###Integration of the LLVM sanitizers with the NetBSD base system
\n\n\n\n\nOver the past month I’ve merged the LLVM compiler-rt sanitizers (LLVM svn r350590) with the base system. I’ve also managed to get a functional set of Makefile rules to build all of them, namely:
\n
\nASan, UBSan, TSan, MSan, libFuzzer, SafeStack, XRay.
\nIn all supported variations and modes that are supported by the original LLVM compiler-rt package.
###Distrowatch FreeNAS 11.2 review
\n\n\n\n\nThe project’s latest release is FreeNAS 11.2 and, at first, I nearly overlooked the new version because it appeared to be a minor point release. However, a lot of work went into the new version and 11.2 offers a lot of changes when compared next to 11.1, “including a major revamp of the web interface, support for self-encrypting drives, and new, backwards-compatible REST and WebSocket APIs. This update also introduces iocage for improved plugins and jails management and simplified plugin development.”
\n
##Beastie Bits
\n\n##Feedback/Questions
\n\nWe recap FOSDEM 2019, FreeBSD Foundation January update, OPNsense 19.1 released, the hardware-assisted virtualization challenge, ZFS and GPL terror, ClonOS 19.01-RELEASE, and more.
\n\n\n\n\nDear FreeBSD Community Member,
\n
\nHappy New Year! It’s always exciting starting the new year with ambitious plans to support FreeBSD in new and existing areas. We achieved our fundraising goal for 2018, so we plan on funding a lot of work this year! Though it’s the new year, this newsletter highlights some of the work we accomplished in December. We also put together a list of technologies and features we are considering supporting, and are looking for feedback on what users want to help inform our 2019 development plans. Our advocacy and education efforts are in full swing as we prepare for upcoming conferences including FOSDEM, SANOG33, and SCaLE.
\nFinally, we created a year-end video to talk about the work we did in 2018. That in itself was an endeavor, so please take a few minutes to watch it! We’re working on improving the methods we use to inform the community on the work we are doing to support the Project, and are always open to feedback. Now, sit back, grab a refreshing beverage, and enjoy our newsletter!
\nHappy reading!!
\nDeb
\n\n\nFor more than four years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
\n
\nThe 19.1 release, nicknamed “Inspiring Iguana”, consists of a total of 620 individual changes since 18.7 came out 6 months ago, spread out over 12 intermediate releases including the recent release candidates. That is the average of 2 stable releases per month, security updates and important bug fixes included! If we had to pick a few highlights it would be: The firewall alias API is finally in place. The migration to HardenedBSD 11.2 has been completed. 2FA now works with a remote LDAP / local TOTP combination. And the OpenVPN client export was rewritten for full API support as well.
These are the most prominent changes since version 18.7:
\nfully functional firewall alias API
\nPIE firewall shaper support
\nfirewall NAT rule logging support
\n2FA via LDAP-TOTP combination
\nWPAD / PAC and parent proxy support in the web proxy
\nP12 certificate export with custom passwords
\nDpinger is now the default gateway monitor
\nET Pro Telemetry edition plugin[2]
\nextended IPv6 DUID support
\nDnsmasq DNSSEC support
\nOpenVPN client export API
\nRealtek NIC driver version 1.95
\nHardenedBSD 11.2, LibreSSL 2.7
\nUnbound 1.8, Suricata 4.1
\nPhalcon 3.4, Perl 5.28
\nfirmware health check extended to cover all OS files, HTTPS mirror default
\nupdates are browser cache-safe regarding CSS and JavaScript assets
\ncollapsible side bar menu in the default theme
\nlanguage updates for Chinese, Czech, French, German, Japanese, Portuguese and Russian
\nAPI backup export, Bind, Hardware widget, Nginx, Ntopng, VnStat and Dnscrypt-proxy plugins
\nHere are the full changes against version 19.1-RC2:
\nipsec: add firewall interface as soon as phase 1 is enabled
\nipsec: phase 1 selection GUI JavaScript compatibility fix
\nmonit: widget improvements and bug fix (contributed by Frank Brendel)
\nui: fix regression in single host or network subnet select in static pages
\nplugins: os-frr 1.7 updates OSFP outbound rules (contributed by Fabian Franz)
\nplugins: os-telegraf 1.7.4 fixes packet filter input
\nplugins: os-theme-rebellion 1.8.2 adds image colour invert
\nplugins: os-vnstat 1.1[3]
\nplugins: os-zabbix-agent now uses Zabbix version 4.0
\nsrc: revert mmc_calculate_clock() as HS200/HS400 support breaks legacy support
\nsrc: update sqlite3-3.20.0 to sqlite3-3.26.0[4]
\nsrc: import tzdata 2018h, 2018i[5]
\nsrc: avoid unsynchronized updates to kn_status[6]
\nports: ca_root_nss 3.42
\nports: dhcp6c 20190128 prevent rawops double-free (contributed by Team Rebellion)
\nports: sudo patch to fix listpw=never[7]
\n\n\n\nOver two years ago, I made a pledge to use NetBSD as my sole OS and only operating system, and to resist booting into any other OS until I had implemented hardware-accelerated virtualization in the NetBSD kernel (the equivalent of Linux’ KVM, or Hyper-V).
\n
\nToday, I am here to report: Mission Accomplished!
\nIt’s been a long road, but we now have hardware-accelerated virtualization in the kernel! And while I had only initially planned to get Oracle VirtualBox working, I have with the help of the Intel HAXM engine (the same backend used for virtualization in Android Studio) and a qemu frontend, successfully managed to boot a range of mainstream operating systems.
\n\n\nZFS is todays most advanced filesystem. It originated on the Solaris operating system and thanks to Sun’s decision to open it up, we have it available on quite a number of Unix-like operating systems. That’s just great! Great for everyone.
\n
\nFor everyone? Nope. There are people out there who don’t like ZFS. Which is totally fine, they don’t need to use it after all. But worse: There are people who actively hate ZFS and think that others should not use it. Ok, it’s nothing new that some random guys on the net are acting like assholes, trying to tell you what you must not do, right? Whoever has been online for more than a couple of days probably already got used to it. Unfortunately its still worse: One such spoilsport is Greg Kroah-Hartman, Linux guru and informal second-in-command after Linus Torvalds.
\nThere have been some attempts to defend the stance of this kernel developer. One was to point at the fact that the “ZFS on Linux” (ZoL) port uses two kernel functions, __kernel_fpu_begin() and __kernel_fpu_end(), which have been deprecated for a very long time and that it makes sense to finally get rid of them since nothing in-kernel uses it anymore. Nobody is going to argue against that. The problem becomes clear by looking at the bigger picture, though:
\nThe need for functions doing just what the old ones did has of course not vanished. The functions have been replaced with other ones. And those ones are deliberately made GPL-only. Yes, that’s right: There’s no technical reason whatsoever! It’s purely ideology – and it’s a terrible one.
\n\n\nClonOS is a turnkey Open Source platform based on FreeBSD and the CBSD framework. ClonOS offers a complete web UI for easily controlling, deploying and managing FreeBSD jails containers and Bhyve/Xen hyperviser virtual environments.
\n
\nClonOS is currently the only platform available which allow both Xen and Bhyve hypervisor to coexist on the same host. Being a FreeBSD base platform, ClonOS ability to create and manage jails allows you to run FreeBSD applications without losing performance.
Features:
\neasy management via web UI interface
\nlive Bhyve migration [coming soon, roadmap]
\nBhyve management (create, delete VM)
\nXen management (create, delete VM) [coming soon, roadmap]
\nconnection to the “physical” guest console via VNC from the browser or directly
\nReal time system monitoring
\naccess to load statistics through SQLite3 and beanstalkd
\nsupport for ZFS features (cloning, snapshots)
\nimport/export of virtual environments
\npublic repository with virtual machine templates
\npuppet-based helpers for configuring popular services
\nClonOS is a free open-source FreeBSD-based platform for virtual environments creation and management. In the core:
\nFreeBSD OS as hoster platform
\nbhyve(8) as hypervisor engine
\nXen as hypervisor engine
\nvale(4) as Virtual Ethernet Switch
\njail(8) as container engine
\nCBSD Project as management tools
\nPuppet as configuration management
\nWe’re at FOSDEM 2019 this week having fun. We’d never leave you in a lurch, so we have recorded an interview with Niclas Zeising of the FreeBSD graphics team for you. Enjoy.
\n\n##Interview - Niclas Zeising - zeising@FreeBSD.org / @niclaszeising
\nInterview topic: FreeBSD Graphics Stack
##Feedback/Questions
\n\nProject Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more.
\n\n##Headlines
\n\n###AsiaBSDCon 2019 Call for Papers
\n\n###Project Trident 18.12 Released
\n\n###Building Spotifyd on NetBSD
\n\n\n\n\nThese are the steps I went through to build and run Spotifyd (this commit at the time of writing) on NetBSD AMD64. It’s a Spotify Connect client so it means I still need to control Spotify from another device (typically my phone), but the audio is played through my desktop… which is where my speakers and headphones are plugged in - it means I don’t have to unplug stuff and re-plug into my phone, work laptop, etc. This is 100% a “good enough for now solution” for me; I have had a quick play with the Go based microcontroller from spotcontrol and that allows a completely NetBSD only experience (although it is just an example application so doesn’t provide many features - great as a basis to build on though).
\n
##News Roundup
\n\n\n\n\n\n\n2019 means 19.1 is almost here. In the meantime accept this small
\n
\nincremental update with goodies such as Suricata 4.1, custom passwords
\nfor P12 certificate export as well as fresh fixes in the FreeBSD base.
\nA lot of cleanups went into this update to make sure there will be a
\nsmooth transition to 19.1-RC for you early birds. We expect RC1 in 1-2
\nweeks and the final 19.1 on January 29.
###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation
\n\n\n\n\nA few weeks ago, I got an itch to build a workstation with AMD EPYC. There are a few constraints. First, I needed a higher-clock part. Second, I knew the whole build would be focused more on being an ultra high-end workstation rather than simply utilizing gaming components. With that, I decided it was time to hit on a bit of nostalgia for our readers. Mainly, I wanted to do an homage to Sun Microsystems. Sun made the server gear that the industry ran on for years, and as a fun fact, if you go behind the 1 Hacker Way sign at Facebook’s campus, they left the Sun Microsystems logo. Seeing that made me wonder if we could do an ultimate AMD EPYC build in a Sun Microsystems workstation.
\n
###OpenRsync
\n\n\n\n\nThis is a clean-room implementation of rsync with a BSD (ISC) license. It is designed to be compatible with a modern rsync (3.1.3 is used for testing). It currently compiles and runs only on OpenBSD.
\n
\nThis project is still very new and very fast-moving.
\nIt’s not ready for wide-spread testing. Or even narrow-spread beyond getting all of the bits to work. It’s not ready for strong attention. Or really any attention but by careful programming.
\nMany have asked about portability. We’re just not there yet, folks. But don’t worry, the system is easily portable. The hard part for porters is matching OpenBSD’s pledge and unveil.
###The first report on LLD porting
\n\n\n\n\nLLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default).
\n
\nThe first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld.
\nIn this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD.
\n\n\nIt’s the second week of 2019 already, which means I’m curious what Nate is going to do with his series This week in usability … reset the numbering from week 1? That series is a great read, to keep up with all the little things that change in KDE source each week — aside from the release notes.
\n
\nFor the big ticket items of KDE on FreeBSD, you should read this blog instead.
##Beastie Bits
\n\n##Feedback/Questions
\n\nSCP client vulnerabilities, BSDs vs Linux benchmarks on a Tyan EPYC Server, fame for the Unix inventors, Die IPv4, GhostBSD 18.12 released, Unix in pictures, and more.
\n\n##Headlines
\n###scp client multiple vulnerabilities
\n\n\nThe discovered vulnerabilities, described in more detail below, enables the attack
\n
\ndescribed here in brief.
user@local:~$ scp user@remote:readme.txt .
\nreadme.txt 100% 494 1.6KB/s 00:00
\nuser@local:~$
###FreeBSD 12.0 vs. DragonFlyBSD 5.4 vs. TrueOS 18.12 vs. Linux On A Tyan EPYC Server
\n\n\n\n\nLast month when running FreeBSD 12.0 benchmarks on a 2P EPYC server I wasn’t able to run any side-by-side benchmarks with the new DragonFlyBSD 5.4 as this BSD was crashing during the boot process on that board. But fortunately on another AMD EPYC server available, the EPYC 1P TYAN Transport SX TN70A-B8026, DragonFlyBSD 5.4.1 runs fine. So for this first round of BSD benchmarking in 2019 are tests of FreeBSD 11.2, FreeBSD 12.0, DragonFlyBSD 5.4.1, the new TrueOS 18.12, and a few Linux distributions (CentOS 7, Ubuntu 18.04.1 LTS, and Clear Linux) on this EPYC 7601 server in a variety of workloads.
\n
\n\n\nDragonFlyBSD 5.4.1 ran fine on this Tyan server and could boot fine unlike the issue encountered on the Dell PowerEdge R7425 for this particular BSD. But on the Tyan server, DragonFlyBSD 5.2.2 wouldn’t boot so only this latest DragonFlyBSD release series was used as part of the comparison.
\n
A summary of the operating systems tested for this EPYC 7601 OS benchmark comparison included:
\nDragonFlyBSD 5.4.1 - The latest release of Matthew Dillon’s operating system while using the HAMMER2 file-system and GCC 8.1 compiler that is now the default system compiler for this BSD.
\nFreeBSD 11.2 - The previous stable release of FreeBSD. Installed with a ZFS file-system.
\nFreeBSD 12.0 - The latest stable release of FreeBSD and installed with its ZFS option.
\nTrueOS 18.12 - The latest release of the iX systems’ FreeBSD derivative. TrueOS 18.12 is based on FreeBSD 13.0-CURRENT and uses ZFS by default and was using the Clang 7.0.1 compiler compared to Clang 6.0.1 on FreeBSD 12.0.
\nCentOS Linux 7 - The latest EL7 operating system performance.
\nUbuntu 18.04.1 LTS - The latest Ubuntu Long Term Support release.
\nClear Linux 27120 - The latest rolling release as of testing out of Intel’s Open-Source Technology Center. Clear Linux often reflects as close to the gold standard for performance as possible with its insanely tuned software stack for offering optimal performance on x86_64 performance for generally showing best what the hardware is capable of.
\n\n\n\nThroughout all of this testing, the Tyan 2U server was kept to its same configuration of an AMD EPYC 7601 (32 cores / 64 threads) at stock speeds, 8 x 16GB DDR4-2666 ECC memory, and 280GB Intel Optane 900p SSD benchmarks.
\n
##News Roundup
\n###National Inventors Hall of Fame honors creators of Unix
\n\n\nDennis Ritchie (Posthumous) and Ken Thompson: UNIX Operating System
\n
\nThompson and Ritchie’s creation of the UNIX operating system and the C programming language were pivotal developments in the progress of computer science. Today, 50 years after its beginnings, UNIX and UNIX-like systems continue to run machinery from supercomputers to smartphones. The UNIX operating system remains the basis of much of the world’s computing infrastructure, and C language – written to simplify the development of UNIX – is one of the most widely used languages today.
\n\n\nImagine, it is 2019. Easy, ha? Imagine, it is 2019 and you want to turn off IPv4. Like, off off. Really off. Not disabling IPv6, but disabling IPv4.
\n
\n\n\nYou might be coming here wondering, why would anybody want to do what we are asking to be done. Well, it is dead simple: We are running data centers (like Data Center Light) with a lot of IPv6 only equipment. There simply is no need for IPv4. So why would we want to have it enabled?
\n
\nAlso, here at ungleich, we defined 2019 as the year to move away from IPv4.
\n\n\nDo you like puzzles? Competitions? Challenges? Hacking? Well. If ANY of this is of your interest, here is a real challenge for you:
\n
\nWe offer a 100 CHF (roughly 100 USD) for anyone who can give us a detailed description of how to turn IPv4 completely off in an operating system and allowing it to communicate with IPv6 only. This should obviously include a tiny proof that your operating system is really unable to use IPv4 at all. Just flushing IPv4 addresses and keeping the IPv4 stack loaded, does not count.
\n\n\nGhostBSD 18.12 is an updated iso of GhostBSD 18.10 with some little changes to the live DVD/USB and with updated packages.
\n
###And Now for a laugh : #unixinpictures
\n\n##Beastie Bits
\n\n##Feedback/Questions
\n\nA EULA in FOSS clothing, NetBSD with more LLVM support, Thoughts on FreeBSD 12.0, FreeBSD Performance against Windows and Linux on Xeon, Microsoft shipping NetBSD, and more.
\n\nThere was a tremendous amount of reaction to and discussion about my blog entry on the midlife crisis in open source. As part of this discussion on HN, Jay Kreps of Confluent took the time to write a detailed response — which he shortly thereafter elevated into a blog entry.\n\n
Let me be clear that I hold Jay in high regard, as both a software engineer and an entrepreneur — and I appreciate the time he took to write a thoughtful response. That said, there are aspects of his response that I found troubling enough to closely re-read the Confluent Community License — and that in turn has led me to a deeply disturbing realization about what is potentially going on here.\n\n
To GitHub: Assuming that this is in fact a EULA, I think it is perilous to allow EULAs to sit in public repositories. It’s one thing to have one click through to accept a license (though again, that itself is dubious), but to say that a git clone is an implicit acceptance of a contract that happens to be sitting somewhere in the repository beggars belief. With efforts like choosealicense.com, GitHub has been a model in guiding projects with respect to licensing; it would be helpful for GitHub’s counsel to weigh in on their view of this new strain of source-available proprietary software and the degree to which it comes into conflict with GitHub’s own terms of service.\n\n
To foundations concerned with software liberties, including the Apache Foundation, the Linux Foundation, the Free Software Foundation, the Electronic Frontier Foundation, the Open Source Initiative, and the Software Freedom Conservancy: the open source community needs your legal review on this! I don’t think I’m being too alarmist when I say that this is potentially a dangerous new precedent being set; it would be very helpful to have your lawyers offer their perspectives on this, even if they disagree with one another. We seem to be in some terrible new era of frankenlicenses, where the worst of proprietary licenses are bolted on to the goodwill created by open source licenses; we need your legal voices before these creatures destroy the village!\n\n
NetBSD entering 2019 with more complete LLVM support
\n\nI’m recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. As you can read in my previous report, I’ve been focusing on fixing build and test failures for the purpose of improving the buildbot coverage.\nPreviously, I’ve resolved test failures in LLVM, Clang, LLD, libunwind, openmp and partially libc++. During the remainder of the month, I’ve been working on the remaining libc++ test failures, improving the NetBSD clang driver and helping Kamil Rytarowski with compiler-rt.\n\n
The process of upstreaming support to LLVM sanitizers has been finalized
\n\nI’ve finished the process of upstreaming patches to LLVM sanitizers (almost 2000LOC of local code) and submitted to upstream new improvements for the NetBSD support. Today out of the box (in unpatched version) we have support for a variety of compiler-rt LLVM features: ASan (finds unauthorized memory access), UBSan (finds unspecified code semantics), TSan (finds threading bugs), MSan (finds uninitialized memory use), SafeStack (double stack hardening), Profile (code coverage), XRay (dynamic code tracing); while other ones such as Scudo (hardened allocator) or DFSan (generic data flow sanitizer) are not far away from completeness.\nThe NetBSD support is no longer visibly lacking behind Linux in sanitizers, although there are still failing tests on NetBSD that are not observed on Linux. On the other hand there are features working on NetBSD that are not functional on Linux, like sanitizing programs during early initialization process of OS (this is caused by /proc dependency on Linux that is mounted by startup programs, while NetBSD relies on sysctl(3) interfaces that is always available).\n\n
Playing with FreeBSD with past week I don’t feel as though there were any big surprises or changes in this release compared to FreeBSD 11. In typical FreeBSD fashion, progress tends to be evolutionary rather than revolutionary, and this release feels like a polished and improved incremental step forward. I like that the installer handles both UFS and ZFS guided partitioning now and in a friendly manner. In the past I had trouble getting FreeBSD’s boot menu to work with boot environments, but that has been fixed for this release.\nI like the security options in the installer too. These are not new, but I think worth mentioning. FreeBSD, unlike most Linux distributions, offers several low-level security options (like hiding other users’ processes and randomizing PIDs) and I like having these presented at install time. It’s harder for people to attack what they cannot see, or predict, and FreeBSD optionally makes these little adjustment for us.\nSomething which stands out about FreeBSD, compared to most Linux distributions I run, is that FreeBSD rarely holds the user’s hand, but also rarely surprises the user. This means there is more reading to do up front and new users may struggle to get used to editing configuration files in a text editor. But FreeBSD rarely does anything unless told to do it. Updates rarely change the system’s behaviour, working technology rarely gets swapped out for something new, the system and its applications never crashed during my trial. Everything was rock solid. The operating system may seem like a minimal, blank slate to new users, but it’s wonderfully dependable and predictable in my experience.\nI probably wouldn’t recommend FreeBSD for desktop use. It’s close relative, GhostBSD, ships with a friendly desktop and does special work to make end user applications run smoothly. But for people who want to run servers, possible for years without change or issues, FreeBSD is a great option. It’s also an attractive choice, in my opinion, for people who like to build their system from the ground up, like you would with Debian’s server install or Arch Linux. Apart from the base tools and documentation, there is nothing on a FreeBSD system apart from what we put on it.\n\n
Last week I posted benchmarks of Windows Server 2019 against various Linux distributions using a Tyan dual socket Intel Xeon server. In this article are some complementary results when adding in the performance of FreeBSD 11.2 against the new FreeBSD 12.0 stable release for this leading BSD operating system. As some fun benchmarks to end out 2018, here are the results of FreeBSD 11.2/12.0 (including an additional run when using GCC rather than Clang) up against Windows Server and several enterprise-ready Linux distributions.\nWhile FreeBSD 12.0 had picked up just one win of the Windows/Linux comparisons run, the FreeBSD performance is moving in the right direction. FreeBSD 12.0 was certainly faster than FreeBSD 11.2 on this dual Intel Xeon Scalable server based on a Tyan 1U platform. Meanwhile, to no surprise given the data last week, Clear Linux was by far the fastest out-of-the-box operating system tested.\nI did run some extra benchmarks on FreeBSD 11.2/12.0 with this hardware: in total I ran 120 benchmarks for these BSD tests. Of the 120 tests, there were just 15 cases where FreeBSD 11.2 was faster than 12.0. Seeing FreeBSD 12.0 faster than 11.2 nearly 90% of the time is an accomplishment and usually with other operating systems we see more of a mixed bag on new releases with not such solidly better performance. It was also great seeing the competitive performance out of FreeBSD when using the Clang compiler for the source-based tests compared to the GCC8 performance. Additional data available via this OpenBenchmarking.org result file.\n\n
Google cache in case the site is down
\n\nIn 2000, Joe Britt, Matt Hershenson and Andy Rubin formed Danger Incorporated. Danger developed the world’s first recognizable smartphone, the Danger HipTop. T-Mobile sold the first HipTop under the brand name Sidekick in October of 2002.\nDanger had a well developed kernel that had been designed and built in house. The kernel came to be viewed as not a core intellectual property and Danger started a search for a replacement. For business reasons, mostly to do with legal concerns over the Gnu Public License, Danger rejected Linux and began to consider BSD Unix as a replacement for the kernel.\nIn 2006 I was hired by Mike Chen, the manager of the kernel development group to investigate the feasibility of replacing the Danger kernel with a BSD kernel, to select the version of BSD to use, to develop a prototype and to develop the plan for adapting BSD to Danger’s requirements.\nNetBSD was easily the best choice among the BSD variations at the time because it had well developed cross development tools. It was easy to use a NetBSD desktop running an Intel release to cross compile a NetBSD kernel and runtime for a device running an ARM processor. (Those interested in mailing list archaeology might be amused to investigate NetBSD technical mailing list for mail from picovex, particularly from Bucky Katz at picovex.)\nWe began product development on the specific prototype of the phone that would become the Sidekick LX2009 in 2007 and contracts for the phone were written with T-Mobile. We were about half way through the two year development cycle when Microsoft purchased Danger in 2008.\nMicrosoft would have preferred to ship the Sidekick running Windows/CE rather than NetBSD, but a schedule analysis performed by me, and another by an independent outside contractor, indicated that doing so would result in unacceptable delay.\n\n
The future of ZFS in FreeBSD, we pick highlights from the FreeBSD quarterly status report, flying with the raven, modern KDE on FreeBSD, many ways to launch FreeBSD in EC2, GOG installers on NetBSD, and more.
\n\nThe sources for FreeBSD’s ZFS support are currently taken directly from Illumos with local ifdefs to support the peculiarities of FreeBSD where the Solaris Portability Layer (SPL) shims fall short. FreeBSD has regularly pulled changes from Illumos and tried to push back any bug fixes and new features done in the context of FreeBSD. In the past few years the vast majority of new development in ZFS has taken place in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced that they will be moving to ZoL: https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means that there will be little to no net new development of Illumos. While working through the git history of ZoL I have also discovered that many races and locking bugs have been fixed in ZoL and never made it back to Illumos and thus FreeBSD. This state of affairs has led to a general agreement among the stakeholders that I have spoken to that it makes sense to rebase FreeBSD’s ZFS on ZoL. Brian Behlendorf has graciously encouraged me to add FreeBSD support directly to ZoL https://github.com/zfsonfreebsd/ZoF so that we might all have a single shared code base.\nA port for ZoF can be found at https://github.com/miwi-fbsd/zof-port Before it can be committed some additional functionality needs to be added to the FreeBSD opencrypto framework. These can be found at https://reviews.freebsd.org/D18520\nThis port will provide FreeBSD users with multi modifier protection, project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, and various command line improvements.\n\n
With FreeBSD having gone all the way to 12, it is perhaps useful to take a look back at all the things that have been accomplished, in terms of many visible changes, as well as all the things that happen behind the scenes to ensure that FreeBSD continues to offer an alternative in both design, implementation, and execution.\nThe things you can look forward to reading about are too numerous to summarize, but cover just about everything from finalizing releases, administrative work, optimizations and depessimizations, features added and fixed, and many areas of improvement that might just surprise you a little.\nPlease have a cup of coffee, tea, hot cocoa, or other beverage of choice, and enjoy this culmulative set of reports covering everything that’s been done since October, 2017.\n—Daniel Ebdrup\n\n
It has been a little over one year now that I’m with the Ravenports project. Time to reflect my involvement, my expectations and hopes.\n\n
Ravenports is a universal packaging framework for *nix operating systems. For the user it provides easy access to binary packages of common software for multiple platforms. It has been the long-lasting champion on Repology’s top 10 repositories regarding package freshness (rarely dropping below 96 percent while all other projects keep below 90!).\n\n
For the porter it offers a well-designed and elegant means of writing cross-platform buildsheets that allow building the same version of the software with (completely or mostly) the same compile-time configuration on different operating systems or distributions.\n\n
And for the developer it means a real-world project that’s written in modern Ada (ravenadm) and C (pkg) – as well as some Perl for support scripts and make. Things feel very optimized and fast. Not being a programmer though, I cannot really say anything about the actual code and thus leave it to the interested reader’s judgement.\n\n
New stuff in the official FreeBSD repositories! The X11 team has landed a newer version of libinput, opening up the way for KDE Plasma 5.14 in ports. That’s a pretty big update and it may frighten people with a new wallpaper.\nWhat this means is that the graphical stack is once again on-par with what Plasma upstream expects, and we can get back to chasing releases as soon as they happen, rather than gnashing our teeth at missing dependencies. The KDE-FreeBSD CI servers are in the process of being upgraded to 12-STABLE, and we’re integrating with the new experimental CI systems as well. This means we are chasing sensibly-modern systems (13-CURRENT is out of scope).\n\n
Talking to FreeBSD users recently, I became aware that while I’ve created a lot of tools, I haven’t done a very good job of explaining how, and more importantly when to use them. So for all of the EC2-curious FreeBSD users out there: Here are the many ways to launch and configure FreeBSD in EC2 — ranging from the simplest to the most complicated (but most powerful):\n\n
I hope I’ve provided tools which help you to run FreeBSD in EC2, no matter how common or unusual your needs are. If you find my work useful, please consider supporting my work in this area; while this is both something I enjoy working on and something which is useful for my day job (Tarsnap, my online backup service), having support would make it easier for me to prioritize FreeBSD/EC2 issues over other projects.\n\n
GOG.com prefers that you use their GOG Galaxy desktop app to download, install and manage all of your GOG games. But customers always have the option to install the game on their own terms, with a platform-specific installer.\nGOG offers these installers for Mac, Windows and/or Linux, depending on which platforms the game is available for.\n\n
Of course, none of those are NetBSD. So, if I wanted to even attempt to play a game distributed by GOG.com on NetBSD, which one should I pick? The obvious choice is the Linux installer, since Linux is the most similar to NetBSD, right? Au contraire! In practice, I found that it is easier to download the Windows installer.\n\n
Here’s what I mean. For example, I ported the open source version of Aquaria to pkgsrc, but that package is only the game’s engine, not the multimedia data. The multimedia data is still copyrighted. Therefore, you need to get it from somewhere else. GOG is usually a good choice, because they distribute their games without DRM. And as mentioned earlier, picking the Linux installer seemed like a natural choice.\n\n
Now, actually PLAYING the games on NetBSD is a separate matter entirely. The game I’ve got here, though, my current obsession Pyre, is built with MonoGame and therefore could theoretically work on NetBSD, too, with the help of a library called FNA and a script for OpenBSD called fnaify. I do hope to create a pkgsrc package for FNA and port the fnaify script to NetBSD at some point.\n\n
We sat down at BSDCan 2018 to interview Kirk McKusick about various topics ranging about the early years of Berkeley Unix, his continuing work on UFS, the governance of FreeBSD, and more.
\n\n##Interview - Kirk McKusick - mckusick@mckusick.com
\n25 years of FreeBSD
We want to extend a big thank you to the entire BSD community for making this show possible, and to all of our viewers for watching and providing the feedback that makes this show successful. We wish you all a happy and prosperous new year, and we’ll see you next week.
\n\nThe Open Source midlife crisis, Donald Knuth The Yoda of Silicon Valley, Certbot For OpenBSD's httpd, how to upgrade FreeBSD from 11 to 12, level up your nmap game, NetBSD desktop, and more.
\n\n##Headlines
\n###Open Source Confronts its midlife crisis
\n\n\nMidlife is tough: the idealism of youth has faded, as has inevitably some of its fitness and vigor. At the same time, the responsibilities of adulthood have grown. Making things more challenging, while you are navigating the turbulence of teenagers, your own parents are likely entering life’s twilight, needing help in new ways from their adult children. By midlife, in addition to the singular joys of life, you have also likely experienced its terrible sorrows: death, heartbreak, betrayal. Taken together, the fading of youth, the growth in responsibility and the endurance of misfortune can lead to cynicism or (worse) drastic and poorly thought-out choices. Add in a little fear of mortality and some existential dread, and you have the stuff of which midlife crises are made…
\n
\nI raise this not because of my own adventures at midlife, but because it is clear to me that open source — now several decades old and fully adult — is going through its own midlife crisis. This has long been in the making: for years, I (and others) have been critical of service providers’ parasitic relationship with open source, as cloud service providers turn open source software into a service offering without giving back to the communities upon which they implicitly depend. At the same time, open source has been (rightfully) entirely unsympathetic to the proprietary software models that have been burned to the ground — but also seemingly oblivious as to the larger economic waves that have buoyed them.
\nSo it seemed like only a matter of time before the companies built around open source software would have to confront their own crisis of confidence: open source business models are really tough, selling software-as-a-service is one of the most natural of them, the cloud service providers are really good at it — and their commercial appetites seem boundless. And, like a new cherry red two-seater sports car next to a minivan in a suburban driveway, some open source companies are dealing with this crisis exceptionally poorly: they are trying to restrict the way that their open source software can be used. These companies want it both ways: they want the advantages of open source — the community, the positivity, the energy, the adoption, the downloads — but they also want to enjoy the fruits of proprietary software companies in software lock-in and its monopolistic rents. If this were entirely transparent (that is, if some bits were merely being made explicitly proprietary), it would be fine: we could accept these companies as essentially proprietary software companies, albeit with an open source loss-leader. But instead, these companies are trying to license their way into this self-contradictory world: continuing to claim to be entirely open source, but perverting the license under which portions of that source are available. Most gallingly, they are doing this by hijacking open source nomenclature. Of these, the laughably named commons clause is the worst offender (it is plainly designed to be confused with the purely virtuous creative commons), but others (including CockroachDB’s Community License, MongoDB’s Server Side Public License, and Confluent’s Community License) are little better. And in particular, as it apparently needs to be said: no, “community” is not the opposite of “open source” — please stop sullying its good name by attaching it to licenses that are deliberately not open source! But even if they were more aptly named (e.g. “the restricted clause” or “the controlled use license” or — perhaps most honest of all — “the please-don’t-put-me-out-of-business-during-the-next-reInvent-keynote clause”), these licenses suffer from a serious problem: they are almost certainly asserting rights that the copyright holder doesn’t in fact have.
\nIf I sell you a book that I wrote, I can restrict your right to read it aloud for an audience, or sell a translation, or write a sequel; these restrictions are rights afforded the copyright holder. I cannot, however, tell you that you can’t put the book on the same bookshelf as that of my rival, or that you can’t read the book while flying a particular airline I dislike, or that you aren’t allowed to read the book and also work for a company that competes with mine. (Lest you think that last example absurd, that’s almost verbatim the language in the new Confluent Community (sic) License.) I personally think that none of these licenses would withstand a court challenge, but I also don’t think it will come to that: because the vendors behind these licenses will surely fear that they wouldn’t survive litigation, they will deliberately avoid inviting such challenges. In some ways, this netherworld is even worse, as the license becomes a vessel for unverifiable fear of arbitrary liability.
\nlet me put this to you as directly as possible: cloud services providers are emphatically not going to license your proprietary software. I mean, you knew that, right? The whole premise with your proprietary license is that you are finding that there is no way to compete with the operational dominance of the cloud services providers; did you really believe that those same dominant cloud services providers can’t simply reimplement your LDAP integration or whatever? The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won’t even move the needle in terms of their effort.
\nWorse than all of this (and the reason why this madness needs to stop): licenses that are vague with respect to permitted use are corporate toxin. Any company that has been through an acquisition can speak of the peril of the due diligence license audit: the acquiring entity is almost always deep pocketed and (not unrelatedly) risk averse; the last thing that any company wants is for a deal to go sideways because of concern over unbounded liability to some third-party knuckle-head. So companies that engage in license tomfoolery are doing worse than merely not solving their own problem: they are potentially poisoning the wellspring of their own community.
\nin the end, open source will survive its midlife questioning just as people in midlife get through theirs: by returning to its core values and by finding rejuvenation in its communities. Indeed, we can all find solace in the fact that while life is finite, our values and our communities survive us — and that our engagement with them is our most important legacy.
###Donald Knuth - The Yoda of Silicon Valley
\n\n\n\n\nFor half a century, the Stanford computer scientist Donald Knuth, who bears a slight resemblance to Yoda — albeit standing 6-foot-4 and wearing glasses — has reigned as the spirit-guide of the algorithmic realm.
\n
\nHe is the author of “The Art of Computer Programming,” a continuing four-volume opus that is his life’s work. The first volume debuted in 1968, and the collected volumes (sold as a boxed set for about $250) were included by American Scientist in 2013 on its list of books that shaped the last century of science — alongside a special edition of “The Autobiography of Charles Darwin,” Tom Wolfe’s “The Right Stuff,” Rachel Carson’s “Silent Spring” and monographs by Albert Einstein, John von Neumann and Richard Feynman.
\nWith more than one million copies in print, “The Art of Computer Programming” is the Bible of its field. “Like an actual bible, it is long and comprehensive; no other book is as comprehensive,” said Peter Norvig, a director of research at Google. After 652 pages, volume one closes with a blurb on the back cover from Bill Gates: “You should definitely send me a résumé if you can read the whole thing.”
\nThe volume opens with an excerpt from “McCall’s Cookbook”:
Here is your book, the one your thousands of letters have asked us to publish. It has taken us years to do, checking and rechecking countless recipes to bring you only the best, only the interesting, only the perfect.
\n\n\nInside are algorithms, the recipes that feed the digital age — although, as Dr. Knuth likes to point out, algorithms can also be found on Babylonian tablets from 3,800 years ago. He is an esteemed algorithmist; his name is attached to some of the field’s most important specimens, such as the Knuth-Morris-Pratt string-searching algorithm. Devised in 1970, it finds all occurrences of a given word or pattern of letters in a text — for instance, when you hit Command+F to search for a keyword in a document.
\n
\nNow 80, Dr. Knuth usually dresses like the youthful geek he was when he embarked on this odyssey: long-sleeved T-shirt under a short-sleeved T-shirt, with jeans, at least at this time of year. In those early days, he worked close to the machine, writing “in the raw,” tinkering with the zeros and ones.
##News Roundup
\n###Let’s Encrypt: Certbot For OpenBSD’s httpd
\n\n\nLet’s Encrypt is “a free, automated, and open Certificate Authority”.
\n
\nCertbot is “an easy-to-use automatic client that fetches and deploys SSL/TLS certificates for your web server”, well known as “the official Let’s Encrypt client”.
\nI remember well how excited I felt when I read Let’s Encrypt’s “Our First Certificate Is Now Live” in 2015.
\nHow wonderful the goal of them is; it’s to “give people the digital certificates they need in order to enable HTTPS (SSL/TLS) for websites, for free” “to create a more secure and privacy-respecting Web”!
\nSince this year, they have begun to support even ACME v2 and Wildcard Certificate!
\nWell, in OpenBSD as well as other operating systems, it’s easy and comfortable to have their big help 😊
###FreeBSD 12 released: Here is how to upgrade FreeBSD 11 to 12
\n\n\n\n\nThe FreeBSD project announces the availability of FreeBSD 12.0-RELEASE. It is the first release of the stable/12 branch. The new version comes with updated software and features for a wild variety of architectures. The latest release provides performance improvements and better support for FreeBSD jails and more. One can benefit greatly using an upgraded version of FreeBSD.
\n
\n\n\nFreeBSD 12.0 supports amd64, i386, powerpc, powerpc64, powerpcspe, sparc64, armv6, armv7, and aarch64 architectures. One can run it on a standalone server or desktop system. Another option is to run it on Raspberry PI computer. FreeBSD 12 also runs on popular cloud service providers such as AWS EC2/Lightsail or Google compute VM.
\n
New features and highlights:
\nOpenSSL version 1.1.1a (LTS)
\nOpenSSH server 7.8p1
\nUnbound server 1.8.1
\nClang and co 6.0.1
\nThe FreeBSD installer supports EFI+GELI as an installation option
\nVIMAGE FreeBSD kernel configuration option has been enabled by default. VIMAGE was the main reason I custom compiled FreeBSD for the last few years. No more custom compile for me.
\nGraphics drivers for modern ATI/AMD and Intel graphics cards are now available in the FreeBSD ports collection
\nZFS has been updated to include new sysctl(s), vfs.zfs.arc_min_prefetch_ms and vfs.zfs.arc_min_prescient_prefetch_ms, which improve performance of the zpool scrub subcommand
\nThe pf packet filter is now usable within a jail using vnet
\nKDE updated to version 5.12.5
\nThe NFS version 4.1 includes pNFS server support
\nPerl 5.26.2
\nThe default PAGER now defaults to less for most commands
\nThe dd utility has been updated to add the status=progress option to match GNU/Linux dd command to show progress bar while running dd
\nFreeBSD now supports ext4 for read/write operation
\nPython 2.7
\nmuch more
\n###Six Ways to Level Up Your nmap Game
\n\n\n\n\nnmap is a network exploration tool and security / port scanner.
\n
\nIf you’ve heard of it, and you’re like me, you’ve most likely used it like this:
\nie, you’ve pointed it at an IP address and observed the output which tells you the open ports on a host.
\nI used nmap like this for years, but only recently grokked the manual to see what else it could do. Here’s a quick look and some of the more useful things I found out.
###[NetBSD Desktop]
\n\n##Beastie Bits
\n\n##Feedback/Questions
\n\nFreeBSD 12.0 is finally here, partly-cloudy IPsec VPN, KLEAK with NetBSD, How to create synth repos, GhostBSD author interview, and more.
\n\n##Headlines
\n###FreeBSD 12.0 is available
\n\n\nUserland:
\n
\nGroup permissions on /dev/acpi have been changed to allow users in the operator GID to invoke acpiconf(8) to suspend the system.
\nThe default devfs.rules(5) configuration has been updated to allow mount_fusefs(8) with jail(8).
\nThe default PAGER now defaults to less(1) for most commands.
\nThe newsyslog(8) utility has been updated to reject configuration entries that specify setuid(2) or executable log files.
\nThe WITH_REPRODUCIBLE_BUILD src.conf(5) knob has been enabled by default.
\nA new src.conf(5) knob, WITH_RETPOLINE, has been added to enable the retpoline mitigation for userland builds.
\nUserland applications:
\nThe dtrace(1) utility has been updated to support if and else statements.
\nThe legacy gdb(1) utility included in the base system is now installed to /usr/libexec for use with crashinfo(8). The gdbserver and gdbtui utilities are no longer installed. For interactive debugging, lldb(1) or a modern version of gdb(1) from devel/gdb should be used. A new src.conf(5) knob, WITHOUT_GDB_LIBEXEC has been added to disable building gdb(1). The gdb(1) utility is still installed in /usr/bin on sparc64.
\nThe setfacl(1) utility has been updated to include a new flag, -R, used to operate recursively on directories.
\nThe geli(8) utility has been updated to provide support for initializing multiple providers at once when they use the same passphrase and/or key.
\nThe dd(1) utility has been updated to add the status=progress option, which prints the status of its operation on a single line once per second, similar to GNU dd(1).
\nThe date(1) utility has been updated to include a new flag, -I, which prints its output in ISO 8601 formatting.
\nThe bectl(8) utility has been added, providing an administrative interface for managing ZFS boot environments, similar to sysutils/beadm.
\nThe bhyve(8) utility has been updated to add a new subcommand to the -l and -s flags, help, which when used, prints a list of supported LPC and PCI devices, respectively.
\nThe tftp(1) utility has been updated to change the default transfer mode from ASCII to binary.
\nThe chown(8) utility has been updated to prevent overflow of UID or GID arguments where the argument exceeded UID_MAX or GID_MAX, respectively.
\nKernel:
\nThe ACPI subsystem has been updated to implement Device object types for ACPI 6.0 support, required for some Dell, Inc. Poweredge™ AMD® Epyc™ systems.
\nThe amdsmn(4) and amdtemp(4) drivers have been updated to attach to AMD® Ryzen 2™ host bridges.
\nThe amdtemp(4) driver has been updated to fix temperature reporting for AMD® 2990WX CPUs.
\nKernel Configuration:
\nThe VIMAGE kernel configuration option has been enabled by default.
\nThe dumpon(8) utility has been updated to add support for compressed kernel crash dumps when the kernel configuration file includes the GZIO option. See rc.conf(5) and dumpon(8) for additional information.
\nThe NUMA option has been enabled by default in the amd64 GENERIC and MINIMAL kernel configurations.
\nDevice Drivers:
\nThe random(4) driver has been updated to remove the Yarrow algorithm. The Fortuna algorithm remains the default, and now only, available algorithm.
\nThe vt(4) driver has been updated with performance improvements, drawing text at rates ranging from 2- to 6-times faster.
\nDeprecated Drivers:
\nThe lmc(4) driver has been removed.
\nThe ixgb(4) driver has been removed.
\nThe nxge(4) driver has been removed.
\nThe vxge(4) driver has been removed.
\nThe jedec_ts(4) driver has been removed in 12.0-RELEASE, and its functionality replaced by jedec_dimm(4).
\nThe DRM driver for modern graphics chipsets has been marked deprecated and marked for removal in FreeBSD 13. The DRM kernel modules are available from graphics/drm-stable-kmod or graphics/drm-legacy-kmod in the Ports Collection as well as via pkg(8). Additionally, the kernel modules have been added to the lua loader.conf(5) module_blacklist, as installation from the Ports Collection or pkg(8) is strongly recommended.
\nThe following drivers have been deprecated in FreeBSD 12.0, and not present in FreeBSD 13.0: ae(4), de(4), ed(4), ep(4), ex(4), fe(4), pcn(4), sf(4), sn(4), tl(4), tx(4), txp(4), vx(4), wb(4), xe(4)
\nStorage:
\nThe UFS/FFS filesystem has been updated to support check hashes to cylinder-group maps. Support for check hashes is available only for UFS2.
\nThe UFS/FFS filesystem has been updated to consolidate TRIM/BIO_DELETE commands, reducing read/write requests due to fewer TRIM messages being sent simultaneously.
\nTRIM consolidation support has been enabled by default in the UFS/FFS filesystem. TRIM consolidation can be disabled by setting the vfs.ffs.dotrimcons sysctl(8) to 0, or adding vfs.ffs.dotrimcons=0 to sysctl.conf(5).
\nNFS:
\nThe NFS version 4.1 server has been updated to include pNFS server support.
\nZFS:
\nZFS has been updated to include new sysctl(8)s, vfs.zfs.arc_min_prefetch_ms and vfs.zfs.arc_min_prescient_prefetch_ms, which improve performance of the zpool(8) scrub subcommand.
\nThe new spacemap_v2 zpool feature has been added. This provides more efficient encoding of spacemaps, especially for full vdev spacemaps.
\nThe large_dnode zpool feature been imported, allowing better compatibility with pools created under ZFS-on-Linux 0.7.x
\nMany bug fixes have been applied to the device removal feature. This feature allows you to remove a non-redundant or mirror vdev from a pool by relocating its data to other vdevs.
\nIncludes the fix for PR 229614 that could cause processes to hang in zil_commit()
\nBoot Loader Changes:
\nThe lua loader(8) has been updated to detect a list of installed kernels to boot.
\nThe loader(8) has been updated to support geli(8) for all architectures and all disk-like devices.
\nThe loader(8) has been updated to add support for loading Intel® microcode updates early during the boot process.Networking:
\n
\nThe pf(4) packet filter is now usable within a jail(8) using vnet(9).
\nThe pf(4) packet filter has been updated to use rmlock(9) instead of rwlock(9), resulting in significant performance improvements.
\nThe SO_REUSEPORT_LB option has been added to the network stack, allowing multiple programs or threads to bind to the same port, and incoming connections load balanced using a hash function.
###Abandon Linux. Move to FreeBSD or Illumos
\n\n\n\n\nIf you use GNU/Linux and you are only on opensource, you may be doing it wrong. Here’s why.
\n
\nIs your company based on opensource based software only? Do you have a bunch of developers hitting some kind of server you have installed for them to “do their thing”? Being it for economical reasons (remember to donate), being it for philosophycal ones, you may have skipped good alternatives. The BSD’s and Illumos.
\nI bet you are running some sort of Debian, openSuSE or CentOS. It’s very discouraging having entered into the IT field recently and discover many of the people you meet do not even recognise the name BSD. Naming Solaris seems like naming the evil itself. The problem being many do not know why. They can’t point anything specific other than it’s fading out. This has recently shown strong when Oracle officials have stated development for new features has ceased and almost 90 % of developers for Solaris have been layed off. AIX seems alien to almost everybody unless you have a white beard. And all this is silly.
\nAnd here’s why. You are certainly missing two important features that FreeBSD and Illumos derivatives are enjoying. A full virtualization technology, much better and fully developed compared to the LXC containers in the Linux world, such as Jails on BSD, Zones in Solaris/Illumos, and the great ZFS file system which both share.
\nYou have probably heard of a new Linux filesystem named Btrfs, which by the way, development has been dropped from the Red Hat side. Trying to emulate ZFS, Oracle started developing Btrfs file system before they acquired Sun (the original developer of ZFS), and SuSE joined the effort as well as Red Hat. It is not as well developed as ZFS and it hasn’t been tested in production environments as extensively as the former has. That leaves some uncertainty on using it or not. Red Hat leaving it aside does add some more. Although some organizations have used it with various grades of success.
\nBut why is this anyhow interesting for a sysadmin or any organization? Well… FreeBSD (descendant of Berkeley UNIX) and SmartOS (based on Illumos) aglutinate some features that make administration easier, safer, faster and more reliable. The dream of any systems administrator.
\nTo start, the ZFS filesystem combines the typical filesystem with a volume manager. It includes protection against corruption, snapshots and copy-on-write clones, as well as volume manager.
\nJails is another interesting piece of technology. Linux folks usually associate this as a sort of chroot. It isn’t. It is somehow inspired by it but as you may know you can escape from a chroot environment with a blink of an eye. Jails are not called jails casually. The name has a purpose. Contain processes and programs within a defined and totally controlled environment. Jails appeared first in FreeBSD in the year 2000. Solaris Zones debuted on 2005 (now called containers) are the now proprietary version of those.
\nThere are some other technologies on Linux such as Btrfs or Docker. But they have some caveats. Btrfs hasn’t been fully developed yet and it’s hasn’t been proved as much in production environments as ZFS has. And some problems have arisen recently although the developers are pushing the envelope. At some time they will match ZFS capabilities for sure. Docker is growing exponentially and it’s one of the cool technologies of modern times. The caveat is, as before, the development of this technology hasn’t been fully developed. Unlike other virtualization technologies this is not a kernel playing on top of another kernel. This is virtualization at the OS level, meaning differentiated environments can coexist on a single host, “hitting” the same unique kernel which controls and shares the resources. The problem comes when you put Docker on top of any other virtualization technology such as KVM or Xen. It breaks the purpose of it and has a performance penalty.
\nI have arrived into the IT field with very little knowledge, that is true. But what I see strikes me. Working in a bank has allowed me to see a big production environment that needs the highest of the availability and reliability. This is, sometimes, achieved by bruteforce. And it’s legitime and adequate. Redundancy has a reason and a purpose for example. But some other times it looks, it feels, like killing flies with cannons. More hardware, more virtual machines, more people, more of this, more of that. They can afford it, so they try to maintain the cost low but at the end of the day there is a chunky budget to back operations.
\nBut here comes reality. You’re not a bank and you need to squeeze your investment as much as possible. By using FreeBSD jails you can avoid the performance penalty of KVM or Xen virtualization. Do you use VMWare or Hyper-V? You can avoid both and gain in performance. Not only that, control and manageability are equal as before, and sometimes easier to administer. There are four ways to operate them which can be divided in two categories. Hardcore and Human Being. For the Hardcore use the FreeBSD handbook and investigate as much as you can. For the Human Being way there are three options to use. Ezjail, Iocage and CBSD which are frameworks or programs as you may call to manage jails. I personally use Iocage but I have also used Ezjail.
\nHow can you use jails on your benefit? Ever tried to configure some new software and failed miserably? You can have three different jails running at the same time with different configurations. Want to try a new configuration in a production piece of hardware without applying it on the final users? You can do that with a small jail while the production environment is on in another bigger, chunkier jail.
\nWant to divide the hardware as a replica of the division of the team/s you are working with? Want to sell virtual machines with bare metal performance? Do you want to isolate some piece of critical software or even data in a more controlled environment? Do you have different clients and you want to use the same hardware but you want to avoid them seeing each other at the same time you maintain performance and reliability?
\nAre you a developer and you have to have reliable and portable snapshots of your work? Do you want to try new options-designs without breaking your previous work, in a timeless fashion? You can work on something, clone the jail and apply the new ideas on the project in a matter of seconds. You can stop there, export the filesystem snapshot containing all the environment and all your work and place it on a thumbdrive to later import it on a big production system. Want to change that image properties such as the network stack interface and ip? This is just one command away from you.
\nBut what properties can you assign to a jail and how can I manage them you may be wondering. Hostname, disk quota, i/o, memory, cpu limits, network isolation, network virtualization, snapshots and the manage of those, migration and root privilege isolation to name a few. You can also clone them and import and export them between different systems. Some of these things because of ZFS. Iocage is a python program to manage jails and it takes profit from ZFS advantages.
\nBut FreeBSD is not Linux you may say. No it is not. There are no run levels. The systemd factor is out of this equation. This is so since the begginning. Ever wondered where did vi come from? The TCP/IP stack? Your beloved macOS from Apple? All this is coming from the FreeBSD project. If you are used to Linux your adaptation period with any BSD will be short, very short. You will almost feel at home. Used to packaged software using yum or apt-get? No worries. With pkgng, the package management tool used in FreeBSD has almost 27.000 compiled packages for you to use. Almost all software found on any of the important GNU/Linux distros can be found here. Java, Python, C, C++, Clang, GCC, Javascript frameworks, Ruby, PHP, MySQL and the major forks, etc. All this opensource software, and much more, is available at your fingertips.
\nI am a developer and… frankly my time is money and I appreciate both much more than dealing with systems configuration, etc. You can set a VM using VMWare or VirtualBox and play with barebones FreeBSD or you can use TrueOS (a derivative) which comes in a server version and a desktop oriented one. The latter will be easier for you to play with. You may be doing this already with Linux. There is a third and very sensible option. FreeNAS, developed by iXSystems. It is FreeBSD based and offers all these technologies with a GUI. VMWare, Hyper-V? Nowadays you can get your hands off the CLI and get a decent, usable, nice GUI.
\nYou say you play on the cloud. The major players already include FreeBSD in their offerings. You can find it in Amazon AWS or Azure (with official Microsoft support contracts too!). You can also find it in DigitalOcean and other hosting providers. There is no excuse. You can use it at home, at the office, with old or new hardware and in the cloud as well. You can even pay for a support contract to use it. Joyent, the developers of SmartOS have their own cloud with different locations around the globe. Have a look on them too.
\nIf you want the original of ZFS and zones you may think of Solaris. But it’s fading away. But it really isn’t. When Oracle bouth Sun many people ran away in an stampide fashion. Some of the good folks working at Sun founded new projects. One of these is Illumos. Joyent is a company formed by people who developed these technologies. They are a cloud operator, have been recently bought by Samsung and have a very competent team of people providing great tech solutions. They have developed an OS, called SmartOS (based on Illumos) with all these features. The source from this goes back to the early days of UNIX. Do you remember the days of OpenSolaris when Sun opensourced the crown jewels? There you have it. A modern opensource UNIX operating system with the roots in their original place and the head planted on today’s needs.
\nIn conclusion. If you are on GNU/Linux and you only use opensource software you may be doing it wrong. And missing goodies you may need and like. Once you put your hands on them, trust me, you won’t look back. And if you have some “old fashioned” admins who know Solaris, you can bring them to a new profitable and exciting life with both systems.
\nStill not convinced? Would you have ever imagined Microsoft supporting Linux? Even loving it? They do love now FreeBSD. And not only that, they provide their own image in the Azure Cloud and you can get Microsoft support, payed support if you want to use the platform on Azure. Ain’t it… surprising? Convincing at all?
\nPS: I haven’t mentioned both softwares, FreeBSD and SmartOS do have a Linux translation layer. This means you can run Linux binaries on them and the program won’t cough at all. Since the ABI stays stable the only thing you need to run a Linux binary is a translation between the different system calls and the libraries. Remember POSIX? Choose your poison and enjoy it.
\n\n\nI’m assuming that readers have at least a basic knowledge of TCP/IP networking and some UNIX or UNIX-like systems, but not necessarily OpenBSD or FreeBSD. This post will therefore be light on details that aren’t OS specific and are likely to be encountered in normal use (e.g., how to use vi or another text editor.) For more information on these topics, read Absolute FreeBSD (3ed.) by Michael W. Lucas.
\n
\n\n\nI’m redoing my DigitalOcean virtual machines (which they call droplets). My requirements are:
\n
\n\n\nThe last item is on the list because I don’t actually have a public IP address at home; my firewall’s external address is in the RFC 1918 space, and the entire apartment building shares a single public IPv4 address.1 (IPv6? Don’t I wish.) The end-state network will include one OpenBSD droplet providing firewall, router, and VPN services; and one FreeBSD droplet hosting multiple jailed services.
\n
\nI’ll be providing access via these droplets to a NextCloud instance at home. A simple NAT on the DO router droplet isn’t going to work, because packets going from home to the internet would exit through the apartment building’s connection and not through the VPN. It’s possible that I could do work around this issue with packet tagging using the pf firewall, but HAProxy is simple to configure and unlikely to result in hard-to-debug problems. relayd is also an option, but doesn’t have the TLS parsing abilities of HAProxy, which I’ll be using later on.
\nSince this system includes jails running on a VPS, and they’ve got RFC 1918 addresses, I want them reachable from my home network. Once that’s done, I can access the private address space from anywhere through a VPN connection to the cloudy router.
\nThe VPN itself will be of the IPsec variety. IPsec is the traditional enterprise VPN standard, and is even used for classified applications, but has a (somewhat-deserved) reputation for complexity, but recent versions of OpenBSD turn down the difficulty by quite a bit.
\n\n\nThis VPN both separates internal network traffic from public traffic and uses encryption to prevent interception or tampering.
\n
\nOnce traffic has been encrypted, decrypting it without the key would, as Bruce Schneier once put it, require a computer built from something other than matter that occupies something other than space. Dyson spheres and a frakton of causality violation would possibly work, as would mathemagical technology that alters the local calendar such that P=NP.2 Black-bag jobs and/or suborning cloud provider employees doesn’t quite have that guarantee of impossibility, however. If you have serious security requirements, you’ll need to do better than a random blog entry.
##News Roundup
\n###KLEAK: Practical Kernel Memory Disclosure Detection
\n\n\nModern operating systems such as NetBSD, macOS, and Windows isolate their kernel from userspace programs to increase fault tolerance and to protect against malicious manipulations [10]. User space programs have to call into the kernel to request resources, via system calls or ioctls. This communication between user space and kernel space crosses a security boundary. Kernel memory disclosures - also known as kernel information leaks - denote the inadvertent copying of uninitialized bytes from kernel space to user space. Such disclosed memory may contain cryptographic keys, information about the kernel memory layout, or other forms of secret data. Even though kernel memory disclosures do not allow direct exploitation of a system, they lay the ground for it.
\n
\nWe introduce KLEAK, a simple approach to dynamically detect kernel information leaks. Simply said, KLEAK utilizes a rudimentary form of taint tracking: it taints kernel memory with marker values, lets the data travel through the kernel and scans the buffers exchanged between the kernel and the user space for these marker values. By using compiler instrumentation and rotating the markers at regular intervals, KLEAK significantly reduces the number of false positives, and is able to yield relevant results with little effort.
\nOur approach is practically feasible as we prove with an implementation for the NetBSD kernel. A small performance penalty is introduced, but the system remains usable. In addition to implementing KLEAK in the NetBSD kernel, we applied our approach to FreeBSD 11.2. In total, we detected 21 previously unknown kernel memory disclosures in NetBSD-current and FreeBSD 11.2, which were fixed subsequently. As a follow-up, the projects’ developers manually audited related kernel areas and identified dozens of other kernel memory disclosures.
\nThe remainder of this paper is structured as follows. Section II discusses the bug class of kernel memory disclosures. Section III presents KLEAK to dynamically detect instances of this bug class. Section IV discusses the results of applying KLEAK to NetBSD-current and FreeBSD 11.2. Section V reviews prior research. Finally, Section VI concludes this paper.
###How To Create Official Synth Repo
\n\nSystem Environment
\nMake sure /usr/dports is updated and that it contains no cruft (git pull; git status). Remove any cruft.
\nMake sure your ‘synth’ is up-to-date ‘pkg upgrade synth’. If you already updated your system you may have to build synth from scratch, from /usr/dports/ports-mgmt/synth.
\nMake sure /etc/make.conf is clean.
\nUpdate /usr/src to the current master, make sure there is no cruft in it
\nDo a full buildworld, buildkernel, installkernel and installworld
\nReboot
\nAfter the reboot, before proceeding, run ‘uname -a’ and make sure you are now on the desired release or development kernel.
\nSynth Environment
\n/usr/local/etc/synth/ contains the synth configuration. It should contain a synth.ini file (you may have to rename the template), and you will have to create or edit a LiveSystem-make.conf file.
\nSystem requirements are hefty. Just linking chromium alone eats at least 30GB, for example. Concurrent c++ compiles can eat up to 2GB per process. We recommend at least 100GB of SSD based swap space and 300GB of free space on the filesystem.
\nsynth.ini should contain this. Plus modify the builders and jobs to suit your system. With 128G of ram, 30/30 or 40/25 works well. If you have 32G of ram, maybe 8/8 or less.
\n; Take care when hand editing!
\n
\n[Global Configuration]
\nprofile_selected= LiveSystem
\n
\n[LiveSystem]
\nOperating_system= DragonFly
\nDirectory_packages= /build/synth/live_packages
\nDirectory_repository= /build/synth/live_packages/All
\nDirectory_portsdir= /build/synth/dports
\nDirectory_options= /build/synth/options
\nDirectory_distfiles= /usr/distfiles
\nDirectory_buildbase= /build/synth/build
\nDirectory_logs= /build/synth/logs
\nDirectory_ccache= disabled
\nDirectory_system= /
\nNumber_of_builders= 30
\nMax_jobs_per_builder= 30
\nTmpfs_workdir= true
\nTmpfs_localbase= true
\nDisplay_with_ncurses= true
\nleverage_prebuilt= false
LICENSES_ACCEPTED= NONE
Make sure there is no other cruft in /usr/local/etc/synth/
\nIn the example above, the synth working dirs are in “/build/synth”. Make sure the base directories exist. Clean out any cruft for a fresh build from-scratch:
\nrm -rf /build/synth/live_packages/*
\nrm -rf /build/synth/logs
\nmkdir /build/synth/logs
(optionally start a screen session)
\nsynth everything
###Interview with founder and maintainer of GhostBSD, Eric Turgeon
\n\n##Beastie Bits
\n\n##Feedback/Questions
\n\nDragonflyBSD 5.4 has been released, down the Gopher hole with OpenBSD, OpenBSD in stereo with VFIO, BSD/OS the best candidate for legally tested open source Unix, OpenBGPD adds diversity to the routing server landscape, and more.
\n\nDragonFly version 5.4 brings a new system compiler in GCC 8, improved NUMA support, a large of number network and virtual machine driver updates, and updates to video support. This release is 64-bit only, as with previous releases.\nThe details of all commits between the 5.2 and 5.4 branches are available in the associated commit messages for 5.4.0rc and 5.4.0.\n\n
MD5 (dfly-x86_64-5.4.0_REL.img) = 7277d7cffc92837c7d1c5dd11a11b98f
\nMD5 (dfly-x86_64-5.4.0_REL.iso) = 6da7abf036fe9267479837b3c3078408
\nMD5 (dfly-x86_64-5.4.0_REL.img.bz2) = a77a072c864f4b72fd56b4250c983ff1
\nMD5 (dfly-x86_64-5.4.0_REL.iso.bz2) = 4dbfec6ccfc1d59c5049455db914d499
DragonFly BSD is 64-bit only, as announced during the 3.8 release.\n\n
In the early 2000s I thought I had seen the worst of the web - Java applets, Macromedia (>Adobe) Flash, animated GIFs, javascript snow that kept you warm in the winter by burning out your CPU, and so on. For a time we learned from these mistakes, and started putting the burden on the server-side - then with improvements in javascript engines we started abusing it again with JSON/AJAX and it all went down hill from there.\n\n
Like cloud computing, blockchains, machine learning and a tonne of other a la mode technologies around today - most users and service providers don’t need websites that consume 1GB of memory processing JS and downloading 50MB of compressed data just to read Alice’s one-page travel blog or Bob’s notes on porting NetBSD to his blood-pressure monitor.\n\n
Before the HTTP web we relied on Prestel/Minitel style systems, BBS systems, and arguably the most accessible of all - Gopher! Gopher was similar to the locally accessed AmigaGuide format, in that it allowed users to search and retrieve documents interactively, with links and cross-references. Its efficiency and distraction-free nature make it attractive to those who are tired of the invasive, clickbait, ad-filled, javascript-laden web2/3.x. But enough complaining and evangelism - here’s how to get your own Gopher Hole!\n\n
Gophernicus is a modern gopher daemon which aims to be secure (although it still uses inetd -_-); it’s even in OpenBSD ports so at least we can rely on it to be reasonably audited.\n\n
If you need a starting point with Gopher, SDF-EU’s wiki has a good article here.\n\n\n\n
Finally, if you don’t like gopher(1) - there’s always lynx(1) or NCSA Mosaic!\n\n\n\n
I’ve added TLS support to Gophernicus so you don’t need to use stunnel anymore. The code is ugly and unpolished though so I wouldn’t recommend for production use.\n\n
I use a Huawei Matebook X as my primary OpenBSD laptop and one aspect of its hardware support has always been lacking: audio never played out of the right-side speaker. The speaker did actually work, but only in Windows and only after the Realtek Dolby Atmos audio driver from Huawei was installed. Under OpenBSD and Linux, and even Windows with the default Intel sound driver, audio only ever played out of the left speaker.\nNow, after some extensive reverse engineering and debugging with the help of VFIO on Linux, I finally have audio playing out of both speakers on OpenBSD.\n\n
The Linux kernel has functionality called VFIO which enables direct access to a physical device (like a PCI card) from userspace, usually passing it to an emulator like QEMU.\nTo my surprise, these days, it seems to be primarily by gamers who boot Linux, then use QEMU to run a game in Windows and use VFIO to pass the computer’s GPU device through to Windows.\nBy using Linux and VFIO, I was able to boot Windows 10 inside of QEMU and pass my laptop’s PCI audio device through to Windows, allowing the Realtek audio drivers to natively control the audio device. Combined with QEMU’s tracing functionality, I was able to get a log of all PCI I/O between Windows and the PCI audio device.\n\n
To use VFIO to pass-through a PCI device, it first needs to be stubbed out so the Linux kernel’s default drivers don’t attach to it. GRUB can be configured to instruct the kernel to ignore the PCI audio device (8086:9d71) and explicitly enable the Intel IOMMU driver by adding the following to /etc/default/grub and running update-grub\nWith the audio device stubbed out, a new VFIO device can be created from it\nThen the VFIO device (00:1f.3) can be passed to QEMU\nI was using my own build of QEMU for this, due to some custom logging I needed (more on that later), but the default QEMU package should work fine. The events.txt was a file of all VFIO events I wanted logged (which was all of them).\nSince I was frequently killing QEMU and restarting it, Windows 10 wanted to go through its unexpected shutdown routine each time (and would sometimes just fail to boot again). To avoid this and to get a consistent set of logs each time, I used qemu-img to take a snapshot of a base image first, then boot QEMU with that snapshot. The snapshot just gets thrown away the next time qemu-img is run and Windows always starts from a consistent state.\nQEMU will now log each VFIO event which gets saved to a debug-output file.\nWith a full log of all PCI I/O activity from Windows, I compared it to the output from OpenBSD and tried to find the magic register writes that enabled the second speaker. After days of combing through the logs and annotating them by looking up hex values in the documentation, diffing runtime register values, and even brute-forcing it by mechanically duplicating all PCI I/O activity in the OpenBSD driver, nothing would activate the right speaker.\nOne strange thing that I noticed was if I booted Windows 10 in QEMU and it activated the speaker, then booted OpenBSD in QEMU without resetting the PCI device’s power in-between (as a normal system reboot would do), both speakers worked in OpenBSD and the configuration that the HDA controller presented was different, even without any changes in OpenBSD.\n\n
A Primer on Intel HDA\nMost modern computers with integrated sound chips use an Intel High Definition Audio (HDA) Controller device, with one or more codecs (like the Realtek ALC269) hanging off of it. These codecs do the actual audio processing and communicate with DACs and ADCs to send digital audio to the connected speakers, or read analog audio from a microphone and convert it to a digital input stream. In my Huawei Matebook X, this is done through a Realtek ALC298 codec.\nOn OpenBSD, these HDA controllers are supported by the azalia(4) driver, with all of the per-codec details in the lengthy azalia_codec.c file. This file has grown quite large with lots of codec- and machine-specific quirks to route things properly, toggle various GPIO pins, and unmute speakers that are for some reason muted by default.\nThe azalia driver talks to the HDA controller and sets up various buffers and then walks the list of codecs. Each codec supports a number of widget nodes which can be interconnected in various ways. Some of these nodes can be reconfigured on the fly to do things like turning a microphone port into a headphone port.\nThe newer Huawei Matebook X Pro released a few months ago is also plagued with this speaker problem, although it has four speakers and only two work by default. A fix is being proposed for the Linux kernel which just reconfigures those widget pins in the Intel HDA driver. Unfortunately no pin reconfiguration is enough to fix my Matebook X with its two speakers.\nWhile reading more documentation on the HDA, I realized there was a lot more activity going on than I was able to see through the PCI tracing.\nFor speed and efficiency, HDA controllers use a DMA engine to transfer audio streams as well as the commands from the OS driver to the codecs. In the output above, the CORBWP=0; size=256 and RIRBRP=0, size=256 indicate the setup of the CORB (Command Output Ring Buffer) and RIRB (Response Input Ring Buffer) each with 256 entries. The HDA driver allocates a DMA address and then writes it to the two CORBLBASE and CORBUBASE registers, and again for the RIRB.\nWhen the driver wants to send a command to a codec, such as CORB_GET_PARAMETER with a parameter of COP_VOLUME_KNOB_CAPABILITIES, it encodes the codec address, the node index, the command verb, and the parameter, and then writes that value to the CORB ring at the address it set up with the controller at initialization time (CORBLBASE/CORBUBASE) plus the offset of the ring index. Once the command is on the ring, it does a PCI write to the CORBWP register, advancing it by one. This lets the controller know a new command is queued, which it then acts on and writes the response value on the RIRB ring at the same position as the command (but at the RIRB’s DMA address). It then generates an interrupt, telling the driver to read the new RIRBWP value and process the new results.\nSince the actual command contents and responses are handled through DMA writes and reads, these important values weren’t showing up in the VFIO PCI trace output that I had gathered. Time to hack QEMU.\n\n
Since DMA activity wouldn’t show up through QEMU’s VFIO tracing and I obviously couldn’t get Windows to dump these values like I could in OpenBSD, I could make QEMU recognize the PCI write to the CORBWP register as an indication that a command has just been written to the CORB ring.\nMy custom hack in QEMU adds some HDA awareness to remember the CORB and RIRB DMA addresses as they get programmed in the controller. Then any time a PCI write to the CORBWP register is done, QEMU fetches the new CORB command from DMA memory, decodes it into the codec address, node address, command, and parameter, and prints it out. When a PCI read of the RIRBWP register is requested, QEMU reads the response and prints the corresponding CORB command that it stored earlier.\nWith this hack in place, I now had a full log of all CORB commands and RIRB responses sent to and read from the codec:\nAn early version of this patch left me stumped for a few days because, even after submitting all of the same CORB commands in OpenBSD, the second speaker still didn’t work. It wasn’t until re-reading the HDA spec that I realized the Windows driver was submitting more than one command at a time, writing multiple CORB entries and writing a CORBWP value that was advanced by two. This required turning my CORB/RIRB reading into a for loop, reading each new command and response between the new CORBWP/RIRBWP value and the one previously seen.\nSure enough, the magic commands to enable the second speaker were sent in these periods where it submitted more than one command at a time.\n\n
The full log of VFIO PCI activity from the Windows driver was over 65,000 lines and contained 3,150 CORB commands, which is a lot to sort through. It took me a couple more days to reduce that down to a small subset that was actually required to activate the second speaker, and that could only be done through trial and error:\n\n
This required a dozen or so iterations because sometimes I’d comment out too many commands and the right speaker would stop working. Other times the combination of commands would hang the controller and it wouldn’t process any further commands. At one point the combination of commands actually flipped the channels around so the right channel audio was playing through the left speaker.\n\n
After about a week of this routine, I ended up with a list of 662 CORB commands that are needed to get the second speaker working. Based on the number of repeated-but-slightly-different values written with the 0x500 and 0x400 commands, I’m guessing this is some kind of training data and that this is doing the full Dolby/Atmos system initialization, not just turning on the second speaker, but I could be completely wrong.\nIn any case, the stereo sound from OpenBSD is wonderful now and I can finally stop downmixing everything to mono to play from the left speaker. In case you ever need to do this, sndiod can be run with -c 0:0 to reduce the channels to one.\nDue to the massive size of the code needed for this quirk, I’m not sure if I’ll be committing it upstream in OpenBSD or just saving it for my own tree. But at least now the hardware support chart for my Matebook is all yeses for the things I care about.\nI’ve also updated the Linux bug report that I opened before venturing down this path, hoping one of the maintainers of that HDA code that works at Intel or Realtek knew of a solution I could just port to OpenBSD. I’m curious to see what they’ll do with it.\n\n
The UNIX® system is an old operating system, possibly older than many of the readers of this post. However, despite its age, it still has not been open sourced completely. In this post, I will try to detail which parts of which UNIX systems have not yet been open sourced. I will focus on the legal situation in Germany in particular, taking it representative of European law in general – albeit that is a stretch, knowing the diversity of European jurisdictions. Please note that familiarity with basic terms of copyright law is assumed.\n\n
The term “Ancient UNIX” refers to the versions of UNIX up to and including Seventh Edition UNIX (1979) including the 32V port to the VAX. Ancient UNIX was created at Bell Laboratories, a subsidiary of AT&T at the time. It was later transferred of the AT&T UNIX Support Group, then AT&T Information Systems and finally the AT&T subsidiary UNIX System Laboratories, Inc. (USL). The legal situation differs between the United States of America and Germany.\nIn a ruling as part of the UNIX System Laboratories, Inc. v. Berkeley Software Design, Inc. (USL v. BSDi) case, a U.S. court found that USL had no copyright to the Seventh Edition UNIX system and 32V – arguably, by extension, all earlier versions of Ancient UNIX as well – because USL/AT&T had failed to affix copyright notices and could not demonstrate a trade secret. Due to the obsessive tendency of U.S. courts to consider themselves bound to precedents (cf. the infamous Pierson v. Post case), it can be reasonably expected that this ruling would be honored and applied in subsequent cases. Thus under U.S. law, Ancient UNIX can be safely assumed to belong in the public domain.\nThe situation differs in Germany. Unlike the U.S., copyright never needed registration in order to exist. Computer programs are works in the sense of the German 1965 Act on Copyright and Related Rights (Copyright Act, henceforth CopyA) as per CopyA § 2(1) no. 1. Even prior to the amendment of CopyA § 2(1) to include computer programs, computer programs have been recognized as copyrightable works by the German Supreme Court (BGHZ 112, 264 Betriebssystem, no. 19); CopyA § 137d(1) rightly clarifies that. The copyright holder at 1979 would still have been USL via Bell Labs and AT&T. Copyright of computer programs is transferred to the employer upon creation under CopyA § 69(1).\nNote that this does not affect expiry (Daniel Kaboth/Benjamin Spies, commentary on CopyA §§ 69a‒69g, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), Urheberrecht: UrhG, KUG, VerlG, VGG, Kommentar, 4th ed., C. H. Beck, 2018, no. 16 ad CopyA § 69b; cf. Bundestag-Drucksache [BT-Drs.] 12/4022, p. 10). Expiry occurs 70 years after the death of the (co-)author that died most recently as per CopyA § 65(1) and 64; this has been the case since at least the 1960s, meaning there is no way for copyright to have expired already (old version, as per Bundesgesetzblatt Part I No. 51 of September 16, 1965, pp. 1273‒1294).\nIn Germany, private international law applies the so-called “Territorialitätsprinzip” for intellectual property rights. This means that the effect of an intellectual property right is limited to the territory of a state (Anne Lauber-Rönsberg, KollisionsR, in: Hartwig Ahlberg/Horst-Peter Götting (eds.), ibid., pp. 2241 et seqq., no. 4). Additionally, the “Schutzlandprinzip” applies; this means that protection of intellectual property follows the lex loci protectionis, i.e. the law of the country for which protection is sought (BGH GRUR 2015, 264 HiHotel II, no. 25; BGH GRUR 2003, 328 Sender Felsberg, no. 24), albeit this is criticized in parts of doctrine (Lauber-Rönsberg, ibid., no. 10). The “Schutzlandprinzip” requires that the existence of an intellectual property right be verified as well (BGH ZUM 2016, 522 Wagenfeld-Leuchte II, no. 19).\nThus, in Germany, copyright on Ancient UNIX is still alive and well. Who has it, though? A ruling by the U.S. Court of Appeals, Tenth Circuit, in the case of The SCO Group, Inc. v. Novell, Inc. (SCO v. Novell) in the U.S. made clear that Novell owns the rights to System V – thus presumably UNIX System III as well – and Ancient UNIX, though SCO acquired enough rights to develop UnixWare/OpenServer (Ruling 10-4122 [D.C. No. 2:04-CV-00139-TS], pp. 19 et seq.). Novell itself was purchased by the Attachmate Group, which was in turn acquired by the COBOL vendor Micro Focus. Therefore, the rights to SVRX and – outside the U.S. – are with Micro Focus right now. If all you care about is the U.S., you can stop reading about Ancient UNIX here.\nSo how does the Caldera license factor into all of this? For some context, the license was issued January 23, 2002 and covers Ancient UNIX (V1 through V7 including 32V), specifically excluding System III and System V. Caldera, Inc. was founded in 1994. The Santa Cruz Operation, Inc. sold its rights to UNIX to Caldera in 2001, renamed itself to Tarantella Inc. and Caldera renamed itself The SCO Group. Nemo plus iuris ad alium transferre potest quam ipse habet; no one can transfer more rights than he has. The question now becomes whether Caldera had the rights to issue the Caldera license.\nI’ve noted it above but it needs restating: Foreign decisions are not necessarily accepted in Germany due to the “Territorialitätsprinzip” and “Schutzlandprinzip” – however, I will be citing a U.S. ruling for its assessment of the facts for the sake of simplicity. As per ruling 10-4122, “The district court found the parties intended for SCO to serve as Novell’s agent with respect to the old SVRX licenses and the only portion of the UNIX business transferred outright under the APA [asset purchase agreement] was the ability to exploit and further develop the newer UnixWare system. SCO was able to protect that business because it was able to copyright its own improvements to the system. The only reason to protect the earlier UNIX code would be to protect the existing SVRX licenses, and the court concluded Novell retained ultimate control over that portion of the business under the APA.” The relevant agreements consist of multiple pieces:\nthe base Asset Purchase Agreement “APA” (Part I)\nthe base Asset Purchase Agreement “APA” (Part II)\nthe Operating Agremeent and Amendment 1 to the APA\nthe Amendment 2 to the APA\nThe APA dates September 19, 1995, from before the Caldera license. Caldera cannot possibly have acquired rights that The Santa Cruz Operation, Inc. itself never had. Furthermore, I’ve failed to find any mention of Ancient UNIX; all that is transferred is rights to SVRX. Overall, I believe that the U.S. courts’ assesment of the facts represents the situation accurately. Thus for all intents and purposes, UNIX up to and including System V remained with Novell/Attachmate/Micro Focus. Caldera therefore never had any rights to Ancient UNIX, which means it never had the rights to issue the Caldera license. The Caldera license is null and void – in the U.S. because the copyright has been lost due to formalities, everywhere else because Caldera never had the rights to issue it.\nThe first step to truly freeing UNIX would this be to get Micro Focus to re-issue the Caldera license for Ancient UNIX, ideally it would now also include System III and System V.\n\n
Another operating system near UNIX is of interest. The USL v. BSDi lawsuit includes two parties: USL, which we have seen above, and Berkeley Software Design, Inc. BSDi sold BSD/386 (later BSD/OS), which was a derivative of 4.4BSD. The software parts of the BSDi company were acquired by Wind River Systems, whereas the hardware parts went to iXsystems. Copyright is not disputed there, though Wind River Systems ceased selling BSD/OS products 15 years ago, in 2003. In addition, Wind River System let their trademark on BSD expire, though this is without consequence for copyright.\nBSD/OS is notable in the sense that it powered much of early internet infrastructure. Traces of its legacy can still be found on Richard Stevens’ FAQ.\nTo truly make UNIX history free, BSD/OS would arguably also need to see a source code release. BSD/OS at least in its earliest releases under BSDi would ship with source code, though under a non-free license, far from BSD or even GPL licensing.\n\n
The fate of System V as a whole is difficult to determine. Various licenses have been granted to a number of vendors (Dell UNIX comes to mind; HP for HP-UX, IBM for AIX, SGI UNIX, etc.). Sun released OpenSolaris – notoriously, Oracle closed the source to Solaris again after its release –, which is a System V Release 4 descendant. However, this means nothing for the copyright or licensing status of System V itself. Presumably, the rights with System V still remain with Novell (now Micro Focus): SCO managed to sublicense rights to develop and sell UnixWare/OpenServer, themselves System V/III descendants, to unXis, Inc. (now known as Xinuos, Inc.), which implies that Xinuos is not the copyright holder of System V.\nObviously, to free UNIX, System V and its entire family of descendants would also need to be open sourced. However, I expect tremendous resistance on part of all the companies mentioned. As noted in the “Ancient UNIX” section, Micro Focus alone would probably be sufficient to release System V, though this would mean nothing for the other commercial System V derivatives.\n\n
The fate of Bell Labs would be a different one; it would go on to be purchased by Lucent, now part of Nokia. After commercial UNIX got separated out to USL, Research UNIX would continue to exist inside of Bell Labs. Research UNIX V8, V9 and V10 were not quite released by Alcatel-Lucent USA Inc. and Nokia in 2017.\nHowever, this is merely a notice that the companies involved will not assert their copyrights only with respect to any non-commercial usage of the code. It is still not possible, over 30 years later, to freely use the V8 code.\n\n
A small note about patents: Some technologies used in newer iterations of the UNIX system (in particular the System V derivatives) may be encumbered with software patents. An open source license will not help against patent infringement claims. However, the patents on anything used in the historical operating systems will certainly have expired by now. In addition, European readers can ignore this entirely – software patents just aren’t a thing.\n\n
As of last year, there was effectively only a single solution in the Route Server vendor market: the BIRD Internet routing daemon. NIC.CZ (the organisation developing BIRD) has done fantastic work on maintaining their BGP-4 implementation, however, it’s not healthy to have virtually every Internet Exchange Point (IXP) in the RIPE NCC service region depend on a single open source project. The current situation can be compared to the state of the DNS root nameservers back in 2002 - their dependence on the BIND nameserver daemon and the resulting development of NSD as an alternative by NLnet, in cooperation with the RIPE NCC.\nOpenBGPD used to be one of the most popular Route Server implementations until the early 2010s. OpenBGPD’s main problem was that its performance couldn’t keep up with the Internet’s growth, so it lost market share. An analysis by Job Snijders suggested that a modernised OpenBGPD distribution would be a most viable option to regain diversity on the Route Server level.\n\n
The following main missing features were identified in OpenBGPD:\n\n
In previous versions of OpenBGPD, the filtering performance didn’t allow proper filtering of all EBGP sessions. Current best practice at IXP Route Servers is to carefully evaluate and validate of all routes learned from EBGP peers. The OpenBGPD ruleset required to do correct filtering (in many deployment scenarios) was simply too lengthy - and negatively impacted service performance during configuration reloads. While filtering performance is the biggest bottleneck, general improvements to the Routing Information Base were also made to improve scalability. IXP Route Servers with a few hundred peering sessions are commonplace and adding new sessions shouldn’t impact the Route Servers’ service to other peers. We found that performance was the most pressing issue that needed to be tackled.\n\n
As we’ve seen, Internet operators are moving to adopt RPKI based BGP Origin Validation. While it was theoretically possible to emulate RFC 6811-style Origin Validation in previous versions of OpenBGPD, the required configuration wasn’t optimised for performance and wasn’t user friendly. We believe that BGP Origin Validation should be as easy as possible - this requires BGP-4 vendors to implement native, optimised routines for Origin Validation. Of course, enabling Origin Validation shouldn’t have an impact on performance either when processing BGP updates or when updating the Route Origin Authorisation (ROA) table itself.\n\n
OpenBGPD is an integral part of OpenBSD, but IXPs may prefer to run their services infrastructure on an operating system of their choice. Making sure that there’s a portable OpenBGPD version which follows the OpenBSD project release cycle will give IXPs this option.\n\n
By addressing the issues mentioned above, we could bring back OpenBGPD as a viable Route Server implementation.\nSince I was one of the core OpenBGPD developers, I was asked if I wanted to pick up this project again. Thanks to the funding from the RIPE NCC Project Fund, this was possible. Starting in June 2018, I worked full time on this important community project. Over the last few months, many of the problems are already addressed and are now part of the OpenBSD 6.4 release. So far, 154 commits were made to OpenBGPD during the 6.4 development cycle - around 8% of all commits ever to OpenBGPD! This shows that due to funding and dedicated resources, a lot of work could be pushed into the latest release of OpenBGPD.\n\n
The OpenBGPD version, as part of OpenBSD 6.4 release, demonstrates great progress. Even though there have been many changes to the core of OpenBGPD, the released version is as solid and reliable as previous releases and the many bug fixes and improvements make this the best OpenBGPD release so far. The changes in the filter language allow users to write more efficient rulesets while the introduction of RPKI origination validation fixes an important missing feature. For IXPs, OpenBGPD now is an alternative again. There are still open issues, but the gap is closing!\n\n
The following changes should be highlighted:\n\n
Users can only benefit from the changes introduced in OpenBGPD 6.4 when the surrounding 3rd party tools are adjusted accordingly. Two opensource projects such as bgpq3 and arouteserver are frequently used by network operators and IXPs to generate BGP configurations. Thanks to our contributions to those projects, we were able to get them ready for all the new features in OpenBGPD.\n\n
A sizeable chunk of work still left on the table is the rework of the RIB data structures in OpenBGPD - these haven’t been changed since the initial design of OpenBGPD in 2003. There’s currently ongoing work (in small steps, to avoid jeopardising the stability of OpenBGPD) to modernise these data-structures. The goal is to provide better decoupling of the filter step from storing RIB database changes, to pave the way to multi-threaded operations at a later point.\n\n
It’s been incredibly productive to create an environment where a core developer is allowed to work full time on the OpenBGPD code base. However, it’s important to note there still is room for a number of new features to help improve its operational capabilities (such as BMP, RFC 7313, ADD_PATH, etc). It’d be beneficial to the Internet community at large if we can extend Claudio Jeker’s involvement for another year. Open source software doesn’t grow on trees! Strategic investments are the only way to keep OpenBGPD’s roadmap aligned with Internet growth and operator requirements.\n\n
Assembly language on OpenBSD, using bhyve for FreeBSD development, FreeBSD Gaming, FreeBSD for Thanksgiving, no space left on Dragonfly’s hammer2, and more.
\n\n##Headlines
\n###Assembly language on OpenBSD amd64+arm64
\n\n\nThis is a short introduction to assembly language programming on OpenBSD/amd64+arm64. Because of security features in the kernel, I have had to rethink a series of tutorials covering Aarch64 assembly language on OpenBSD, and therefore this will serve as a placeholder-cum-reminder.
\n
\n\n\nOpenBSD, like many UNIX and unix-like operating systems, now uses the Executable and Linkable Format (ELF) for its binary libraries and executables. Although the structure of this format is beyond the scope of this short introduction, it is necessary for me to explain part of one of the headers.
\n
\n\n\nWithin the program header there are sections known as PT_NOTE that OpenBSD and other systems use to distinguish their ELF executables - OpenBSD looks for this section to check if it should attempt to execute the program or not.
\n
\n\n\nIt’s often a good idea to prototype your assembly programs in a high level language such as C - it can then double up as both a set of notes and a working program that you can debug and compile into assembly language to compare with your own asm code.
\n
###Using bhyve for FreeBSD Development
\n\n\n\n\nThe bhyve hypervisor requires a 64-bit x86 processor with hardware support for virtualization. This requirement allows for a simple, clean hypervisor implementation, but it does require a fairly recent
\n
\nprocessor. The current hypervisor requires an Intel processor, but there is an active development branch with support for AMD processors.
\nThe hypervisor itself contains both user and kernel components. The kernel driver is contained in the vmm.ko module and can be loaded either at boot from the boot loader or at runtime. It must
\nbe loaded before any guests can be created. When a guest is created, the kernel driver creates a device file in /dev/vmm which is used by the user programs to interact with the guest.
\nThe primary user component is the bhyve(8) program. It constructs the emulated device tree in the guest and provides the implementation for most of the emulated devices. It also calls the kernel driver to execute the guest. Note that the guest always executes inside the driver itself, so guest execution time in the host is counted as system time in the bhyve process.
\nCurrently, bhyve does not provide a system firmware interface to the guest (neither BIOS nor UEFI). Instead, a user program running on the host is used to perform boot time operations including loading the guest operating system kernel into the guest’s memory and setting the initial guest state so that the guest begins execution at the kernel’s entry point. For FreeBSD guests, the bhyveload(8) program can be used to load the kernel and prepare the guest for execution. Support for some other operating systems is available via the grub2-bhyve program which is available via the sysutils/grub2-bhyve port or as a prebuilt package.
\nThe bhyveload(8) program in FreeBSD 10.0 only supports 64-bit guests. Support for 32-bit guests will be included in FreeBSD 10.1.
See the article for the very technical breakdown of the following:
\nNetwork Setup
\nBridged Configuration
\nPrivate Network with NAT
\nUsing dnsmasq with a Private Network
\nRunning Guests via vmrun.sh
\nConfiguring Guests
\nUsing a bhyve Guest as a Target
\nConclusion
\n\n\n\nThe bhyve hypervisor is a nice addition to a FreeBSD developer’s toolbox. Guests can be used both to develop new features and to test merges to stable branches. The hypervisor has a wide variety of uses beyond developing FreeBSD as well.
\n
##News Roundup
\n###Games on FreeBSD
\n\n\nWhat do all programmers like to do after work? Ok, what do most programers like to do after work? The answer is simple: play a good game! Recently at the Polish BSD User Group meetup mulander was telling us how you can play games on OpenBSD. Today let’s discuss how this looks in the FreeBSD world using the “server only” operating system.
\n
\n\n\nOne of the ways of playing natively is to play indie games which use XNA. XNA is a framework from Microsoft which uses .NET, for creating games. Fortunately, in the BSD world we have Mono, an open source implementation of Microsoft’s .NET Framework which you can use to run games. There is also FNA framework which is a reimplementation of XNA which allows you to run the games under Linux. Thomas Frohwein, from OpenBSD, prepared a script, fnaify. Fnaify translate all dependencies used by an FNA game to OpenBSD dependencies.
\n
\nI decided to port the script to FreeBSD. The script is using /bin/sh which in the case of OpenBSD is a Korn Shell.
\n\n\nI didn’t test it with many games, but I don’t see any reason why it shouldn’t work with all the games tested by the OpenBSD guys. For example, with:
\n
Cryptark
\nRouge Legacy
\nApotheon
\nEscape Goat
\nBastion
\nCrossCode
\nAtom Zombie Smasher
\nOpen-Source games
\n\n\n\nIn FreeBSD and OpenBSD we also will find popular games which were open sourced. For example, I spend a lot of time playing in Quake 3 Arena on my FreeBSD machine. You can very simply install it using pkg:
\n# pkg install ioquake3
\n\n\nThen move the files for the skins and maps to the .ioquake3 directory from your copy of Quake. In the past I also played UrbanTerror which is a fully open source shooter based on the Quake 3 Arena engine. It’s is also very easy to install it from ports:
\n# pkg install iourbanterror
\n\n\nIn the ports tree in the games directory you can find over 1000 directories, many of them with fully implemented games. I didn’t test many games in this category, but you can find some interesting titles like:
\n
\n\n\nAll those titles are simply installed through the packages. In that case I don’t think FreeBSD has any difference from OpenBSD.
\n
\n\n\nOne of the big advantages of FreeBSD over OpenBSD is that FreeBSD supports wine. Wine allows you to run Windows applications under other operating systems (including mac). If you are a FreeBSD 11 user, you can simply fetch wine from packages:
\n# pkg install i386-wine
\n\n\nTo run Windows games, you need to have a 32-bit wine because most of the games on Windows are built on 32-bits (maybe this has changed – I don’t play so much these days). In my case, because I run FreeBSD-CURRENT I needed to build wine from ports. It wasn’t nice, but it also wasn’t unpleasant. The whole step-by-step building process of a wine from ports can be found here.
\n
\n\n\nAs you can see there are many titles available for *BSDs. Thanks to the FNA and fnaify, OpenBSD and FreeBSD can work with indie games which use the XNA framework. There are many interesting games implemented using this framework. Open source is not only for big server machines, and there are many re-implementations of popular games like Theme Hospital or RollerCoaster Tycoon 2. The biggest market is still enabled through wine, although its creates a lot of problems to run the games. Also, if you are an OpenBSD user only this option is not available for you. Please also note that we didn’t discuss any other emulators besides wine. In OpenBSD and FreeBSD there are many of them for GameBoy, SNES, NeoGeo and other games consoles.
\n
\n\n\nI’ve been working on FreeBSD for Intel for almost 6 months now. In the world of programmers, I am considered an old dog, and these 6 months have been all about learning new tricks. Luckily, I’ve found myself in a remarkably inclusive and receptive community whose patience seems plentiful. As I get ready to take some time off for the holidays, and move into that retrospective time of year, I thought I’d beat the rush a bit and update on the progress
\n
\nEarlier this year, I decided to move from architect of the Linux graphics driver into a more nebulous role of FreeBSD enabling. I was excited, but also uncertain if I was making the right decision.
\nEarlier this half, I decided some general work in power management was highly important and began working there. I attended BSDCam (handsome guy on the right), and led a session on Power Management. I was honored to be able to lead this kind of effort.
\nEarlier this quarter, I put the first round of my patches up for review, implementing suspend-to-idle. I have some rougher patches to handle s0ix support when suspending-to-idle. I gave a talk MeetBSD about our team’s work.
\nEarlier this month, I noticed that FreeBSD doesn’t have an implementation for Intel Speed Shift (HWPstates), and I started working on that.
\nEarlier this week, I was promoted from a lowly mentee committer to a full src committer.
\nEarlier today, I decided to relegate my Linux laptop to the role of my backup machine, and I am writing this from my Dell XPS13 running FreeBSD
vandamme 13.0-CURRENT FreeBSD 13.0-CURRENT #45 881fee072ff(hwp)-dirty: Mon Nov 19 16:19:32 PST 2018 bwidawsk@vandamme:/usr/home/bwidawsk/usr/obj/usr/home/bwidawsk/usr/src/amd64.amd64/sys/DEVMACHINE amd64
\n\n\n6 months later, I feel a lot less uncertain about making the right decision. In fact, I think both opportunities would be great, and I’m thankful this Thanksgiving that this is my life and career. I have more plans and things I want to get done. I’m looking forward to being thankful again next year.
\n
###hammer2: no space left on device on Dragonfly BSD
\n\n\n\n\nhammer2 does not actually delete a file when you rm or unlink it. Since recovery of the file is possible (this is the design of hammer2), there will still be an entry taking up data. It’s similar to how git works.
\n
\nEven with 75% usage listed here, the filesystem could still have filled up. If you are using it as your root filesystem, then attempts to clean up data may fail. If the kernel panics over this, you will see something like this.
\n\n\nIf you have a recent enough version of the rescue ramdisk installed, on bootup you can press ‘r’ and access the rescue ramdisk. Your provider will have to offer some sort of remote interface for interacting with the operating system before it boots, like VNC or IPMI. You can then mount your filesystem using:
\n
[root@ ~]# mkdir /tmp/fs
\n[root@ ~]# mount_hammer2 -o local /dev/vbd0s1a /tmp/fs
\n\n\nIf you receive an error that /sbin/hammer2 is not found, then your rescue ramdisk is not up to date enough. In that scenario, download the latest 5.2 iso from dragonflybsd.org and boot from the cd-rom on your virtual machine or physical device. Just login as root instead of installer.
\n
\nIf the mount does succeed, then all you have to do is run the following twice:
[root@ ~]# /sbin/hammer2 bulkfree /tmp/fs
\n\n\nIf you do not have enough memory on your machine, you may need to mount swap. Add your swap partition to the /etc/fstab and then do:
\n
[root@ ~]# swapon -a
\n\n\nOnce you have ran the bulkfree command twice, the usage reported by df -h will be correct. However, there is a chance on reboot that a core dump will be placed in /var/crash/ so be prepared to have plenty of space free in case that happens. You should also delete any files you can and run the bulkfree operation twice afterwards to clear up additional space.
\n
##Beastie Bits
\n\n##Feedback/Questions
\n\nThoughts on NetBSD 8.0, Monitoring love for a GigaBit OpenBSD firewall, cat’s source history, X.org root permission bug, thoughts on OpenBSD as a desktop, and NomadBSD review.
\n\n##Headlines
\n###Some thoughts on NetBSD 8.0
\n\n\nNetBSD is a highly portable operating system which can be run on dozens of different hardware architectures. The operating system’s clean and minimal design allow it to be run in all sorts of environments, ranging from embedded devices, to servers, to workstations. While the base operating system is minimal, NetBSD users have access to a large repository of binary packages and a ports tree which I will touch upon later.
\n
\nI last tried NetBSD 7.0 about three years ago and decided it was time to test drive the operating system again. In the past three years NetBSD has introduced a few new features, many of them security enhancements. For example, NetBSD now supports write exclusive-or execute (W^X) protection and address space layout randomization (ASLR) to protect programs against common attacks. NetBSD 8.0 also includes USB3 support and the ability to work with ZFS storage volumes.
\n\n\nSince I had set up NetBSD with a Full install and enabled xdm during the setup process, the operating system booted to a graphical login screen. From here we can sign into our account. The login screen does not provide options to shut down or restart the computer. Logging into our account brings up the twm window manager and provides a virtual terminal, courtesy of xterm. There is a panel that provides a method for logging out of the window manager. The twm environment is sparse, fast and devoid of distractions.
\n
\n\n\nNetBSD ships with a fairly standard collection of command line tools and manual pages, but otherwise it is a fairly minimal platform. If we want to run network services, have access to a web browser, or use a word processor we are going to need to install more software. There are two main approaches to installing new packages. The first, and easier approach, is to use the pkgin package manager. The pkgin utility works much the same way APT or DNF work in the Linux world, or as pkg works on FreeBSD. We can search for software by name, install or remove items. I found pkgin worked well, though its output can be terse. My only complaint with pkgin is that it does not handle “close enough” package names. For example, if I tried to run “pkgin install vlc” or “pkgin install firefox” I would quickly be told these items did not exist. But a more forgiving package manager will realize items like vlc2 or firefox45 are available and offer to install those.
\n
\nThe pkgin tool installs new programs in the /usr/pkg/bin directory. Depending on your configuration and shell, this location may not be in your user’s path, and it will be helpful to adjust your PATH variable accordingly.
\nThe other common approach to acquiring new software is to use the pkgsrc framework. I have talked about using pkgsrc before and I will skip the details. Basically, we can download a collection of recipes for building popular open source software and run a command to download and install these items from their source code. Using pkgsrc basically gives us the same software as using pkgin would, but with some added flexibility on the options we use.
\nOnce new software has been installed, it may need to be enabled and activated, particularly if it uses (or is) a background service. New items can be enabled in the /etc/rc.conf file and started or stopped using the service command. This works about the same as the service command on FreeBSD and most non-systemd Linux distributions.
\n\n\nI found that, when logged into the twm environment, NetBSD used about 130MB of RAM. This included kernel memory and all active memory. A fresh, Full install used up 1.5GB of disk space. I generally found NetBSD ran well in both VirtualBox and on my desktop computer. The system was quick and stable. I did have trouble getting a higher screen resolution in both environments. NetBSD does not offer VirtualBox add-on modules. There are NetBSD patches for VirtualBox out there, but there is some manual work involved in getting them working. When running on my desktop computer I think the resolution issue was one of finding and dealing with the correct video driver. Screen resolution aside, NetBSD performed well and detected all my hardware.
\n
\n\n\nSince NetBSD provides users with a small, core operating system without many utilities if we want to use NetBSD for something we need to have a project in mind. I had four mini projects in mind I wanted to try this week: install a desktop environment, enable file sharing for computers on the local network, test multimedia (video, audio and YouTube capabilities), and set up a ZFS volume for storage.
\n
\nI began with the desktop. Specifically, I followed the same tutorial I used three years ago to try to set up the Xfce desktop. While Xfce and its supporting services installed, I was unable to get a working desktop out of the experience. I could get the Xfce window manager working, but not the entire session. This tutorial worked beautifully with NetBSD 7.0, but not with version 8.0. Undeterred, I switched gears and installed Fluxbox instead. This gave me a slightly more powerful graphical environment than what I had before with twm while maintaining performance. Fluxbox ran without any problems, though its application menu was automatically populated with many programs which were not actually installed.
\nNext, I tried installing a few multimedia applications to play audio and video files. Here I ran into a couple of interesting problems. I found the music players I installed would play audio files, but the audio was quite slow. It always sounded like a cassette tape dragging. When I tried to play a video, the entire graphical session would crash, taking me back to the login screen. When I installed Firefox, I found I could play YouTube videos, and the video played smoothly, but again the audio was unusually slow.
\nI set up two methods of sharing files on the local network: OpenSSH and FTP. NetBSD basically gives us OpenSSH for free at install time and I added an FTP server through the pkgin package manager which worked beautifully with its default configuration.
\nI experimented with ZFS support a little, just enough to confirm I could create and access ZFS volumes. ZFS seems to work on NetBSD just as well, and with the same basic features, as it does on FreeBSD and mainstream Linux distributions. I think this is a good feature for the portable operating system to have since it means we can stick NetBSD on nearly any networked computer and use it as a NAS.
\n\n\nNetBSD, like its close cousins (FreeBSD and OpenBSD) does not do a lot of hand holding or automation. It offers a foundation that will run on most CPUs and we can choose to build on that foundation. I mention this because, on its own, NetBSD does not do much. If we want to get something out of it, we need to be willing to build on its foundation - we need a project. This is important to keep in mind as I think going into NetBSD and thinking, “Oh I’ll just explore around and expand on this as I go,” will likely lead to disappointment. I recommend figuring out what you want to do before installing NetBSD and making sure the required tools are available in the operating system’s repositories.
\n
\nSome of the projects I embarked on this week (using ZFS and setting up file sharing) worked well. Others, like getting multimedia support and a full-featured desktop, did not. Given more time, I’m sure I could find a suitable desktop to install (along with the required documentation to get it and its services running), or customize one based on one of the available window managers. However, any full featured desktop is going to require some manual work. Media support was not great. The right players and codecs were there, but I was not able to get audio to play smoothly.
\nMy main complaint with NetBSD relates to my struggle to get some features working to my satisfaction: the documentation is scattered. There are four different sections of the project’s website for documentation (FAQs, The Guide, manual pages and the wiki). Whatever we are looking for is likely to be in one of those, but which one? Or, just as likely, the tutorial we want is not there, but is on a forum or blog somewhere. I found that the documentation provided was often thin, more of a quick reference to remind people how something works rather than a full explanation.
\nAs an example, I found a couple of documents relating to setting up a firewall. One dealt with networking NetBSD on a LAN, another explored IPv6 support, but neither gave an overview on syntax or a basic guide to blocking all but one or two ports. It seemed like that information should already be known, or picked up elsewhere.
\nNewcomers are likely to be a bit confused by software management guides for the same reason. Some pages refer to using a tool called pkg_add, others use pkgsrc and its make utility, others mention pkgin. Ultimately, these tools each give approximately the same result, but work differently and yet are mentioned almost interchangeably. I have used NetBSD before a few times and could stumble through these guides, but new users are likely to come away confused.
\nOne quirk of NetBSD, which may be a security feature or an inconvenience, depending on one’s point of view, is super user programs are not included in regular users’ paths. This means we need to change our path if we want to be able to run programs typically used by root. For example, shutdown and mount are not in regular users’ paths by default. This made checking some things tricky for me.
\nUltimately though, NetBSD is not famous for its convenience or features so much as its flexibility. The operating system will run on virtually any processor and should work almost identically across multiple platforms. That gives NetBSD users a good deal of consistency across a range of hardware and the chance to experiment with a member of the Unix family on hardware that might not be compatible with Linux or the other BSDs.
###Showing a Gigabit OpenBSD Firewall Some Monitoring Love
\n\n\n\n\nI have a pretty long history of running my home servers or firewalls on “exotic” hardware. At first, it was Sun Microsystem hardware, then it moved to the excellent Soekris line, with some cool single board computers thrown in the mix. Recently I’ve been running OpenBSD Octeon on the Ubiquiti Edge Router Lite, an amazing little piece of kit at an amazing price point.
\n
\n\n\nThis setup has served me for some time and I’ve been extremely happy with it. But, in the #firstworldproblems category, I recently upgraded the household to the amazing Gigabit fibre offering from Sonic. A great problem to have, but also too much of a problem for the little Edge Router Lite (ERL).
\n
\nThe way the OpenBSD PF firewall works, it’s only able to process packets on a single core. Not a problem for the dual-core 500 MHz ERL when you’re pushing under ~200 Mbps, but more of a problem when you’re trying to push 1000 Mbps.
\nI needed something that was faster on a per core basis but still satisfied my usual firewall requirements. Loosely:
\n\n\nAfter evaluating a LOT of different options I settled on the Protectli Vault FW2B. With the specs required for the firewall (2 GB RAM and 8 GB drive) it comes in at a mere $239 USD! Installation of OpenBSD 6.4 was pretty straight forward, with the only problem I had was Etcher did not want to recognize the ‘.fs’ extension on the install image as bootable image. I quickly fixed this with good old Unix dd(1) on the Mac. Everything else was incredibly smooth.
\n
\nAfter loading the same rulesets on my new install, the results were fantastic!
\n\n\n\n\nNow that the machine was up and running (and fast!), I wanted to know what it was doing. Over the years, I’ve always relied on the venerable pfstat software to give me an overview of my traffic, blocked packets, etc. It looks like this:
\n
\nAs you can see it’s based on RRDtool, which was simply incredible in its time. Having worked on monitoring almost continuously for almost the past decade, I wanted to see if we could re-implement the same functionality using more modern tools as RRDtool and pfstat definitely have their limitations. This might be an opportunity to learn some new things as well.
\nI came across pf-graphite which seemed to be a great start! He had everything I needed and I added a few more stats from the detailed interface statistics and the ability for the code to exit for running from cron(8), which is a bit more OpenBSD style. I added code for sending to some SaaS metrics platforms but ultimately stuck with straight Graphite. One important thing to note was to use the Graphite pickle port (2004) instead of the default plaintext port for submission. Also you will need to set a loginterface in your ‘pf.conf’.
\nA bit of tweaking with Graphite and Grafana, and I had a pretty darn good recreation of my original PF stats dashboard!
\nAs you can see it’s based on RRDtool, which was simply incredible in its time. Having worked on monitoring almost continuously for almost the past decade, I wanted to see if we could re-implement the same functionality using more modern tools as RRDtool and pfstat definitely have their limitations. This might be an opportunity to learn some new things as well.
\nI came across pf-graphite which seemed to be a great start! He had everything I needed and I added a few more stats from the detailed interface statistics and the ability for the code to exit for running from cron(8), which is a bit more OpenBSD style. I added code for sending to some SaaS metrics platforms but ultimately stuck with straight Graphite. One important thing to note was to use the Graphite pickle port (2004) instead of the default plaintext port for submission. Also you will need to set a loginterface in your ‘pf.conf’.
\nA bit of tweaking with Graphite and Grafana, and I had a pretty darn good recreation of my original PF stats dashboard!
\n\n\nI once had a debate with members of my extended family about whether a computer science degree is a degree worth pursuing. I was in college at the time and trying to decide whether I should major in computer science. My aunt and a cousin of mine believed that I shouldn’t. They conceded that knowing how to program is of course a useful and lucrative thing, but they argued that the field of computer science advances so quickly that everything I learned would almost immediately be outdated. Better to pick up programming on the side and instead major in a field like economics or physics where the basic principles would be applicable throughout my lifetime.
\n
\nI knew that my aunt and cousin were wrong and decided to major in computer science. (Sorry, aunt and cousin!) It is easy to see why the average person might believe that a field like computer science, or a profession like software engineering, completely reinvents itself every few years. We had personal computers, then the web, then phones, then machine learning… technology is always changing, so surely all the underlying principles and techniques change too. Of course, the amazing thing is how little actually changes. Most people, I’m sure, would be stunned to know just how old some of the important software on their computer really is. I’m not talking about flashy application software, admittedly—my copy of Firefox, the program I probably use the most on my computer, is not even two weeks old. But, if you pull up the manual page for something like grep, you will see that it has not been updated since 2010 (at least on MacOS). And the original version of grep was written in 1974, which in the computing world was back when dinosaurs roamed Silicon Valley. People (and programs) still depend on grep every day.
\nMy aunt and cousin thought of computer technology as a series of increasingly elaborate sand castles supplanting one another after each high tide clears the beach. The reality, at least in many areas, is that we steadily accumulate programs that have solved problems. We might have to occasionally modify these programs to avoid software rot, but otherwise they can be left alone. grep is a simple program that solves a still-relevant problem, so it survives. Most application programming is done at a very high level, atop a pyramid of much older code solving much older problems. The ideas and concepts of 30 or 40 years ago, far from being obsolete today, have in many cases been embodied in software that you can still find installed on your laptop.
\nI thought it would be interesting to take a look at one such old program and see how much it had changed since it was first written. cat is maybe the simplest of all the Unix utilities, so I’m going to use it as my example. Ken Thompson wrote the original implementation of cat in 1969. If I were to tell somebody that I have a program on my computer from 1969, would that be accurate? How much has cat really evolved over the decades? How old is the software on our computers?
\nThanks to repositories like this one, we can see exactly how cat has evolved since 1969. I’m going to focus on implementations of cat that are ancestors of the implementation I have on my Macbook. You will see, as we trace cat from the first versions of Unix down to the cat in MacOS today, that the program has been rewritten more times than you might expect—but it ultimately works more or less the same way it did fifty years ago.
\n\n\nKen Thompson and Dennis Ritchie began writing Unix on a PDP 7. This was in 1969, before C, so all of the early Unix software was written in PDP 7 assembly. The exact flavor of assembly they used was unique to Unix, since Ken Thompson wrote his own assembler that added some features on top of the assembler provided by DEC, the PDP 7’s manufacturer. Thompson’s changes are all documented in the original Unix Programmer’s Manual under the entry for as, the assembler.
\n
\nThe first implementation of cat is thus in PDP 7 assembly. I’ve added comments that try to explain what each instruction is doing, but the program is still difficult to follow unless you understand some of the extensions Thompson made while writing his assembler. There are two important ones. First, the ; character can be used to separate multiple statements on the same line. It appears that this was used most often to put system call arguments on the same line as the sys instruction. Second, Thompson added support for “temporary labels” using the digits 0 through 9. These are labels that can be reused throughout a program, thus being, according to the Unix Programmer’s Manual, “less taxing both on the imagination of the programmer and on the symbol space of the assembler.” From any given instruction, you can refer to the next or most recent temporary label n using nf and nb respectively. For example, if you have some code in a block labeled 1:, you can jump back to that block from further down by using the instruction jmp 1b. (But you cannot jump forward to that block from above without using jmp 1f instead.)
\nThe most interesting thing about this first version of cat is that it contains two names we should recognize. There is a block of instructions labeled getc and a block of instructions labeled putc, demonstrating that these names are older than the C standard library. The first version of cat actually contained implementations of both functions. The implementations buffered input so that reads and writes were not done a character at a time.
\nThe first version of cat did not last long. Ken Thompson and Dennis Ritchie were able to persuade Bell Labs to buy them a PDP 11 so that they could continue to expand and improve Unix. The PDP 11 had a different instruction set, so cat had to be rewritten. I’ve marked up this second version of cat with comments as well. It uses new assembler mnemonics for the new instruction set and takes advantage of the PDP 11’s various addressing modes. (If you are confused by the parentheses and dollar signs in the source code, those are used to indicate different addressing modes.) But it also leverages the ; character and temporary labels just like the first version of cat, meaning that these features must have been retained when as was adapted for the PDP 11.
\nThe second version of cat is significantly simpler than the first. It is also more “Unix-y” in that it doesn’t just expect a list of filename arguments—it will, when given no arguments, read from stdin, which is what cat still does today. You can also give this version of cat an argument of - to indicate that it should read from stdin.
\nIn 1973, in preparation for the release of the Fourth Edition of Unix, much of Unix was rewritten in C. But cat does not seem to have been rewritten in C until a while after that. The first C implementation of cat only shows up in the Seventh Edition of Unix. This implementation is really fun to look through because it is so simple. Of all the implementations to follow, this one most resembles the idealized cat used as a pedagogic demonstration in K&R C. The heart of the program is the classic two-liner:
while ((c = getc(fi)) != EOF)
\nputchar(c);
\n\n\nThere is of course quite a bit more code than that, but the extra code is mostly there to ensure that you aren’t reading and writing to the same file. The other interesting thing to note is that this implementation of cat only recognized one flag, -u. The -u flag could be used to avoid buffering input and output, which cat would otherwise do in blocks of 512 bytes.
\n
\n\n\nAfter the Seventh Edition, Unix spawned all sorts of derivatives and offshoots. MacOS is built on top of Darwin, which in turn is derived from the Berkeley Software Distribution (BSD), so BSD is the Unix offshoot we are most interested in. BSD was originally just a collection of useful programs and add-ons for Unix, but it eventually became a complete operating system. BSD seems to have relied on the original cat implementation up until the fourth BSD release, known as 4BSD, when support was added for a whole slew of new flags. The 4BSD implementation of cat is clearly derived from the original implementation, though it adds a new function to implement the behavior triggered by the new flags. The naming conventions already used in the file were adhered to—the fflg variable, used to mark whether input was being read from stdin or a file, was joined by nflg, bflg, vflg, sflg, eflg, and tflg, all there to record whether or not each new flag was supplied in the invocation of the program. These were the last command-line flags added to cat; the man page for cat today lists these flags and no others, at least on Mac OS. 4BSD was released in 1980, so this set of flags is 38 years old.
\n
\ncat would be entirely rewritten a final time for BSD Net/2, which was, among other things, an attempt to avoid licensing issues by replacing all AT&T Unix-derived code with new code. BSD Net/2 was released in 1991. This final rewrite of cat was done by Kevin Fall, who graduated from Berkeley in 1988 and spent the next year working as a staff member at the Computer Systems Research Group (CSRG). Fall told me that a list of Unix utilities still implemented using AT&T code was put up on a wall at CSRG and staff were told to pick the utilities they wanted to reimplement. Fall picked cat and mknod. The cat implementation bundled with MacOS today is built from a source file that still bears his name at the very top. His version of cat, even though it is a relatively trivial program, is today used by millions.
\nFall’s original implementation of cat is much longer than anything we have seen so far. Other than support for a -? help flag, it adds nothing in the way of new functionality. Conceptually, it is very similar to the 4BSD implementation. It is only longer because Fall separates the implementation into a “raw” mode and a “cooked” mode. The “raw” mode is cat classic; it prints a file character for character. The “cooked” mode is cat with all the 4BSD command-line options. The distinction makes sense but it also pads out the implementation so that it seems more complex at first glance than it actually is. There is also a fancy error handling function at the end of the file that further adds to its length.
\n\n\nThe very first release of Mac OS X thus includes an implementation of cat pulled from the NetBSD project. So the first Mac OS X implementation of cat is Kevin Fall’s cat. The only thing that had changed over the intervening decade was that Fall’s error-handling function err() was removed and the err() function made available by err.h was used in its place. err.h is a BSD extension to the C standard library.
\n
\nThe NetBSD implementation of cat was later swapped out for FreeBSD’s implementation of cat. According to Wikipedia, Apple began using FreeBSD instead of NetBSD in Mac OS X 10.3 (Panther). But the Mac OS X implementation of cat, according to Apple’s own open source releases, was not replaced until Mac OS X 10.5 (Leopard) was released in 2007. The FreeBSD implementation that Apple swapped in for the Leopard release is the same implementation on Apple computers today. As of 2018, the implementation has not been updated or changed at all since 2007.
\nSo the Mac OS cat is old. As it happens, it is actually two years older than its 2007 appearance in MacOS X would suggest. This 2005 change, which is visible in FreeBSD’s Github mirror, was the last change made to FreeBSD’s cat before Apple pulled it into Mac OS X. So the Mac OS X cat implementation, which has not been kept in sync with FreeBSD’s cat implementation, is officially 13 years old. There’s a larger debate to be had about how much software can change before it really counts as the same software; in this case, the source file has not changed at all since 2005.
\nThe cat implementation used by Mac OS today is not that different from the implementation that Fall wrote for the 1991 BSD Net/2 release. The biggest difference is that a whole new function was added to provide Unix domain socket support. At some point, a FreeBSD developer also seems to have decided that Fall’s raw_args() function and cook_args() should be combined into a single function called scanfiles(). Otherwise, the heart of the program is still Fall’s code.
\nI asked Fall how he felt about having written the cat implementation now used by millions of Apple users, either directly or indirectly through some program that relies on cat being present. Fall, who is now a consultant and a co-author of the most recent editions of TCP/IP Illustrated, says that he is surprised when people get such a thrill out of learning about his work on cat. Fall has had a long career in computing and has worked on many high-profile projects, but it seems that many people still get most excited about the six months of work he put into rewriting cat in 1989.
\n\n\nIn the grand scheme of things, computers are not an old invention. We’re used to hundred-year-old photographs or even hundred-year-old camera footage. But computer programs are in a different category—they’re high-tech and new. At least, they are now. As the computing industry matures, will we someday find ourselves using programs that approach the hundred-year-old mark?
\n
\nComputer hardware will presumably change enough that we won’t be able to take an executable compiled today and run it on hardware a century from now. Perhaps advances in programming language design will also mean that nobody will understand C in the future and cat will have long since been rewritten in another language. (Though C has already been around for fifty years, and it doesn’t look like it is about to be replaced any time soon.) But barring all that, why not just keep using the cat we have forever?
\nI think the history of cat shows that some ideas in computer science are in fact very durable. Indeed, with cat, both the idea and the program itself are old. It may not be accurate to say that the cat on my computer is from 1969. But I could make a case for saying that the cat on my computer is from 1989, when Fall wrote his implementation of cat. Lots of other software is just as ancient. So maybe we shouldn’t think of computer science and software development primarily as fields that disrupt the status quo and invent new things. Our computer systems are built out of historical artifacts. At some point, we may all spend more time trying to understand and maintain those historical artifacts than we spend writing new code.
##News Roundup
\n###Trivial Bug in X.Org Gives Root Permission on Linux and BSD Systems
\n\n\nA vulnerability that is trivial to exploit allows privilege escalation to root level on Linux and BSD distributions using X.Org server, the open source implementation of the X Window System that offers the graphical environment.
\n
\nThe flaw is now identified as CVE-2018-14665 (credited to security researcher Narendra Shinde). It has been present in xorg-server for two years, since version 1.19.0 and is exploitable by a limited user as long as the X server runs with elevated permissions.
\n\n\nAn advisory on Thursday describes the problem as an “incorrect command-line parameter validation” that also allows an attacker to overwrite arbitrary files.
\n
\nPrivilege escalation can be accomplished via the -modulepath argument by setting an insecure path to modules loaded by the X.org server. Arbitrary file overwrite is possible through the -logfile argument, because of improper verification when parsing the option.
\n\n\nOpenBSD, the free and open-source operating system with a strong focus on security, uses xorg. On October 18, the project released version 6.4 of the OS, affected by CVE-2018-14665. This could have been avoided, though.
\n
\nTheo de Raadt, founder and leader of the OpenBSD project, says that X maintainer knew about the problem since at least October 11. For some reason, the OpenBSD developers received the message one hour before the public announcement this Thursday, a week after their new OS release.
\n“As yet we don’t have answers about why our X maintainer (on the X security team) and his team provided information to other projects (some who don’t even ship with this new X server) but chose to not give us a heads-up which could have saved all the new 6.4 users a lot of grief,” Raadt says.
\nHad OpenBSD developers known about the bug before the release, they could have taken steps to mitigate the problem or delay the launch for a week or two.
\nTo remedy the problem, the OpenBSD project provides a source code patch, which requires compiling and rebuilding the X server.
\nAs a temporary solution, users can disable the Xorg binary by running the following command:
chmod u-s /usr/X11R6/bin/Xorg
\n\n\nCVE-2018-14665 does not help compromise systems, but it is useful in the following stages of an attack.
\n
\nLeveraging it after gaining access to a vulnerable machine is fairly easy. Matthew Hickey, co-founder, and head of Hacker House security outfit created and published an exploit, saying that it can be triggered from a remote SSH session.
\nThree hours after the public announcement of the security gap, Daemon Security CEO Michael Shirk replied with one line that overwrote shadow files on the system. Hickey did one better and fit the entire local privilege escalation exploit in one line.
\nApart from OpenBSD, other operating systems affected by the bug include Debian and Ubuntu, Fedora and its downstream distro Red Hat Enterprise Linux along with its community-supported counterpart CentOS.
###OpenBSD on the Desktop: some thoughts
\n\n\n\n\nI’ve been using OpenBSD on my ThinkPad X230 for some weeks now, and the experience has been peculiar in some ways.
\n
\nThe OS itself in my opinion is not ready for widespread desktop usage, and the development team is not trying to push it in the throat of anybody who wants a Windows or macOS alternative.
\nYou need to understand a little bit of how *NIX systems work, because you’ll use CLI more than UI.
\nThat’s not necessarily bad, and I’m sure I learned a trick or two that could translate easily to Linux or macOS.
\nTheir development process is purely based on developers that love to contribute and hack around, just because it’s fun.
\nEven the mailing list is a cool place to hang on!
\nCode correctness and security are a must, nothing gets committed if it doesn’t get reviewed thoroughly first - nowadays the first two properties should be enforced in every major operating system.
\nI like the idea of a platform that continually evolves.
\npledge(2) and unveil(2) are the proof that with a little effort, you can secure existing software better than ever.
\nI like the “sensible defaults” approach, having an OS ready to be used - UI included if you selected it during the setup process - is great.
\nJust install a browser and you’re ready to go.
\nManual pages on OpenBSD are real manuals, not an extension of the “–help” command found in most CLI softwares.
\nThey help you understand inner workings of the operating system, no internet connection needed.
\nThere are some trade-offs, too.
\nPerformance is not first-class, mostly because of all the security mitigations and checks done at runtime3.
\nI write Go code in neovim, and sometimes you can feel a slight slowdown when you’re compiling and editing multiple files at the same time, but usually I can’t notice any meaningful difference.
\nBrowsers are a different matter though, you can definitely feel something differs from the experience you can have on mainstream operating systems.
\nBut again, trade-offs.
\nTo use OpenBSD on the desktop you must be ready to sacrifice some of the goodies of mainstream OSes, but if you’re searching for a zen place to do your computing stuff, it’s the best you can get right now.
\n\n\nOne of the most recent additions to the DistroWatch database is NomadBSD. According to the NomadBSD website: “NomadBSD is a 64-bit live system for USB flash drives, based on FreeBSD. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery.”
\n
\nThe latest release of NomadBSD (or simply “Nomad”, as I will refer to the project in this review) is version 1.1. It is based on FreeBSD 11.2 and is offered in two builds, one for generic personal computers and one for Macbooks. The release announcement mentions version 1.1 offers improved video driver support for Intel and AMD cards. The operating system ships with Octopkg for graphical package management and the system should automatically detect, and work with, VirtualBox environments.
\nNomad 1.1 is available as a 2GB download, which we then decompress to produce a 4GB file which can be written to a USB thumb drive. There is no optical media build of Nomad as it is designed to be run entirely from the USB drive, and write data persistently to the drive, rather than simply being installed from the USB media.
\n\n\nBooting from the USB drive brings up a series of text-based menus which ask us to configure key parts of the operating system. We are asked to select our time zone, keyboard layout, keyboard model, keyboard mapping and our preferred language. While we can select options from a list, the options tend to be short and cryptic. Rather than “English (US)”, for example, we might be given “en_US”. We are also asked to create a password for the root user account and another one for a regular user which is called “nomad”. We can then select which shell nomad will use. The default is zsh, but there are plenty of other options, including csh and bash. We have the option of encrypting our user’s home directory.
\n
\nI feel it is important to point out that these settings, and nomad’s home directory, are stored on the USB drive. The options and settings we select will not be saved to our local hard drive and our configuration choices will not affect other operating systems already installed on our computer. At the end, the configuration wizard asks if we want to run the BSDstats service. This option is not explained at all, but it contacts BSDstats to provide some basic statistics on BSD users.
\nThe system then takes a few minutes to apply its changes to the USB drive and automatically reboots the computer. While running the initial setup wizard, I had nearly identical experiences when running Nomad on a physical computer and running the operating system in a VirtualBox virtual machine. However, after the initial setup process was over, I had quite different experiences depending on the environment so I want to divide my experiences into two different sections.
\n\n\nAt first, Nomad failed to boot on my desktop computer. From the operating system’s boot loader, I enabled Safe Mode which allowed Nomad to boot. At that point, Nomad was able to start up, but would only display a text console. The desktop environment failed to start when running in Safe Mode.
\n
\nNetworking was also disabled by default and I had to enable a network interface and DHCP address assignment to connect to the Internet. Instructions for enabling networking can be found in FreeBSD’s Handbook. Once we are on-line we can use the pkg command line package manager to install and update software. Had the desktop environment worked then the Octopkg graphical package manager would also be available to make browsing and installing software a point-n-click experience.
\nHad I been able to run the desktop for prolonged amounts of time I could have made use of such pre-installed items as the Firefox web browser, the VLC media player, LibreOffice and Thunderbird. Nomad offers a fairly small collection of desktop applications, but what is there is mostly popular, capable software.
\nWhen running the operating system I noted that, with one user logged in, Nomad only runs 15 processes with the default configuration. These processes require less than 100MB of RAM, and the whole system fits comfortably on a 4GB USB drive.
\n\n\nUltimately using Nomad was not a practical option for me. The operating system did not work well with my hardware, or the virtual environment. In the virtual machine, Nomad crashed consistently after just a few minutes of uptime. On the desktop computer, I could not get a desktop environment to run. The command line tools worked well, and the system performed tasks very quickly, but a command line only environment is not well suited to my workflow.
\n
\nI like the idea of what NomadBSD is offering. There are not many live desktop flavours of FreeBSD, apart from GhostBSD. It was nice to see developers trying to make a FreeBSD-based, plug-and-go operating system that would offer a desktop and persistent storage. I suspect the system would work and perform its stated functions on different hardware, but in my case my experiment was necessarily short lived.
##Beastie Bits
\n\n##Feedback/Questions
\n\nByproducts of reading OpenBSD’s netcat code, learnings from porting your own projects to FreeBSD, OpenBSD’s unveil(), NetBSD’s Virtual Machine Monitor, what 'dependency' means in Unix init systems, jailing bhyve, and more.
\n
##Headlines
###The byproducts of reading OpenBSD netcat code
When I took part in a training last year, I heard about netcat for the first time. During that class, the tutor showed some hacks and tricks of using netcat which appealed to me and motivated me to learn the guts of it. Fortunately, in the past 2 months, I was not so busy that I can spend my spare time to dive into OpenBSD‘s netcat source code, and got abundant byproducts during this process.
(1) Brush up socket programming. I wrote my first network application more than 10 years ago, and always think the socket APIs are marvelous. Just ~10 functions (socket, bind, listen, accept…) with some IO multiplexing buddies (select, poll, epoll…) connect the whole world, wonderful! From that time, I developed a habit that is when touching a new programming language, network programming is an essential exercise. Even though I don’t write socket related code now, reading netcat socket code indeed refresh my knowledge and teach me new stuff.
(2) Write a tutorial about netcat. I am mediocre programmer and will forget things when I don’t use it for a long time. So I just take notes of what I think is useful. IMHO, this “tutorial” doesn’t really mean teach others something, but just a journal which I can refer when I need in the future.
(3) Submit patches to netcat. During reading code, I also found bugs and some enhancements. Though trivial contributions to OpenBSD, I am still happy and enjoy it.
(4) Implement a C++ encapsulation of libtls. OpenBSD‘s netcat supports tls/ssl connection, but it needs you take full care of resource management (memory, socket, etc), otherwise a small mistake can lead to resource leak which is fatal for long-live applications (In fact, the two bugs I reported to OpenBSD are all related resource leak). Therefore I develop a simple C++ library which wraps the libtls and hope it can free developer from this troublesome problem and put more energy in application logic part.
Long story to short, reading classical source code is a rewarding process, and you can consider to try it yourself.
###What I learned from porting my projects to FreeBSD
I set up a local FreeBSD VirtualBox VM to test something, and it seems to work very well. Due to the novelty factor, I decided to get my software projects to build and pass the tests there.
The Projects
https://github.com/shlomif/shlomif-computer-settings/ (my dotfiles).
https://www.shlomifish.org/open-source/projects/black-hole-solitaire-solver/
Written using a mix of C, Perl 5, Python, Ruby, GNU Bash, XML, CMake, XSLT, XHTML5, XHTML1.1, Website META Language, JavaScript and more.
Work fine on several Linux distributions and have https://en.wikipedia.org/wiki/Travis_CI using Ubuntu 14.04 hosts
Some pass builds and tests on AppVeyor/Win64
What I Learned:
FreeBSD on VBox has become very reliable
Some executables on FreeBSD are in /usr/local/bin instead of /usr/bin
make on FreeBSD is not GNU make
m4 on FreeBSD is not compatible with GNU m4
Some CPAN Modules fail to install using local-lib there
DocBook/XSL Does Not Live Under /usr/share/sgml
FreeBSD’s grep does not have a “-P” flag by default
FreeBSD has no “nproc” command
Conclusion:
It is easier to port a shell than a shell script. — Larry Wall
I ran into some cases where my scriptology was lacking and suboptimal, even for my own personal use, and fixed them.
##News Roundup
###OpenBSD’s unveil()
One of the key aspects of hardening the user-space side of an operating system is to provide mechanisms for restricting which parts of the filesystem hierarchy a given process can access. Linux has a number of mechanisms of varying capability and complexity for this purpose, but other kernels have taken a different approach. Over the last few months, OpenBSD has inaugurated a new system call named unveil() for this type of hardening that differs significantly from the mechanisms found in Linux.
The value of restricting access to the filesystem, from a security point of view, is fairly obvious. A compromised process cannot exfiltrate data that it cannot read, and it cannot corrupt files that it cannot write. Preventing unwanted access is, of course, the purpose of the permissions bits attached to every file, but permissions fall short in an important way: just because a particular user has access to a given file does not necessarily imply that every program run by that user should also have access to that file. There is no reason why your PDF viewer should be able to read your SSH keys, for example. Relying on just the permission bits makes it easy for a compromised process to access files that have nothing to do with that process’s actual job.
In a Linux system, there are many ways of trying to restrict that access; that is one of the purposes behind the Linux security module (LSM) architecture, for example. The SELinux LSM uses a complex matrix of labels and roles to make access-control decisions. The AppArmor LSM, instead, uses a relatively simple table of permissible pathnames associated with each application; that approach was highly controversial when AppArmor was first merged, and is still looked down upon by some security developers. Mount namespaces can be used to create a special view of the filesystem hierarchy for a set of processes, rendering much of that hierarchy invisible and, thus, inaccessible. The seccomp mechanism can be used to make decisions on attempts by a process to access files, but that approach is complex and error-prone. Yet another approach can be seen in the Qubes OS distribution, which runs applications in virtual machines to strictly control what they can access.
Compared to many of the options found in Linux, unveil() is an exercise in simplicity. This system call, introduced in July, has this prototype:
int unveil(const char *path, const char *permissions);
A process that has never called unveil() has full access to the filesystem hierarchy, modulo the usual file permissions and any restrictions that may have been applied by calling pledge(). Calling unveil() for the first time will “drop a veil” across the entire filesystem, rendering the whole thing invisible to the process, with one exception: the file or directory hierarchy starting at path will be accessible with the given permissions. The permissions string can contain any of “r” for read access, “w” for write, “x” for execute, and “c” for the ability to create or remove the path.
Subsequent calls to unveil() will make other parts of the filesystem hierarchy accessible; the unveil() system call itself still has access to the entire hierarchy, so there is no problem with unveiling distinct subtrees that are, until the call is made, invisible to the process. If one unveil() call applies to a subtree of a hierarchy unveiled by another call, the permissions associated with the more specific call apply.
Calling unveil() with both arguments as null will block any further calls, setting the current view of the filesystem in stone. Calls to unveil() can also be blocked using pledge(). Either way, once the view of the filesystem has been set up appropriately, it is possible to lock it so that the process cannot expand its access in the future should it be taken over and turn hostile.
unveil() thus looks a bit like AppArmor, in that it is a path-based mechanism for restricting access to files. In either case, one must first study the program in question to gain a solid understanding of which files it needs to access before closing things down, or the program is likely to break. One significant difference (beyond the other sorts of behavior that AppArmor can control) is that AppArmor’s permissions are stored in an external policy file, while unveil() calls are made by the application itself. That approach keeps the access rules tightly tied to the application and easy for the developers to modify, but it also makes it harder for system administrators to change them without having to rebuild the application from source.
One can certainly aim a number of criticisms at unveil() — all of the complaints that have been leveled at path-based access control and more. But the simplicity of unveil() brings a certain kind of utility, as can be seen in the large number of OpenBSD applications that are being modified to use it. OpenBSD is gaining a base level of protection against unintended program behavior; while it is arguably possible to protect a Linux system to a much greater extent, the complexity of the mechanisms involved keeps that from happening in a lot of real-world deployments. There is a certain kind of virtue to simplicity in security mechanisms.
###NetBSD Virtual Machine Monitor (NVVM)
The NVMM driver provides hardware-accelerated virtualization support on NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is provided in libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.
The source code of NVMM, plus the associated tools, can be downloaded here.
NVMM can support up to 128 virtual machines, each having a maximum of 256 VCPUs and 4GB of RAM.
Each virtual machine is granted access to most of the CPU registers: the GPRs (obviously), the Segment Registers, the Control Registers, the Debug Registers, the FPU (x87 and SSE), and several MSRs.
Events can be injected in the virtual machines, to emulate device interrupts. A delay mechanism is used, and allows VMM software to schedule the interrupt right when the VCPU can receive it. NMIs can be injected as well, and use a similar mechanism.
The host must always be x86_64, but the guest has no constraint on the mode, so it can be x86_32, PAE, real mode, and so on.
The TSC of each VCPU is always re-based on the host CPU it is executing on, and is therefore guaranteed to increase regardless of the host CPU. However, it may not increase monotonically, because it is not possible to fully hide the host effects on the guest during #VMEXITs.
When there are more VCPUs than the host TLB can deal with, NVMM uses a shared ASID, and flushes the shared-ASID VCPUs on each VM switch.
The different intercepts are configured in such a way that they cover everything that needs to be emulated. In particular, the LAPIC can be emulated by VMM software, by intercepting reads/writes to the LAPIC page in memory, and monitoring changes to CR8 in the exit state.
###What ‘dependency’ means in Unix init systems is underspecified (utoronto.ca)
I was reading Davin McCall’s On the vagaries of init systems (via) when I ran across the following, about the relationship between various daemons (services, etc):
I do not see any compelling reason for having ordering relationships without actual dependency, as both Nosh and Systemd provide for. In comparison, Dinit’s dependencies also imply an ordering, which obviates the need to list a dependency twice in the service description.
Well, this may be an easy one but it depends on what an init system means by ‘dependency’. Let’s consider ®syslog and the SSH daemon. I want the syslog daemon to be started before the SSH daemon is started, so that the SSH daemon can log things to it from the beginning. However, I very much do not want the SSH daemon to not be started (or to be shut down) if the syslog daemon fails to start or later fails. If syslog fails, I still want the SSH daemon to be there so that I can perhaps SSH in to the machine and fix the problem.
This is generally true of almost all daemons; I want them to start after syslog, so that they can syslog things, but I almost never want them to not be running if syslog failed. (And if for some reason syslog is not configured to start, I want enabling and starting, say, SSH, to also enable and start the syslog daemon.)
In general, there are three different relationships between services that I tend to encounter:
a hard requirement, where service B is useless or dangerous without service A. For instance, many NFS v2 and NFS v3 daemons basically don’t function without the RPC portmapper alive and active. On any number of systems, firewall rules being in place are a hard requirement to start most network services; you would rather your network services not start at all than that they start without your defenses in place.
a want, where service B wants service A to be running before B starts up, and service A should be started even if it wouldn’t otherwise be, but the failure of A still leaves B functional. Many daemons want the syslog daemon to be started before they start but will run without it, and often you want them to do so so that at least some of the system works even if there is, say, a corrupt syslog configuration file that causes the daemon to error out on start. (But some environments want to hard-fail if they can’t collect security related logging information, so they might make rsyslogd a requirement instead of a want.)
an ordering, where if service A is going to be started, B wants to start after it (or before it), but B isn’t otherwise calling for A to be started. We have some of these in our systems, where we need NFS mounts done before cron starts and runs people’s @reboot jobs but neither cron nor NFS mounts exactly or explicitly want each other. (The system as a whole wants both, but that’s a different thing.)
Given these different relationships and the implications for what the init system should do in different situations, talking about ‘dependency’ in it systems is kind of underspecified. What sort of dependency? What happens if one service doesn’t start or fails later?
My impression is that generally people pick a want relationship as the default meaning for init system ‘dependency’. Usually this is fine; most services aren’t actively dangerous if one of their declared dependencies fails to start, and it’s generally harmless on any particular system to force a want instead of an ordering relationship because you’re going to be starting everything anyway.
###Jailing The bhyve Hypervisor
As FreeBSD nears the final 12.0-RELEASE release engineering cycles, I’d like to take a moment to document a cool new feature coming in 12: jailed bhyve.
You may notice that I use HardenedBSD instead of FreeBSD in this article. There is no functional difference in bhyve on HardenedBSD versus bhyve on FreeBSD. The only difference between HardenedBSD and FreeBSD is the aditional security offered by HardenedBSD.
The steps I outline here work for both FreeBSD and HardenedBSD. These are the bare minimum steps, no extra work needed for either FreeBSD or HardenedBSD.
At work in my spare time, I’m helping develop a malware lab. Due to the nature of the beast, we would like to use bhyve on HardenedBSD. Starting with HardenedBSD 12, non-Cross-DSO CFI, SafeStack, Capsicum, ASLR, and strict WX are all applied to bhyve, making it an extremely hardened hypervisor.
So, the work to support jailed bhyve is sponsored by both HardenedBSD and my employer. We’ve also jointly worked on other bhyve hardening features, like protecting the VM’s address space using guard pages (mmap(…, MAP_GUARD, …)). Further work is being done in a project called “malhyve.” Only those modifications to bhyve/malhyve that make sense to upstream will be upstreamed.
We will not go through the process of creating the jail’s filesystem. That process is documented in the FreeBSD Handbook. For UEFI guests, you will need to install the uefi-edk2-bhyve package inside the jail.
I network these jails with traditional jail networking. I have tested vnet jails with this setup, and that works fine, too. However, there is no real need to hook the jail up to any network so long as bhyve can access the tap device. In some cases, the VM might not need networking, in which case you can use a network-less VM in a network-less jail.
By default, access to the kernel side of bhyve is disabled within jails. We need to set allow.vmm in our jail.conf entry for the bhyve jail.
We will use the following in our jail, so we will need to set up devfs(8) rules for them:
A ZFS volume
A null-modem device (nmdm(4))
UEFI GOP (no devfs rule, but IP assigned to the jail)
A tap device
Conclusion
The bhyve hypervisor works great within a jail. When combined with HardenedBSD, bhyve is extremely hardened:
Bad guys are going to have a hard time breaking out of the userland components of bhyve on HardenedBSD. :)
##Beastie Bits
##Feedback/Questions
MidnightBSD 1.0 released, MeetBSD review, EuroBSDcon trip reports, DNS over TLS in FreeBSD 12, Upgrading OpenBSD with Ansible, how to use smartd to run tests on your drives automatically, and more.
\n\n##Headlines
\n###MidnightBSD 1.0 now available
\n\n\nI’m happy to announce the availability of MidnightBSD 1.0 for amd64 and i386. Over the years, many ambitious goals were set for our 1.0 release. As it approached, it was clear we wouldn’t be able to accomplish all of them. This release is more of a natural progression rather than a groundbreaking event. It includes many updates to the base system, improvements to the package manager, an updated compiler, and tools.
\n
\nOf particular note, you can now boot off of ZFS and use NVME SSDs and some AMD Radeon graphics cards support acceleration. AMD Ryzen support has greatly improved in this release. We also have added bhyve from FreeBSD.
\nThe 1.0 release is finally available. Still building packages for i386 and plan to do an amd64 package build later in the week. The single largest issue with the release process has been the web server performance. The CPU is overloaded and has been at solid 100% for several days. The server has a core i7 7700 in it. I’m trying to figure out what to buy as an upgrade so that we don’t continue to have this issue going forward. As it’s actually blocked in multiple processes, a 6 or 8 core chip might be an improvement for the workload…
\n\n\nMeetBSD 2018 took place at the sprawling Intel Santa Clara campus. The venue itself felt more like an olive branch than a simple friendly gesture by Intel. In truth it felt like a bit of an apology. You get the subtle sense they feel bad about how the BSD’s were treated with the Meltdown and Specter flaws. In fact, you may be right to think they felt a bit sorry towards the entire open source community.
\n
\n\n\nAt most massive venues the parking is the first concern, not so here - in fact that was rather straightforward. No, the real challenge is navigating the buildings. Luckily I had help from navigator extraordinaire, Hadea, who located the correct building, SC12 quickly. Finding the entrance took a moment or two though. The lobby itself was converted by iXsystems efficiently into the MeetBSD expo hall, clean, efficient and roomy with registration, some seating, and an extra conference room for on-on-one sessions. On day two sponsor booths were also setup. All who showed up on day one were warmly greeted with badges, lanyards and goodies by Denise and her friendly team.
\n
\nLike every great BSD event, plenty of food was made available. And as always they make it look effortless. These events showcase iXsystem’s inherent generosity toward its community; with breakfast items in the back of the main auditorium room in the morning, boxed lunches, fruit and cookies at lunch time, and snacks for the rest of the day. But just in case your still hungry, there is a pizza meetup in another Intel room after day one and two.
\nMeetBSD leverages it’s realistically small crowd size on day one. The morning starts off with introductions of the entire group, the mic is passed around the room.
\nThe group is a good mix of pros in the industry (such as Juniper, Intel, Ebay, Groupon, Cisco, etc), iX staff, and a few enthusiast. Lots of people with a focus or passion for networking. And, of course, some friendly Linux bashing went down for good measure, always followed by a good natured chuckle.
\n\n\nI find that I am subtly unnerved at this venue, and at lunch I saw it clearly. I have always had a strong geek radar, allowing me to navigate a new area (like Berkeley for MeetBSD of 2016, or even SCALE earlier this year in Pasadena), and in a glance I can see who is from my conference and who isn’t. This means it is easy, nearly effortless to know who to greet with a smile and a wave. These are MY people. Here at the Intel campus though it is different. The drive in alone reveals behemoth complexes all with well known tech names prominently displayed. This is Silicon Valley, and all of these people look like MY people. So much for knowing who’s from my conference. Thank goodness for those infamous BSD horns. None-the-less I am struck by how massive these tech giants are. And Intel is one of the largest of those giants, and see the physical reminders of this fact brought home the significance that they had opened their doors, wifi, and bathrooms to the BSD community.
\n
###[EuroBSDcon 2018 Trip Reports]
\nhttps://www.freebsdfoundation.org/blog/eurobsd-2018-trip-report-joseph-mingrone/
\nhttps://www.freebsdfoundation.org/blog/eurobsd-2018-trip-report-vinicius-zavam/
\nhttps://www.freebsdfoundation.org/blog/eurobsd-2018-trip-report-emmanuel-vadot/
##News Roundup
\n###DNS over TLS in FreeBSD 12
\n\n\nWith the arrival of OpenSSL 1.1.1, an upgraded Unbound, and some changes to the setup and init scripts, FreeBSD 12.0, currently in beta, now supports DNS over TLS out of the box.
\n
\nDNS over TLS is just what it sounds like: DNS over TCP, but wrapped in a TLS session. It encrypts your requests and the server’s replies, and optionally allows you to verify the identity of the server. The advantages are protection against eavesdropping and manipulation of your DNS traffic; the drawbacks are a slight performance degradation and potential firewall traversal issues, as it runs over a non-standard port (TCP port 853) which may be blocked on some networks. Let’s take a look at how to set it up.
\n\n\nWe’ve seen how to set up Unbound—specifically, the local_unbound service in FreeBSD 12.0—to use DNS over TLS instead of plain UDP or TCP, using Cloudflare’s public DNS service as an example. We’ve looked at the performance impact, and at how to ensure (and verify) that Unbound validates the server certificate to prevent man-in-the-middle attacks.
\n
\nThe question that remains is whether it is all worth it. There is undeniably a performance hit, though this may improve with TLS 1.3. More importantly, there are currently very few DNS-over-TLS providers—only one, really, since Quad9 filter their responses—and you have to weigh the advantage of encrypting your DNS traffic against the disadvantage of sending it all to a single organization. I can’t answer that question for you, but I can tell you that the parameters are evolving quickly, and if your answer is negative today, it may not remain so for long. More providers will appear. Performance will improve with TLS 1.3 and QUIC. Within a year or two, running DNS over TLS may very well become the rule rather than the experimental exception.
###Upgrading OpenBSD with Ansible
\n\n\n\n\nA few months ago, I needed software that had just hit the ports tree. I didn’t want to wait for the next release, so I upgraded my router to use -current. Since then, I’ve continued running -current, which means upgrading to a newer snapshot every so often. Running -current is great, but the process of updating to a newer snapshot was cumbersome. Initially, I had to plug in a serial cable and then reboot into bsd.rd, hit enter ten times, then reboot, run sysmerge and update packages.
\n
\nI eventually switched to upobsd to be able to upgrade without the need for a serial connection. The process was better, but still tiresome. Usually, I would prepare the special version of bsd.rd, boot on bsd.rd, and do something like wash the dishes in the meantime. After about ten minutes, I would dry my hands and then go back to my workstation to see whether the bsd.rd part had finished so I could run sysmerge and pkg_add, and then return to the dishes while it upgraded packages.
\nOut of laziness, I thought: “I should automate this,” but what happened instead is that I simply didn’t upgrade that machine very often. (Yes, laziness). With my router out of commission, life is very dull, because it is my gateway to the Internet. Even services hosted at my place (like my Mastodon instance) are not reachable when the router is down because I use multiple VLANs (so I need the router to jump across VLANs).
\n\n\nI recently got a new job, and one of my first tasks was auditing the Ansible roles written by my predecessors. In one role, the machine rebooted and they used the wait_for_connection module to wait for it to come back up. That sounded quite hackish to me, so out of curiosity, I tried to determine whether there was a better way. I also thought I might be able to use something similar to further automate my OpenBSD upgrades, and wanted to assess the cleanliness of this method. ;-)
\n
\nI learned that with the then-upcoming 2.7 Ansible release, a proper reboot module would be included. I went to the docs, which stated that for a certain parameter:
\nI took this to mean that there was no support for OpenBSD. I looked at the code and, indeed, there was not. However, I believed that it wouldn’t be too hard to add it. I added the missing pieces for OpenBSD, tested it on my poor Pine64 and then submitted it upstream. After a quick back and forth, the module’s author merged it into devel (having a friend working at Red Hat helped the process, merci Cyril !) A couple days later, the release engineer merged it into stable-2.7.
\nI proceeded to actually write the playbook, and then I hit a bug. The parameter reboot_timeout was not recognized by Ansible. This feature would definitely be useful on a slow machine (such as the Pine64 and its dying SD card). Again, my fix was merged into master by the module’s author and then merged into stable-2.7. 2.7.1 will be the first release to feature these fixes, but if you use OpenBSD -current, you already have access to them. I backported the patches when I updated ansible.
\nFun fact about Ansible and reboots: “The win_reboot module was […] included with Ansible 2.1,” while for unix systems it wasn’t added until 2.7. :D For more details, you can read the module’s author blog article.
\n\n\nAnsible runs my script on the remote host to fetch the sets. It creates an answer file from the template and then gives it to upobsd. Once upobsd has created the kernel, Ansible copies it in place of /bsd on the host. The router reboots and boots on /bsd, which is upobsd’s bsd.rd. The installer runs in auto_update mode. Once it comes back from bsd.rd land, it archives the kernel and finishes by upgrading all the packages.
\n
\nIt also supports upgrading without fetching the sets ahead of time. For instance, I upgrade this way on my Pine64 because if I cared about speed, I wouldn’t use this weak computer with its dying SD card. For this case, I just comment out the path_sets variable and Ansible instead creates an answer file that will instruct the installer to fetch the sets from the designated mirror.
\nI’ve been archiving my kernels for a few years. It’s a nice way to fill up / keep a history of my upgrades. If I spot a regression, I can try a previous kernel … which may not work with the then-desynchronized userland, but that’s another story.
\nsysmerge already runs with rc.sysmerge in batch mode and sends the result by email. I don’t think there’s merit to running it again in the playbook. The only perk would be discovering in the terminal whether any files need to be manually merged, rather than reading exactly the same output in the email.
\nInitially, I used the openbsd_pkg module, but it doesn’t work on -current just before a release because pkg_add automatically looks for pub/OpenBSD/${release}/packages/${arch} (which is empty). I wrote and tested this playbook while 6.4 was around the corner, so I switched to command to be able to pass the -Dsnap parameter.
\n\n\nI’m very happy with the playbook! It performs the upgrade with as little intervention as possible and minimal downtime. \\o/
\n
###Using smartd to automatically run tests on your drives
\n\n\n\n\nThose programs can “control and monitor storage systems using the Self-Monitoring, Analysis and Reporting Technology System (SMART) built into most modern ATA/SATA, SCSI/SAS and NVMe disks. In many cases, these utilities will provide advanced warning of disk degradation and failure.” See the smartmontools website for more information.
\n
\n\n\nNOTE: “Due to OS-specific issues and also depending on the different state of smartmontools development on the platforms, device support is not the same for all OS platforms.” – use the documentation for your OS.
\n
\n\n\nI first started using smartd in March 2010 (according to that blog post, that’s when I still writing on both The FreeBSD Diary and this blog). Back then, and until recently, all I did was start smartd. As far as I can tell, all it did was send daily status messages via the FreeBSD periodic tools. I would set my drive devices via daily_status_smart_devices in /etc/periodic.conf and the daily status reports would include drive health information.
\n
##Beastie Bits
\n\n##Feedback/Questions
\n\nOpenBSD 6.4 released, GhostBSD RC2 released, MeetBSD - the ultimate hallway track, DragonflyBSD desktop on a Thinkpad, Porting keybase to NetBSD, OpenSSH 7.9, and draft-ietf-6man-ipv6only-flag in FreeBSD.
\n\n##Headlines
\n###OpenBSD 6.4 released
###GhostBSD 18.10 RC2 Announced
\n\n\n\n\nThis second release candidate of GhostBSD 18.10 is the second official release of GhostBSD with TrueOS under the hood. The official desktop of GhostBSD is MATE. However, in the future, there might be an XFCE community release, but for now, there is no community release yet.
\n
What has changed since RC1
\nRemoved drm-stable-kmod and we will let users installed the propper drm-*-kmod
\nDouglas Joachin added libva-intel-driver libva-vdpau-driver to supports accelerated some video driver for Intel
\nIssues that got fixed
\nBug #70 Cannot run Octopi, missing libgksu error.
\nBug #71 LibreOffice doesn’t start because of missing libcurl.so.4
\nBug #72 libarchive is a missing dependency
\n\n\n\nAgain thanks to iXsystems, TrueOS, Joe Maloney, Kris Moore, Ken Moore, Martin Wilke, Neville Goddard, Vester “Vic” Thacker, Douglas Joachim, Alex Lyakhov, Yetkin Degirmenci and many more who helped to make the transition from FreeBSD to TrueOS smoother.
\n
Updating from RC1 to RC2:
\nsudo pkg update -f
\nsudo pkg install -f libarchive curl libgksu
\nsudo pkg upgrade
\nWhere to download:
\nAll images checksum, hybrid ISO(DVD, USB) and torrent are available here: https://www.ghostbsd.org/download
\n[ScreenShots]
\nhttps://www.ghostbsd.org/sites/default/files/Screenshot_at_2018-10-20_13-22-41.png
\nhttps://www.ghostbsd.org/sites/default/files/Screenshot_at_2018-10-20_13-27-26.png
\n###OpenSSH 7.9 has been released and it has support for OpenSSL 1.1
\n\nChanges since OpenSSH 7.8\n=========================\n\nThis is primarily a bugfix release.\n\nNew Features\n------------\n * ssh(1), sshd(8): allow most port numbers to be specified using\n service names from getservbyname(3) (typically /etc/services).\n * ssh(1): allow the IdentityAgent configuration directive to accept\n environment variable names. This supports the use of multiple\n agent sockets without needing to use fixed paths.\n * sshd(8): support signalling sessions via the SSH protocol.\n A limited subset of signals is supported and only for login or\n command sessions (i.e. not subsystems) that were not subject to\n a forced command via authorized_keys or sshd_config. bz#1424\n * ssh(1): support "ssh -Q sig" to list supported signature options.\n Also "ssh -Q help" to show the full set of supported queries.\n * ssh(1), sshd(8): add a CASignatureAlgorithms option for the\n client and server configs to allow control over which signature\n formats are allowed for CAs to sign certificates. For example,\n this allows banning CAs that sign certificates using the RSA-SHA1\n signature algorithm.\n * sshd(8), ssh-keygen(1): allow key revocation lists (KRLs) to\n revoke keys specified by SHA256 hash.\n * ssh-keygen(1): allow creation of key revocation lists directly\n from base64-encoded SHA256 fingerprints. This supports revoking\n keys using only the information contained in sshd(8)\n authentication log messages.\n\nBugfixes\n--------\n\n * ssh(1), ssh-keygen(1): avoid spurious "invalid format" errors when\n attempting to load PEM private keys while using an incorrect\n passphrase. bz#2901\n * sshd(8): when a channel closed message is received from a client,\n close the stderr file descriptor at the same time stdout is\n closed. This avoids stuck processes if they were waiting for\n stderr to close and were insensitive to stdin/out closing. bz#2863\n * ssh(1): allow ForwardX11Timeout=0 to disable the untrusted X11\n forwarding timeout and support X11 forwarding indefinitely.\n Previously the behaviour of ForwardX11Timeout=0 was undefined.\n * sshd(8): when compiled with GSSAPI support, cache supported method\n OIDs regardless of whether GSSAPI authentication is enabled in the\n main section of sshd_config. This avoids sandbox violations if\n GSSAPI authentication was later enabled in a Match block. bz#2107\n * sshd(8): do not fail closed when configured with a text key\n revocation list that contains a too-short key. bz#2897\n * ssh(1): treat connections with ProxyJump specified the same as\n ones with a ProxyCommand set with regards to hostname\n canonicalisation (i.e. don't try to canonicalise the hostname\n unless CanonicalizeHostname is set to 'always'). bz#2896\n * ssh(1): fix regression in OpenSSH 7.8 that could prevent public-\n key authentication using certificates hosted in a ssh-agent(1)\n or against sshd(8) from OpenSSH <7.8.\n\nPortability\n-----------\n\n * All: support building against the openssl-1.1 API (releases 1.1.0g\n and later). The openssl-1.0 API will remain supported at least\n until OpenSSL terminates security patch support for that API version.\n * sshd(8): allow the futex(2) syscall in the Linux seccomp sandbox;\n apparently required by some glibc/OpenSSL combinations.\n * sshd(8): handle getgrouplist(3) returning more than\n _SC_NGROUPS_MAX groups. Some platforms consider this limit more\n as a guideline.\n
\n\n##News Roundup
\n\n###MeetBSD 2018: The Ultimate Hallway Track
\n\n\n\n\nFounded in Poland in 2007 and first hosted in California in 2008, MeetBSD combines formal talks with UnConference activities to provide a level of interactivity not found at any other BSD conference. The character of each MeetBSD is determined largely by its venue, ranging from Hacker Dojo in 2010 to Intel’s Santa Clara headquarters this year. The Intel SC12 building provided a beautiful auditorium and sponsors’ room, plus a cafeteria for the Friday night social event and the Saturday night FreeBSD 25th Anniversary Celebration. The formal nature of the auditorium motivated the formation of MeetBSD’s first independent Program Committee and public Call for Participation. Together these resulted in a backbone of talks presented by speakers from the USA, Canada, and Poland, combined with UnConference activities tailored to the space.
\n
\n\n\nDay Zero of MeetBSD was a FreeBSD Developer/Vendor Summit hosted in the same auditorium where the talks would take place. Like the conference itself, this event featured a mix of scheduled talks and interactive sessions. The scheduled talks were LWPMFS: LightWeight Persistent Memory Filesystem by Ravi Pokala, Evaluating GIT for FreeBSD by Ed Maste, and NUMA by Mark Johnston. Ed’s overview of the advantages and disadvantages of using Git for FreeBSD development was of the most interest to users and developers, and the discussion continued into the following two days.
\n
\n\n\nThe first official day of MeetBSD 2018 was kicked off with introductions led by emcee JT Pennington and a keynote, “Using TrueOS to boot-strap your FreeBSD-based project” by Kris Moore. Kris described a new JSON-based release infrastructure that he has exercised with FreeBSD, TrueOS, and FreeNAS. Kris’ talk was followed by “Intel & FreeBSD: Better Together” by Ben Widawsky, the FreeBSD program lead at Intel, who gave an overview of Intel’s past and current efforts supporting FreeBSD. Next came lunch, followed by Kamil Rytarowski’s “Bug detecting software in the NetBSD userland: MKSANITIZER”. This was followed by 5-Minute Lightning Talks, Andrew Fengler’s “FreeBSD: What to (Not) Monitor”, and an OpenZFS Panel Discussion featuring OpenZFS experts Michael W. Lucas, Allan Jude, Alexander Motin, Pawel Dawidek, and Dan Langille. Day one concluded with a social event at the Intel cafeteria where the discussions continued into the night.
\n
\n\n\nDay Two of MeetBSD 2018 kicked off with a keynote by Michael W. Lucas entitled “Why BSD?”, where Michael detailed what makes the BSD community different and why it attracts us all. This was followed by Dr. Kirk McKusick’s “The Early Days of BSD” talk, which was followed by “DTrace/dwatch in Production” by Devin Teske. After lunch, we enjoyed “A Curmudgeon’s Language Selection Criteria: Why I Don’t Write Everything in Go, Rust, Elixir, etc” by G. Clifford Williams and, “Best practices of sandboxing applications with Capsicum” by Mariusz Zaborski. I then hosted a Virtualization Panel Discussion that featured eight developers from FreeBSD, OpenBSD, and NetBSD. We then split up for Breakout Sessions and the one on Bloomberg’s controversial article on backdoored Supermicro systems was fascinating given the experts present, all of whom were skeptical of the feasibility of the attack. The day wrapped up with a final talk, “Tales of a Daemontown Performance Peddler: Why ‘it depends’ and what you can do about it” by Nick Principe, followed by the FreeBSD 25th Anniversary Celebration.
\n
\n\n\nI confess the other organizers and I were nervous about how well one large auditorium would suit a BSD event but the flexible personal space it gave everyone allowed for countless meetings and heated hacking that often brought about immediate results. I watched people take ideas through several iterations with the help and input of obvious and unexpected experts, all of whom were within reach. Not having to pick up and leave for a talk in another room organically resulted in essentially a series of mini hackathons that none of us anticipated but were delighted to witness, taking the “hallway track” to a whole new level. The mix of formal and UnConference activities at MeetBSD is certain to evolve. Thank you to everyone who participated with questions, Lightning Talks, and Panel participation. A huge thanks to our sponsors, including Intel for both hosting and sponsoring MeetBSD California 2018, Western Digital, Supermicro, Verisign, Jupiter Broadcasting, the FreeBSD Foundation, Bank of America Merrill Lynch, the NetBSD Foundation, and the team at iXsystems.
\n
\n\n\nSee you at MeetBSD 2020!
\n
###Setup DragonflyBSD with a desktop on real hardware ThinkPad T410
\n+Video Demo
\n\n\nLinux has become too mainstream and standard BSD is a common thing now? How about DragonflyBSD which was created as a fork of FreeBSD 4.8 in conflict over system internals. This tutorial will show how to install it and set up a user-oriented desktop. It should work with DragonflyBSD, FreeBSD and probably all BSDs.
\n
\nSome background: BSD was is ultimately derived from UNIX back in the days. It is not Linux even though it is similar in many ways because Linux was designed to follow UNIX principles. Seeing is believing, so check out the video of the install!
\nI did try two BSD distros before called GhostBSD and TrueOS and you can check out my short reviews. DragonflyBSD comes like FreeBSD bare bones and requires some work to get a desktop running.
Download image file and burn to USB drive or DVD
\nFirst installation
\nSetting up the system and installing a desktop
\nInside the desktop
\nInstall some more programs
\nHow to enable sound?
\nLet’s play some free games
\nSetup WiFi
\nPower mode settings
\nMore to do?
\n\n\n\nYou can check out this blog post if you want a much more detailed tutorial. If you don’t mind standard BSD, get the GhostBSD distro instead which comes with a ready-made desktop xcfe or mate and many functional presets.
\n
A small summary of what we got on the upside:
\nSome downsides:
\n\nLess driver and direct app support than Linux
\n\nInstaller and desktop have some traps and quirks and require work
\n\n\n\n\nKeybase significantly simplifies the whole keypair/PGP thing and makes what is usually a confusing, difficult experience actually rather pleasant. At its heart is an open-source command line utility that does all of the heavy cryptographic lifting. But it’s also hooked up to the network of all other Keybase users, so you don’t have to work very hard to maintain big keychains. Pretty cool!
\n
\nSo, this evening, I tried to get it to all work on NetBSD.
\nThe Keybase client code base is, in my opinion, not very well architected… there exist many different Keybase clients (command line apps, desktop apps, mobile apps) and for some reason the code for all of them are seemingly in this single repository, without even using Git submodules. Not sure what that’s about.
\nAnyway, “go build”-ing the command line program (it’s written in Go) failed immediately because there’s some platform-specific code that just does not seem to recognize that NetBSD exists (but they do for FreeBSD and OpenBSD). Looks like the Keybase developers maintain a Golang wrapper around struct proc, which of course is different from OS to OS. So I literally just copypasted the OpenBSD wrapper, renamed it to “NetBSD”, and the build basically succeeded from there! This is of course super janky and untrustworthy, but it seems to Mostly Just Work…
\nI forked the GitHub repo, you can see the diff on top of keybase 2.7.3 here: bccaaf3096a
\nEventually I ended up with a ~/go/bin/keybase which launches just fine. Meaning, I can main() okay. But the moment you try to do anything interesting, it looks super scary:
charlotte@sakuracity:~/go/bin ./keybase login\n▶ WARNING Running in devel mode\n▶ INFO Forking background server with pid=12932\n▶ ERROR unexpected error in Login: API network error: doRetry failed,\nattempts: 1, timeout 5s, last err: Get\nhttp://localhost:3000/_/api/1.0/merkle/path.json?last=3784314&load_deleted=1&load_reset_chain=1&poll=10&sig_hints_low=3&uid=38ae1dfa49cd6831ea2fdade5c5d0519:\ndial tcp [::1]:3000: connect: connection refused\n
\n\n\n\n\nThere’s a few things about this error message that stuck out to me:
\n
\n\n\nUnfortunately, this nonfunctional “background server” sticks around even when a command as simple as ‘login’ command just failed:
\n
charlotte@sakuracity:~/go/bin ps 12932\n PID TTY STAT TIME COMMAND\n 12932 ? Ssl 0:00.21 ./keybase --debug --log-file\n /home/charlotte/.cache/keybase.devel/keybase.service.log service --chdir\n /home/charlotte/.config/keybase.devel --auto-forked \n
\n\n\n\n\nI’m not exactly sure what the intended purpose of the “background server” even is, but fortunately we can kill it and even tell the keybase command to not even spawn one:
\n
charlotte@sakuracity:~/go/bin ./keybase help advanced | grep -- --standalone\n --standalone Use the client without any daemon support.\n
\n\n\n\n\nAnd then we can fix wanting to connect to localhost by specifying an expected Keybase API server – how about the one hosted at https://keybase.io?
\n
charlotte@sakuracity:~/go/bin ./keybase help advanced | grep -- --server\n --server, -s Specify server API.\n
\n\n\n\n\nBasically, what I’m trying to say is that if you specify both of these options, the keybase command does what I expect on NetBSD:
\n
charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io login\n▶ WARNING Running in devel mode\nPlease enter the Keybase passphrase for dressupgeekout (6+ characters): \n\ncharlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io id dressupgeekout\n▶ WARNING Running in devel mode\n▶ INFO Identifying dressupgeekout\n✔ public key fingerprint: 7873 DA50 A786 9A3F 1662 3A17 20BD 8739 E82C 7F2F\n✔ "dressupgeekout" on github:\nhttps://gist.github.com/0471c7918d254425835bf5e1b4bcda00 [cached 2018-10-11\n20:55:21 PDT]\n✔ "dressupgeekout" on reddit:\nhttps://www.reddit.com/r/KeybaseProofs/comments/9ng5qm/my_keybase_proof_redditdressupgeekout/\n[cached 2018-10-11 20:55:21 PDT]\n
\n\n###Initial implementation of draft-ietf-6man-ipv6only-flag
\n\nThis change defines the RA "6" (IPv6-Only) flag which routers\nmay advertise, kernel logic to check if all routers on a link\nhave the flag set and accordingly update a per-interface flag.\n\nIf all routers agree that it is an IPv6-only link, ether_output_frame(),\nbased on the interface flag, will filter out all ETHERTYPE_IP/ARP\nframes, drop them, and return EAFNOSUPPORT to upper layers.\n\nThe change also updates ndp to show the "6" flag, ifconfig to\ndisplay the IPV6_ONLY nd6 flag if set, and rtadvd to allow\nannouncing the flag.\n\nFurther changes to tcpdump (contrib code) are availble and will\nbe upstreamed.\n\nTested the code (slightly earlier version) with 2 FreeBSD\nIPv6 routers, a FreeBSD laptop on ethernet as well as wifi,\nand with Win10 and OSX clients (which did not fall over with\nthe "6" flag set but not understood).\n\nWe may also want to (a) implement and RX filter, and (b) over\ntime enahnce user space to, say, stop dhclient from running\nwhen the interface flag is set. Also we might want to start\nIPv6 before IPv4 in the future.\n\nAll the code is hidden under the EXPERIMENTAL option and not\ncompiled by default as the draft is a work-in-progress and\nwe cannot rely on the fact that IANA will assign the bits\nas requested by the draft and hence they may change.\n\nDear 6man, you have running code.\n\nDiscussed with: Bob Hinden, Brian E Carpenter\n
\n\n##Beastie Bits
\n\n##Feedback/Questions
\n\nFreeBSD Foundation September Update, tiny C lib for programming Unix daemons, EuroBSDcon trip reports, GhostBSD tested on real hardware, and a BSD auth module for duress.
\n\n##Headlines
\n###FreeBSD Foundation Update, September 2018
\n\n\nDear FreeBSD Community Member, It is hard to believe that September is over. The Foundation team had a busy month promoting FreeBSD all over the globe, bug fixing in preparation for 12.0, and setting plans in motion to kick off our 4th quarter fundraising and advocacy efforts. Take a minute to see what we’ve been up to and please consider making a donation to help us continue our efforts supporting FreeBSD!
\n
\n\n\nIn preparation for the release of FreeBSD 12.0, I have been working on investigating and fixing a backlog of kernel bug reports. Of course, this kind of work is never finished, and we will continue to make progress after the release. In the past couple of months I have fixed a combination of long-standing issues and recent regressions. Of note are a pair of UNIX domain socket bugs which had been affecting various applications for years. In particular, Chromium tabs would frequently hang unless a workaround was manually applied to the system, and the bug had started affecting recent versions of Firefox as well. Fixing these issues gave me an opportunity to revisit and extend our regression testing for UNIX sockets, which, in turn, resulted in some related bugs being identified and fixed.
\n
\nOf late I have also been investigating reports of issues with ZFS, particularly, those reported on FreeBSD 11.2. A number of regressions, including a kernel memory leak and issues with ARC reclamation, have already been fixed for 12.0; investigation of other reports is ongoing. Those who closely follow FreeBSD-CURRENT know that some exciting work to improve memory usage on NUMA systems is now enabled by default. As is usually the case when new code is deployed in a diverse array of systems and workloads, a number of problems since have been identified. We are working on resolving them as soon as possible to ensure the quality of the release.
\nI’m passionate about maintaining FreeBSD’s stability and dependability as it continues to expand and grow new features, and I’m grateful to the FreeBSD Foundation for sponsoring this work. We depend on users to report problems to the mailing lists and via the bug tracker, so please try running the 12.0 candidate builds and help us make 12.0 a great release.
\n\n\nIt’s officially Fall here at Foundation headquarters and we’re heading full-steam into our final fundraising campaign of the year. We couldn’t even have begun to reach our funding goal of $1.25 million dollars without the support from the companies who have partnered with us this year. Thank you to Verisign for becoming a Silver Partner. They now join a growing list of companies like Xiplink, NetApp, Microsoft, Tarsnap, VMware, and NeoSmart Technologies that are stepping up and showing their commitment to FreeBSD!
\n
\nFunding from commercial users like these and individual users like yourself, help us continue our efforts of supporting critical areas of FreeBSD such as:
\n\n\nWe can continue the above work, if we meet our goal this year!
\n
\nIf your company uses FreeBSD, please consider joining our growing list of 2018 partners. If you haven’t made your donation yet, please consider donating today. We are indebted to the individual donors, and companies listed above who have already shown their commitment to open source.
\nThank you for supporting FreeBSD and the Foundation!
\n\n\nThe FreeBSD Release Engineering team continued working on the upcoming 12.0 RELEASE. At present, the 12.0 schedule had been adjusted by one week to allow for necessary works-in-progress to be completed.
\n
\nOf note, one of the works-in-progress includes updating OpenSSL from 1.0.2 to 1.1.1, in order to avoid breaking the application binary interface (ABI) on an established stable branch.
\nDue to the level of non-trivial intrusiveness that had already been discovered and addressed in a project branch of the repository, it is possible (but not yet definite) that the schedule will need to be adjusted by another week to allow more time for larger and related updates for this particular update.
\nShould the 12.0-RELEASE schedule need to be adjusted at any time during the release cycle, the schedule on the FreeBSD project website will be updated accordingly. The current schedule is available at:
\nhttps://www.freebsd.org/releases/12.0R/schedule.html
\n\n\nI’d like to start by thanking the FreeBSD Foundation for sponsoring my trip to BSDCam(bridge) 2018. I wouldn’t have managed to attend otherwise. I’ve used FreeBSD in both personal and professional deployments since the year 2000, and over the last few years I have become more involved with development and documentation.
\n
\nI arrived in Gatwick, London at midnight. On Monday, August 13, I took the train to Cambridge, and decided to do some touristy activities as I walked from the train station to Churchill College. I ran into Allan outside the hotel right before the sky decided it was time for a heavy rainfall. Monday was mostly spent settling in, recouping after travel, and hanging out with Allan, Brad, Will and Andy later in the afternoon/evening. Read more…
\n\n\nThe FreeBSD Foundation has sponsored the development of the Project’s continuous integration system, available at https://ci.FreeBSD.org, since June. Over the summer, we improved both the software and hardware infrastructure, and also added some new jobs for extending test coverage of the -CURRENT and -STABLE branches. Following are some highlights.
\n
\n\n\nThe Foundation purchased 4 new build machines for scaling up the computation power for the various test jobs. These newer, faster machines substantially speed up the time it takes to test amd64 builds, so that failing changes can be identified more quickly. Also, in August, we received a donation of 2 PINE A64-LTS boards from PINE64.org, which will be put in the hardware test lab as one part of the continuous tests.
\n
\n\n\nWe used hardware from a previous generation CI system to build a staging environment for the CI infrastructure, which is available at
\n
\nhttps://ci-dev.freebsd.org. It executes the configurations and scripts from the “staging” branch of the FreeBSD-CI repository, and the development feature branches. We also use it to experiment with the new version of the jenkins server and plugins. Having a staging environment avoids affecting the production CI environment, reducing downtime.
\n\n\nIn July, we turned on failure notification for all the kernel and world build jobs. Committers will receive email containing the build information and failure log to inform them of possible problems with their modification on certain architectures. For amd64 of the -CURRENT branch, we also enabled the notification on failing regression test cases. Currently mail is sent only to the individual committers, but with help from postmaster team, we have created a dev-ci mailing list and will soon be also sending notifications there.
\n
\n\n\nIn August, we updated the embedded script of the virtual machine image. Originally it only executed pre-defined tests, but now this behavior can be modified by the data on the attached disk. This mechanism is used for adding new ZFS tests jobs. We are also working on analyzing and fixing the failing and skipped test cases.
\n
\n\n\nIn August and September, we had two developer summits, one in Cambridge, UK and one in Bucharest, Romania. In these meetings, we discussed running special tests, such as ztest, which need a longer run time. We also planned the network testing for TCP/IP stack
\n
###Daemonize - a Tiny C Library for Programming the UNIX Daemons
\n\n\n\n\nWhatever they say, writing System-V style UNIX daemons is hard. One has to follow many rules to make a daemon process behave correctly on diverse UNIX flavours. Moreover, debugging such a code might be somewhat tricky. On the other hand, the process of daemon initialisation is rigid and well defined so the corresponding code has to be written and debugged once and later can be reused countless number of times.
\n
\nDevelopers of BSD UNIX were very aware of this, as there a C library function daemon() was available starting from version 4.4. The function, although non-standard, is present on many UNIXes. Unfortunately, it does not follow all the required steps to reliably run a process in the background on systems which follow System-V semantics (e.g. Linux). The details are available at the corresponding Linux man page. The main problem here, as I understand it, is that daemon() does not use the double-forking technique to avoid the situation when zombie processes appear.
\nWhenever I encounter a problem like this one, I know it is time to write a tiny C library which solves it. This is exactly how ‘daemonize’ was born (GitHub mirror). The library consists of only two files which are meant to be integrated into the source tree of your project. Recently I have updated the library and realised that it would be good to describe how to use it on this site.
\nIf for some reason you want to make a Windows service, I have a battle tested template code for you as well.
\n\n\nTo make discussion clear we shall quote the steps which have to be performed during a daemon initialisation (according to daemon(7) manual page on Linux). I do it to demonstrate that this task is more tricky than one might expect.
\n
So, here we go:
\nClose all open file descriptors except standard input, output, and error (i.e. the first three file descriptors 0, 1, 2). This ensures that no accidentally passed file descriptor stays around in the daemon process. On Linux, this is best implemented by iterating through /proc/self/fd, with a fallback of iterating from file descriptor 3 to the value returned by getrlimit() for RLIMIT_NOFILE.
\nReset all signal handlers to their default. This is best done by iterating through the available signals up to the limit of _NSIG and resetting them to SIG_DFL.
\nReset the signal mask using sigprocmask().
\nSanitize the environment block, removing or resetting environment variables that might negatively impact daemon runtime.
\nCall fork(), to create a background process.
\nIn the child, call setsid() to detach from any terminal and create an independent session.
\nIn the child, call fork() again, to ensure that the daemon can never re-acquire a terminal again.
\nCall exit() in the first child, so that only the second child (the actual daemon process) stays around. This ensures that the daemon process is re-parented to init/PID 1, as all daemons should be.
\nIn the daemon process, connect /dev/null to standard input, output, and error.
\nIn the daemon process, reset the umask to 0, so that the file modes passed to open(), mkdir() and suchlike directly control the access mode of the created files and directories.
\nIn the daemon process, change the current directory to the root directory (/), in order to avoid that the daemon involuntarily blocks mount points from being unmounted.
\nIn the daemon process, write the daemon PID (as returned by getpid()) to a PID file, for example /run/foobar.pid (for a hypothetical daemon “foobar”) to ensure that the daemon cannot be started more than once. This must be implemented in race-free fashion so that the PID file is only updated when it is verified at the same time that the PID previously stored in the PID file no longer exists or belongs to a foreign process.
\nIn the daemon process, drop privileges, if possible and applicable.
\nFrom the daemon process, notify the original process started that initialization is complete. This can be implemented via an unnamed pipe or similar communication channel that is created before the first fork() and hence available in both the original and the daemon process.
\nCall exit() in the original process. The process that invoked the daemon must be able to rely on that this exit() happens after initialization is complete and all external communication channels are established and accessible.
\n\n\n\nThe discussed library does most of the above-mentioned initialisation steps as it becomes immediately evident that implementation details for some of them heavily dependent on the internal logic of an application itself, so it is not possible to implement them in a universal library. I believe it is not a flaw, though, as the missed parts are safe to implement in an application code.
\n
\n\n\nThe generic programming interface was loosely modelled after above-mentioned BSD’s daemon() function. The library provides two user available functions (one is, in fact, implemented on top of the other) as well as a set of flags to control a daemon creation behaviour.
\n
\n\n\nThe objective of the library is to hide all the trickery of programming a daemon so you could concentrate on the more creative parts of your application. I hope it does this well.
\n
\nIf you are not only interested in writing a daemon, but also want to make yourself familiar with the techniques which are used to accomplish that, the source code is available. Moreover, I would advise anyone, who starts developing for a UNIX environment to do that, as it shows many intricacies of programming for these platforms.
##News Roundup
\n###EuroBSDCon 2018 travel report and obligatory pics
\n\n\nThis was my first big BSD conference. We also planned - planned might be a big word - thought about doing a devsummit on Friday. Since the people who were in charge of that had a change of plans, I was sure it’d go horribly wrong.
\n
\nThe day before the devsummit and still in the wrong country, I mentioned the hours and venue on the wiki, and booked a reservation for a restaurant.
\nIt turns out that everything was totally fine, and since the devsummit was at the conference venue (that was having tutorials that day), they even had signs pointing at the room we were given. Thanks EuroBSDCon conference organizers!
\nAt the devsummit, we spent some time hacking. A few people came with “travel laptops” without access to anything, like Riastradh, so I gave him access to my own laptop. This didn’t hold very long and I kinda forgot about it, but for a few moments he had access to a NetBSD source tree and an 8 thread, 16GB RAM machine with which to build things.
\nWe had a short introduction and I suggested we take some pictures, so here’s the ones we got. A few people were concerned about privacy, so they’re not pictured. We had small team to hold the camera :-)
\nAt the actual conference days, I stayed at the speaker hotel with the other speakers. I’ve attempted to make conversation with some visibly FreeBSD/OpenBSD people, but didn’t have plans to talk about anything, so there was a lot of just following people silently.
\nPerhaps for the next conference I’ll prepare a list of questions to random BSD people and then very obviously grab a piece of paper and ask, “what was…”, read a bit from it, and say, “your latest kernel panic?”, I’m sure it’ll be a great conversation starter.
\nAt the conference itself, was pretty cool to have folks like Kirk McKusick give first person accounts of some past events (Kirk gave a talk about governance at FreeBSD), or the second keynote by Ron Broersma.
\nMy own talk was hastily prepared, it was difficult to bring the topic together into a coherent talk. Nevertheless, I managed to talk about stuff for a while 40 minutes, though usually I skip over so many details that I have trouble putting together a sufficiently long talk.
\nI mentioned some of my coolest bugs to solve (I should probably make a separate article about some!). A few people asked for the slides after the talk, so I guess it wasn’t totally incoherent.
\nIt was really fun to meet some of my favourite NetBSD people. I got to show off my now fairly well working laptop (it took a lot of work by all of us!).
\nAfter the conference I came back with a conference cold, and it took a few days to recover from it. Hopefully I didn’t infect too many people on the way back.
###GhostBSD tested on real hardware T410 – better than TrueOS?
\n\n\n\n\nYou might have heard about FreeBSD which is ultimately derived from UNIX back in the days. It is not Linux even though it is similar in many ways because Linux was designed to follow UNIX principles. Seeing is believing, so check out the video of the install and some apps as well!
\n
\n\n\nNowadays if you want some of that BSD on your personal desktop how to go about? Well there is a full package or distro called GhostBSD which is based on FreeBSD current with a Mate or XFCE desktop preconfigured. I did try another package called TrueOS before and you can check out my blog post as well.
\n
\n\n\nLet’s give it a try on my Lenovo ThinkPad T410. You can download the latest version from ghostbsd.org. Creating a bootable USB drive was surprisingly difficult as rufus did not work and created a corrupted drive. You have to follow this procedure under Windows: download the 2.5GB .iso file and rename the extension to .img. Download Win32 Disk imager and burn the img file to an USB drive and boot from it. You will be able to start a live session and use the onboard setup to install GhostBSD unto a disk.
\n
\n\n\nI did encounter some bugs or quirks along the way. The installer failed the first time for some unknown reason but worked on the second attempt. The first boot stopped upon initialization of the USB3 ports (the T410 does not have USB3) but I could use some ‘exit’ command line magic to continue. The second boot worked fine. Audio was only available through headphones, not speakers but that could partially be fixed using the command line again. Lot’s of installed apps did not show up in the start menu and on goes the quirks list.
\n
\n\n\nOverall it is still better than TrueOS for me because drivers did work very well and I could address most of the existing bugs.
\n
On the upside:
\nFree and open source FreeBSD package ready to go
\nMate or XFCE desktop (Mate is the only option for daily builds)
\nDrivers work fine including LAN, WiFi, video 2D & 3D, audio, etc
\nUFS or ZFS advanced file systems available
\nSome downsides:
\nLess driver and direct app support than Linux
\nInstaller and desktop have some quirks and bugs
\nApp-store is cumbersome, inferior to TrueOS
\n##Beastie Bits
\n\n##Feedback/Questions
\n\n6 metrics for zpool performance, 2FA with ssh on OpenBSD, ZFS maintaining file type information in dirs, everything old is new again, netcat demystified, and more.
\n\n##Headlines
\n###Six Metrics for Measuring ZFS Pool Performance Part 1
\n\n\nThe layout of a ZFS storage pool has a significant impact on system performance under various workloads. Given the importance of picking the right configuration for your workload and the fact that making changes to an in-use ZFS pool is far from trivial, it is important for an administrator to understand the mechanics of pool performance when designing a storage system.
\n
\n\n\nNote that when we calculate data rates and IOPS values for the example system, they are only approximations. Many other factors can impact pool access speeds for better (compression, caching) or worse (poor CPU performance, not enough memory).
\n
\nThere is no single configuration that maximizes all six metrics. Like so many things in life, our objective is to find an appropriate balance of the metrics to match a target workload. For example, a cold-storage backup system will likely want a pool configuration that emphasizes usable storage space and fault tolerance over the other data-rate focused metrics.
\nLet’s start with a quick review of ZFS storage pools before diving into specific configuration options. ZFS storage pools are comprised of one or more virtual devices, or vdevs. Each vdev is comprised of one or more storage providers, typically physical hard disks. All disk-level redundancy is configured at the vdev level. That is, the RAID layout is set on each vdev as opposed to on the storage pool. Data written to the storage pool is then striped across all the vdevs. Because pool data is striped across the vdevs, the loss of any one vdev means total pool failure. This is perhaps the single most important fact to keep in mind when designing a ZFS storage system. We will circle back to this point in the next post, but keep it in mind as we go through the vdev configuration options.
\nBecause storage pools are made up of one or more vdevs with the pool data striped over the top, we’ll take a look at pool configuration in terms of various vdev configurations. There are three basic vdev configurations: striping, mirroring, and RAIDZ (which itself has three different varieties). The first section will cover striped and mirrored vdevs in this post; the second post will cover RAIDZ and some example scenarios.
\nA striped vdev is the simplest configuration. Each vdev consists of a single disk with no redundancy. When several of these single-disk, striped vdevs are combined into a single storage pool, the total usable storage space would be the sum of all the drives. When you write data to a pool made of striped vdevs, the data is broken into small chunks called “blocks” and distributed across all the disks in the pool. The blocks are written in “round-robin” sequence, meaning after all the disks receive one row of blocks, called a stripe, it loops back around and writes another stripe under the first. A striped pool has excellent performance and storage space efficiency, but absolutely zero fault tolerance. If even a single drive in the pool fails, the entire pool will fail and all data stored on that pool will be lost.
\nThe excellent performance of a striped pool comes from the fact that all of the disks can work independently for all read and write operations. If you have a bunch of small read or write operations (IOPS), each disk can work independently to fetch the next block. For streaming reads and writes, each disk can fetch the next block in line synchronized with its neighbors. For example, if a given disk is fetching block n, its neighbor to the left can be fetching block n-1, and its neighbor to the right can be fetching block n+1. Therefore, the speed of all read and write operations as well as the quantity of read and write operations (IOPS) on a striped pool will scale with the number of vdevs. Note here that I said the speeds and IOPS scale with the number of vdevs rather than the number of drives; there’s a reason for this and we’ll cover it in the next post when we discuss RAID-Z.
\nHere’s a summary of the total pool performance (where N is the number of disks in the pool):
\n\n\nLet’s apply this to our example system, configured with a 12-wide striped pool:
\n
\n\n\nThe blocks are simply striped across the 12 disks in the pool. The LBA column on the left stands for “Logical Block Address”. If we treat each disk as a column in an array, each LBA would be a row. It’s also easy to see that if any single disk fails, we would be missing a color in the rainbow and our data would be incomplete. While this configuration has fantastic read and write speeds and can handle a ton of IOPS, the data stored on the pool is very vulnerable. This configuration is not recommended unless you’re comfortable losing all of your pool’s data whenever any single drive fails.
\n
\nA mirrored vdev consists of two or more disks. A mirrored vdev stores an exact copy of all the data written to it on each one of its drives. Traditional RAID-1 mirrors usually only support two drive mirrors, but ZFS allows for more drives per mirror to increase redundancy and fault tolerance. All disks in a mirrored vdev have to fail for the vdev, and thus the whole pool, to fail. Total storage space will be equal to the size of a single drive in the vdev. If you’re using mismatched drive sizes in your mirrors, the total size will be that of the smallest drive in the mirror.
\nStreaming read speeds and read IOPS on a mirrored vdev will be faster than write speeds and IOPS. When reading from a mirrored vdev, the drives can “divide and conquer” the operations, similar to what we saw above in the striped pool. This is because each drive in the mirror has an identical copy of the data. For write operations, all of the drives need to write a copy of the data, so the mirrored vdev will be limited to the streaming write speed and IOPS of a single disk.
\n\n\nHere’s a summary:
\n
N-way mirror:
\nRead IOPS: N * Read IOPS of a single drive
\nWrite IOPS: Write IOPS of a single drive
\nStreaming read speed: N * Streaming read speed of a single drive
\nStreaming write speed: Streaming write speed of a single drive
\nStorage space efficiency: 50% for 2-way, 33% for 3-way, 25% for 4-way, etc. [(N-1)/N]
\nFault tolerance: 1 disk per vdev for 2-way, 2 for 3-way, 3 for 4-way, etc. [N-1]
\nFor our first example configuration, let’s do something ridiculous and create a 12-way mirror. ZFS supports this kind of thing, but your management probably will not.
\n1x 12-way mirror:
\nRead IOPS: 3000
\nWrite IOPS: 250
\nStreaming read speed: 1200 MB/s
\nStreaming write speed: 100 MB/s
\nStorage space efficiency: 8.3% (6 TB)
\nFault tolerance: 11
\n\n\n\nAs we can clearly see from the diagram, every single disk in the vdev gets a full copy of our rainbow data. The chainlink icons between the disk labels in the column headers indicate the disks are part of a single vdev. We can lose up to 11 disks in this vdev and still have a complete rainbow. Of course, the data takes up far too much room on the pool, occupying a full 12 LBAs in the data array.
\n
\n\n\nObviously, this is far from the best use of 12 drives. Let’s do something a little more practical and configure the pool with the ZFS equivalent of RAID-10. We’ll configure six 2-way mirror vdevs. ZFS will stripe the data across all 6 of the vdevs. We can use the work we did in the striped vdev section to determine how the pool as a whole will behave. Let’s first calculate the performance per vdev, then we can work on the full pool:
\n
1x 2-way mirror:
\nRead IOPS: 500
\nWrite IOPS: 250
\nStreaming read speed: 200 MB/s
\nStreaming write speed: 100 MB/s
\nStorage space efficiency: 50% (6 TB)
\nFault tolerance: 1
\nNow we can pretend we have 6 drives with the performance statistics listed above and run them through our striped vdev performance calculator to get the total pool’s performance:
\n6x 2-way mirror:
\nRead IOPS: 3000
\nWrite IOPS: 1500
\nStreaming read speed: 3000 MB/s
\nStreaming write speed: 1500 MB/s
\nStorage space efficiency: 50% (36 TB)
\nFault tolerance: 1 per vdev, 6 total
\nAgain, we will examine the configuration from a visual perspective:
\n\n\n\nEach vdev gets a block of data and ZFS writes that data to all of (or in this case, both of) the disks in the mirror. As long as we have at least one functional disk in each vdev, we can retrieve our rainbow. As before, the chain link icons denote the disks are part of a single vdev. This configuration emphasizes performance over raw capacity but doesn’t totally disregard fault tolerance as our striped pool did. It’s a very popular configuration for systems that need a lot of fast I/O. Let’s look at one more example configuration using four 3-way mirrors. We’ll skip the individual vdev performance calculation and go straight to the full pool:
\n
\n\n\nWhile we have sacrificed some write performance and capacity, the pool is now extremely fault tolerant. This configuration is probably not practical for most applications and it would make more sense to use lower fault tolerance and set up an offsite backup system.
\n
\nStriped and mirrored vdevs are fantastic for access speed performance, but they either leave you with no redundancy whatsoever or impose at least a 50% penalty on the total usable space of your pool. In the next post, we will cover RAIDZ, which lets you keep data redundancy without sacrificing as much storage space efficiency. We’ll also look at some example workload scenarios and decide which layout would be the best fit for each.
\n\n\nFive years ago I wrote about using a yubikey on OpenBSD. The only problem with doing this is that there’s no validation server available on OpenBSD, so you need to use a different OTP slot for each machine. (You don’t want to risk a replay attack if someone succeeds in capturing an OTP on one machine, right?) Yubikey has two OTP slots per device, so you would need a yubikey for every two machines with which you’d like to use it. You could use a bastion—and use only one yubikey—but I don’t like the SPOF aspect of a bastion. YMMV.
\n
\nAfter I played with TOTP, I wanted to use them as a 2FA for ssh. At the time of writing, we can’t do that using only the tools in base. This article focuses on OpenBSD; if you use another operating system, here are two handy links.
\n\n\nThe first thing we need to do is to install the software which will be used to verify the OTPs we submit.
\n
# pkg_add login_oath
\n\n\nWe need to create a secret - aka, the seed - that will be used to calculate the Time-based One-Time Passwords. We should make sure no one can read or change it.
\n
$ openssl rand -hex 20 > ~/.totp-key
\n$ chmod 400 ~/.totp-key
\n\n\nNow we have a hexadecimal key, but apps usually want a base32 secret. I initially wrote a small script to do the conversion.
\n
\nWhile writing this article, I took the opportunity to improve it. When I initially wrote this utility for my use, python-qrcode hadn’t yet been imported to the OpenBSD ports/packages system. It’s easy to install now, so let’s use it.
\nHere’s the improved version. It will ask for the hex key and output the secret as a base32-encoded string, both with and without spacing so you can copy-paste it into your password manager or easily retype it. It will then ask for the information needed to generate a QR code. Adding our new OTP secret to any mobile app using the QR code will be super easy!
\n\n\nWe can now move to the configuration of the system to put our new TOTP to use. As you might guess, it’s going to be quite close to what we did with the yubikey.
\n
\nWe need to tweak login.conf. Be careful and keep a root shell open at all times. The few times I broke my OpenBSD were because I messed with login.conf without showing enough care.
\n\n\nAgain, keeping a root shell around decreases the risk of losing access to the system and being locked outside.
\n
\nA good standard is to use PasswordAuthentication no and to use public key only. Except… have a guess what the P stands for in TOTP. Yes, congrats, you guessed it!
\nWe need to switch to PasswordAuthentication yes. However, if we made this change alone, sshd would then accept a public key OR a password (which are TOTP because of our login.conf). 2FA uses both at the same time.
\nTo inform sshd we intend to use both, we need to set AuthenticationMethods publickey,password. This way, the user trying to login will first need to perform the traditional publickey authentication. Once that’s done, ssh will prompt for a password and the user will need to submit a valid TOTP for the system.
\nWe could do this the other way around, but I think bots could try passwords, wasting resources. Evaluated in this order, failing to provide a public key leads to sshd immediately declining your attempt.
\n\n\nMy phone has a long enough password that most of the time, I fail to type it correctly on the first try. Of course, if I had to unlock my phone, launch my TOTP app and use my keyboard to enter what I see on my phone’s screen, I would quickly disable 2FA.
\n
\nTo find a balance, I have whitelisted certain IP addresses and users. If I connect from a particular IP address or as a specific user, I don’t want to go through 2FA. For some users, I might not even enable 2FA.
\nTo sum up, we covered how to create a seed, how to perform a hexadecimal to base32 conversion and how to create a QR code for mobile applications. We configured the login system with login.conf so that ssh authentication uses the TOTP login system, and we told sshd to ask for both the public key and the Time-based One-Time Password. Now you should be all set to use two-factor ssh authentication on OpenBSD!
##News Roundup
\n###How ZFS maintains file type information in directories
\n\n\nAs an aside in yesterday’s history of file type information being available in Unix directories, I mentioned that it was possible for a filesystem to support this even though its Unix didn’t. By supporting it, I mean that the filesystem maintains this information in its on disk format for directories, even though the rest of the kernel will never ask for it. This is what ZFS does.
\n
\nThe easiest way to see that ZFS does this is to use zdb to dump a directory. I’m going to do this on an OmniOS machine, to make it more convincing, and it turns out that this has some interesting results. Since this is OmniOS, we don’t have the convenience of just naming a directory in zdb, so let’s find the root directory of a filesystem, starting from dnode 1 (as seen before).
# zdb -dddd fs3-corestaff-01/h/281 1
\nDataset [....]
\n[...]
\nmicrozap: 512 bytes, 4 entries
\n[...]
\nROOT = 3
\n
\n# zdb -dddd fs3-corestaff-01/h/281 3
\nObject lvl iblk dblk dsize lsize %full type
\n3 1 16K 1K 8K 1K 100.00 ZFS directory
\n[...]
\nmicrozap: 1024 bytes, 8 entries
\n
\nRESTORED = 4396504 (type: Directory)
\nckstst = 12017 (type: not specified)
\nckstst3 = 25069 (type: Directory)
\n.demo-file = 5832188 (type: Regular File)
\n.peergroup = 12590 (type: not specified)
\ncks = 5 (type: not specified)
\ncksimap1 = 5247832 (type: Directory)
\n.diskuse = 12016 (type: not specified)
\nckstst2 = 12535 (type: not specified)
\n\n\nThis is actually an old filesystem (it dates from Solaris 10 and has been transferred around with ‘zfs send | zfs recv’ since then), but various home directories for real and test users have been created in it over time (you can probably guess which one is the oldest one). Sufficiently old directories and files have no file type information, but more recent ones have this information, including .demo-file, which I made just now so this would have an entry that was a regular file with type information.
\n
\nOnce I dug into it, this turned out to be a change introduced (or activated) in ZFS filesystem version 2, which is described in ‘zfs upgrade -v’ as ‘enhanced directory entries’. As an actual change in (Open)Solaris, it dates from mid 2007, although I’m not sure what Solaris release it made it into. The upshot is that if you made your ZFS filesystem any time in the last decade, you’ll have this file type information in your directories.
\nHow ZFS stores this file type information is interesting and clever, especially when it comes to backwards compatibility. I’ll start by quoting the comment from zfs_znode.h:
/*
\n* The directory entry has the type (currently unused on
\n* Solaris) in the top 4 bits, and the object number in
\n* the low 48 bits. The "middle" 12 bits are unused.
\n*/
\n\n\nIn yesterday’s entry I said that Unix directory entries need to store at least the filename and the inode number of the file. What ZFS is doing here is reusing the 64 bit field used for the ‘inode’ (the ZFS dnode number) to also store the file type, because it knows that object numbers have only a limited range. This also makes old directory entries compatible, by making type 0 (all 4 bits 0) mean ‘not specified’. Since old directory entries only stored the object number and the object number is 48 bits or less, the higher bits are guaranteed to be all zero.
\n
\nThe reason this needed a new ZFS filesystem version is now clear. If you tried to read directory entries with file type information on a version of ZFS that didn’t know about them, the old version would likely see crazy (and non-existent) object numbers and nothing would work. In order to even read a ‘file type in directory entries’ filesystem, you need to know to only look at the low 48 bits of the object number field in directory entries.
###Everything old is new again
\n\n\n\n\nJust because KDE4-era software has been deprecated by the KDE-FreeBSD team in the official ports-repository, doesn’t mean we don’t care for it while we still need to. KDE4 was released on January 11th, 2008 — I still have the T-shirt — which was a very different C++ world than what we now live in. Much of the code pre-dates the availability of C11 — certainly the availability of compilers with C11 support. The language has changed a great deal in those ten years since the original release.
\n
\nThe platforms we run KDE code on have, too — FreeBSD 12 is a long way from the FreeBSD 6 or 7 that were current at release (although at the time, I was more into OpenSolaris). In particular, since then the FreeBSD world has switched over to Clang, and FreeBSD current is experimenting with Clang 7. So we’re seeing KDE4-era code being built, and running, on FreeBSD 12 with Clang 7. That’s a platform with a very different idea of what constitutes correct code, than what the code was originally written for. (Not quite as big a difference as Helio’s KDE1 efforts, though)
\nSo, while we’re counting down to removing KDE4 from the FreeBSD ports tree, we’re also going through and fixing it to work with Clang 7, which defaults to a newer C++ standard and which is quite picky about some things. Some time in the distant past, when pointers were integers and NULL was zero, there was some confusion about booleans. So there’s lots of code that does list.contains(element) > 0 … this must have been a trick before booleans were a supported type in all our compilers. In any case it breaks with Clang 7, since contains() returns a QBool which converts to a nullptr (when false) which isn’t comparable to the integer 0. Suffice to say I’ve spent more time reading KDE4-era code this month, than in the past two years.
\nHowever, work is proceeding apace, so if you really really want to, you can still get your old-school kicks on a new platform. Because we care about packaging things right, even when we want to get rid of it.
\n\n\nOwing to its versatile functionalities, netcat earns the reputation as “TCP/IP Swiss army knife”. For example, you can create a simple chat app using netcat:
\n
# nc -l 3003
\n\n\nThis means a netcat process will listen on 3003 port in this machine (the IP address of current machine is 192.168.35.176).
\n
# nc 192.168.35.176 3003
\nhello
\n\n\nThen in the first machine’s terminal, you will see the “hello” text:
\n
# nc -l 3003
\nhello
\n\n\nA primitive chatroom is built successfully. Very cool! Isn’t it? I think many people can’t wait to explore more features of netcatnow. If you are among them, congratulations! This tutorial may be the correct place for you.
\n
\nIn the following parts, I will delve into OpenBSD’s netcatcode to give a detailed anatomy of it. The reason of picking OpenBSD’s netcat rather than others’ is because its code repository is small (~2000 lines of code) and neat. Furthermore, I also hope this little book can assist you learn more socket programming knowledge not just grasping usage of netcat.
\nWe’re all set. Let’s go!
##Beastie Bits
\n\n##Feedback/Questions
\n\nWe have a long interview with fiction and non-fiction author Michael W. Lucas for you this week as well as questions from the audience.
\n\n##Headlines
\n##Interview - Michael W. Lucas - mwlucas@michaelwlucas.com / @mwlauthor
Auction at https://mwl.io
\nPatreon Link:
##Feedback/Questions
\n\nRunning OpenBSD/NetBSD on FreeBSD using grub2-bhyve, vermaden’s FreeBSD story, thoughts on OpenBSD on the desktop, history of file type info in Unix dirs, Multiboot a Pinebook KDE neon image, and more.
\n\n##Headlines
\n###OpenBSD/NetBSD on FreeBSD using grub2-bhyve
\n\n\nWhen I was writing a blog post about the process title, I needed a couple of virtual machines with OpenBSD, NetBSD, and Ubuntu. Before that day I mainly used FreeBSD and Windows with bhyve. I spent some time trying to set up an OpenBSD using bhyve and UEFI as described here. I had numerous problems trying to use it, and this was the day I discovered the grub2-bhyve tool, and I love it!
\n
\nThe grub2-bhyve allows you to load a kernel using GRUB bootloader. GRUB supports most of the operating systems with a standard configuration, so exactly the same method can be used to install NetBSD or Ubuntu. First, let’s install grub2-bhyve on our FreeBSD box:
# pkg install grub2-bhyve
\n\n\nTo run grub2-bhyve we need to provide at least the name of the VM. In bhyve, if the memsize is not specified the default VM is created with 256MB of the memory.
\n
# grub-bhyve test
\nGNU GRUB version 2.00
\nMinimal BASH-like line editing is supported. For the first word, TAB lists possible command
\ncompletions. Anywhere else TAB lists possible device or file completions.
\n
\n
\ngrub>
\n\n\nAfter running grub-bhyve command we will enter the GRUB loader. If we type the ls command, we will see all the available devices. In the case of the grub2-bhyve there is one additional device called “(host)” that is always available and allows the host filesystem to be accessed. We can list files under that device.
\n
grub> ls
\n(host)
\ngrub> ls (host)/
\nlibexec/ bin/ usr/ bhyve/ compat/ tank/ etc/ boot/ net/ entropy proc/ lib/ root/ sys/ mnt/ rescue/ tmp/ home/ sbin/ media/ jail/ COPYRIGHT var/ dev/
\ngrub>
\n\n\nTo exit console simply type ‘reboot’. I would like to install my new operating system under a ZVOL
\nztank/bhyve/post
. On another terminal, we create:
# zfs create -V 10G ztank/bhyve/post
\n\n\nIf you don’t use ZFS for some crazy reason you can also create a raw blob using the truncate(1) command.
\n
# truncate -s 10G post.img
\n\n\nI recommend installing an operating system from the disk image (installXX.fs for OpenBSD and NetBSD-X.X-amd64-install.img for NetBSD). Now we need to create a device map for a GRUB.
\n
cat > /tmp/post.map << EOF
\n(hd0) /directory/to/disk/image
\n(hd1) /dev/zvol/ztank/bhyve/post
\nEOF
\n\n\nThe mapping files describe the names for files in the GRUB. In our case under hd0 we will have an installation image and in hd1 we will have our ZVOL/blob. You can also try to use an ISO image then instead of using hd0 device name use a cd0. When we will run the grub-bhyve command we will see two additional devices.
\n
# grub-bhyve -m /tmp/post.map post
\ngrub> ls
\n(hd0) (hd0,msdos4) (hd0,msdos1) (hd0,openbsd9) (hd0,openbsd1) (hd1) (host)
\n\n\nThe hd0 (in this example OpenBSD image) contains multiple partitions. We can check what is on it.
\n
grub> ls (hd0,msdos4)/
\nboot bsd 6.4/ etc/
\n\n\nAnd this is the partition that contains a kernel. Now we can set a root device, load an OpenBSD kernel and boot:
\n
grub> set root=(hd0,msdos4)
\ngrub> kopenbsd -h com0 -r sd0a /bsd
\ngrub> boot
\n\n\nAfter that, we can run bhyve virtual machine. In my case it is:
\n
# bhyve -c 1 -w -u -H \\
\n-s 0,amd_hostbridge \\
\n-s 3,ahci-hd,/directory/to/disk/image \\
\n-s 4,ahci-hd,/dev/zvol/ztank/bhyve/post \\
\n-s 31,lpc -l com1,stdio \\
\npost
\n\n\nUnfortunately explaining the whole bhyve(8) command line is beyond this article. After installing the operating system remove hd0 from the mapping file and the image from the bhyve(8) command. If you don’t want to type all those GRUB commands, you can simply redirect them to the standard input.
\n
cat << EOF | grub-bhyve -m /tmp/post.map -M 512 post
\nset root=(hd0,4)
\nkopenbsd -h com0 -r sd0a /bsd
\nboot
\nEOF
\n\n\nMy first devices/computers/consoles (not at the same time) that I remember were Atari 2600 and Pegasus console which was hardware clone of the Nintendo NES.
\n
\nBack then I did not even knew that it was Atari 2600 as I referred to it as Video Computer System … and I did not even knew any english by then. It took me about two decades to get to know (by accident) that this Video Computer System was Atari 2600
\nThen I got AMIGA 600 computer (or should I say my parents bought it for me) which served both for playing computer games and also other activities for the first time. AMIGA is the computer that had the greatest influence on me, as it was the first time I studied the books about Amiga Workbench operating system and learned commands from Amiga Shell terminal. I loved the idea of Ram Disk icon/directory on the desktop that allowed me to transparently put any things in system memory. I still miss that concept on today’s desktop systems … and I still remember how dismal I was when I watched Amiga Deathbed Vigil movie.
\nAt the end of 1998 I got my first PC that of course came with Windows and that computer served both as gaming machine and as well as typical tool. One time I dig into the internals with Windows Registry (which left me disgusted by its concepts and implementation) and its limited command line interface provided by CMD.EXE executable. I remember that the heart of this box was not the CPU or the motherboard but the graphics accelerator – the legendary 3Dfx Voodoo card. This company (3Dfx) – their attitude and philosophy – also left solid fingerprint on my way. Like AMIGA did.
\nAfter ‘migration’ from AMIGA to PC it never again ‘felt right’. The games were cool but the Windows system was horrible. Time has passed and different Windows versions and hardware modifications took place. Windows XP felt really heavy at that time, not to mention Windows 2000 for example with even bigger hardware requirements. I also do not understand all the hate about Windows ME. It crashed with the same frequency as Windows 98 or later Windows 98 Second Edition but maybe my hardware was different ??
\nI do not have any ‘mine’ screenshots from that period as I lost all my 40 GB (huge then) drive of data when I moved/resized the partition with Partition Magic to get some more space from the less filled C: drive. That day I learned hard that “there are people who do backups and people who will do backups”. I never lost data again as I had multiple copies of my data, but the same as Netheril fall the lost data was was gone forever.
\nI always followed various alternatives which led me to try Linux in 2003, after reading about various distributions philosophies I decided to run Slackware Linux with KDE 3. My buddy used Aurox Linux by then (one of the few Linux distributions from Poland) and encouraged me to do the same – especially in the context of fixing possible problems as he already knew it and also as he recently dumped Windows system. But Slackware sounded like a better idea so I took that path instead. At first I dual booted between Windows XP and Slackware Linux cause I had everything worked out on the Windows world while I often felt helpless in the Linux world, so I would reboot into Windows to play some games or find a solution for Linux problem if that was required. I remember how strange the concept of dual clipboards (PRIMARY and SECONDARY) was for me by then. I was amazed why ‘so much better’ system as Linux (at least marketed that way) needs a system tray program to literally manage the clipboard. On Windows it was obvious, you do [CTRL]+[C] to copy and [CTRL]+[V] to paste things, but on Linux there (no I know its X11 feature) there were two clipboards that were synchronized by this little system tray program from KDE 3. It was also unthinkable for me that I will ‘lost’ contents of last/recent [CTRL]+[C] operation if I close the application from which the copy was made. I settled down a little on Slackware but not for long. I really did not liked manual dependency management for packages for example. Also KDE 3 was really ugly and despite trying all possible options I was not able to tweak it into something nice looking.
\nAfter half a year on Slackware I checked the Linux distributions again and decided to try Gentoo Linux. I definitely agree with the image below which visualizes Gentoo Linux experience, especially when You install it for he first time ??
\nOf course I went with the most hardcore version with self building Stage 1 (compiler and toolchain) which was horrible idea at that time because compilation on slow single core machine took forever … but after many hours I got Gentoo installed. I now have to decide which desktop environment to use. I have read a lot of good news about Fluxbox at that time so this is what I tried. It was very weird experience (to create everything in GUI from scratch) but very pleasant one. That recalled me the times of AMIGA … but Linux came in the way too much often. The more I dig into Gentoo Linux the more I read that lots of Gentoo features are based on FreeBSD solutions. Gentoo Portage is a clone of FreeBSD Ports. That ‘central’ /etc/rc.conf system configuration file concept was taken from FreeBSD as well. So I started to gather information about FreeBSD. The (then) FreeBSD website or FreeBSD Ports site (still) felt little outdated to say the least but that did not discouraged me.
\nSomewhere in 2005 I installed FreeBSD 5.4 on my computer. The beginnings were hard, like the earlier step with Gentoo but similarly like Gentoo the FreeBSD project came with a lot of great documentation. While Gentoo documentation is concentrated within various Gentoo Wiki sites the FreeBSD project comes with ‘official’ documentation in the form of Handbook and FAQ. I remember my first questions at the now nonexistent BSDForums.org site – for example one of the first ones – how to scroll the terminal output in the plain console. I now know that I had to push Scroll Lock button but it was something totally new for me.
\nWhy FreeBSD and not OpenBSD or NetBSD? Probably because Gentoo based most their concepts on the FreeBSD solutions, so that led me to FreeBSD instead of the other BSD operating systems. Currently I still use FreeBSD but I keep an steady eye on the OpenBSD, HardenedBSD and DragonFly BSD solutions and improvements.
\nAs the migration path from Linux to FreeBSD is a lot easier – all configuration files from /home can be just copied – the migration was quite fast easy. I again had the Fluxbox configuration which I used on the Gentoo. Now – on FreeBSD – it started to fell even more like AMIGA times. Everything is/has been well thought and had its place and reason. The documentation was good and the FreeBSD Community was second to none.
\nAfter 15 years of using various Windows, UNIX (macOS/AIX/HP-UX/Solaris/OpenSolaris/Illumos/FreeBSD/OpenBSD/NetBSD) and UNIX-like (Linux) systems I always come to conclusion that FreeBSD is the system that sucks least. And sucks least with each release and one day I will write why FreeBSD is such great operating system … if I already haven’t
##News Roundup
\n###OpenBSD on the Desktop: some thoughts
\n\n\nI’ve been using OpenBSD on my ThinkPad X230 for some weeks now, and the experience has been peculiar in some ways.
\n
\nThe OS itself in my opinion is not ready for widespread desktop usage, and the development team is not trying to push it in the throat of anybody who wants a Windows or macOS alternative. You need to understand a little bit of how *NIX systems work, because you’ll use CLI more than UI. That’s not necessarily bad, and I’m sure I learned a trick or two that could translate easily to Linux or macOS. Their development process is purely based on developers that love to contribute and hack around, just because it’s fun. Even the mailing list is a cool place to hang on! Code correctness and security are a must, nothing gets committed if it doesn’t get reviewed thoroughly first - nowadays the first two properties should be enforced in every major operating system.
\nI like the idea of a platform that continually evolves. pledge(2) and unveil(2) are the proof that with a little effort, you can secure existing software better than ever.
\nI like the “sensible defaults” approach, having an OS ready to be used - UI included if you selected it during the setup process - is great.
\nJust install a browser and you’re ready to go.
\nManual pages on OpenBSD are real manuals, not an extension of the “–help” command found in most CLI softwares. They help you understand inner workings of the operating system, no internet connection needed. There are some trade-offs, too.
\nPerformance is not first-class, mostly because of all the security mitigations and checks done at runtime.
\nI write Go code in neovim, and sometimes you can feel a slight slowdown when you’re compiling and editing multiple files at the same time, but usually I can’t notice any meaningful difference. Browsers are a different matter though, you can definitely feel something differs from the experience you can have on mainstream operating systems. But again, trade-offs.
\nTo use OpenBSD on the desktop you must be ready to sacrifice some of the goodies of mainstream OSes, but if you’re searching for a zen place to do your computing stuff, it’s the best you can get right now.
###The history of file type information being available in Unix directories
\n\n\n\n\nThe two things that Unix directory entries absolutely have to have are the name of the directory entry and its ‘inode’, by which we generically mean some stable kernel identifier for the file that will persist if it gets renamed, linked to other directories, and so on. Unsurprisingly, directory entries have had these since the days when you read the raw bytes of directories with read(), and for a long time that was all they had; if you wanted more than the name and the inode number, you had to stat() the file, not just read the directory. Then, well, I’ll quote myself from an old entry on a find optimization:
\n
\n[…], Unix filesystem developers realized that it was very common for programs reading directories to need to know a bit more about directory entries than just their names, especially their file types (find is the obvious case, but also consider things like ‘ls -F’). Given that the type of an active inode never changes, it’s possible to embed this information straight in the directory entry and then return this to user level, and that’s what developers did; on some systems, readdir(3) will now return directory entries with an additional d_type field that has the directory entry’s type.
\nOn Twitter, I recently grumbled about Illumos not having this d_type field. The ensuing conversation wound up with me curious about exactly where d_type came from and how far back it went. The answer turns out to be a bit surprising due to there being two sides of d_type.
\nOn the kernel side, d_type appears to have shown up in 4.4 BSD. The 4.4 BSD /usr/src/sys/dirent.h has a struct dirent that has a d_type field, but the field isn’t documented in either the comments in the file or in the getdirentries(2) manpage; both of those admit only to the traditional BSD dirent fields. This 4.4 BSD d_type was carried through to things that inherited from 4.4 BSD (Lite), specifically FreeBSD, but it continued to be undocumented for at least a while.
\n(In FreeBSD, the most convenient history I can find is here, and the d_type field is present in sys/dirent.h as far back as FreeBSD 2.0, which seems to be as far as the repo goes for releases.)
\nDocumentation for d_type appeared in the getdirentries(2) manpage in FreeBSD 2.2.0, where the manpage itself claims to have been updated on May 3rd 1995 (cf). In FreeBSD, this appears to have been part of merging 4.4 BSD ‘Lite2’, which seems to have been done in 1997. I stumbled over a repo of UCB BSD commit history, and in it the documentation appears in this May 3rd 1995 change, which at least has the same date. It appears that FreeBSD 2.2.0 was released some time in 1997, which is when this would have appeared in an official release.
\nIn Linux, it seems that a dirent structure with a d_type member appeared only just before 2.4.0, which was released at the start of 2001. Linux took this long because the d_type field only appeared in the 64-bit ‘large file support’ version of the dirent structure, and so was only return by the new 64-bit getdents64() system call. This would have been a few years after FreeBSD officially documented d_type, and probably many years after it was actually available if you peeked at the structure definition.
\nAs far as I can tell, d_type is present on Linux, FreeBSD, OpenBSD, NetBSD, Dragonfly BSD, and Darwin (aka MacOS or OS X). It’s not present on Solaris and thus Illumos. As far as other commercial Unixes go, you’re on your own; all the links to manpages for things like AIX from my old entry on the remaining Unixes appear to have rotted away.
\nSidebar: The filesystem also matters on modern Unixes
\nEven if your Unix supports d_type in directory entries, it doesn’t mean that it’s supported by the filesystem of any specific directory. As far as I know, every Unix with d_type support has support for it in their normal local filesystems, but it’s not guaranteed to be in all filesystems, especially non-Unix ones like FAT32. Your code should always be prepared to deal with a file type of DT_UNKNOWN.
\nIt’s also possible to have things the other way around, where you have a filesystem with support for file type information in directories that’s on a Unix that doesn’t support it. There are a number of plausible reasons for this to happen, but they’re either obvious or beyond the scope of this entry.
###Multiboot Pinebook KDE neon
\n\n\n\n\nRecently a KDE neon image for the Pinebook was announced. There is a new image, with a handful of fixes, which the KDE Plasma team has been working on over the past week and a half.
\n
\nHere’s a picture of my Pinebook running KDE neon — watching Panic! At the Disco’s High Hopes — sitting in front of my monitor that’s hooked up to one of my openSUSE systems. There are still some errata, and watching video sucks up battery, but for hacking on documentation from my hammock in the garden, or doing IRC meetings it’s a really nice machine.
\nBut one of the neat things about running KDE neon off of an SD card on the Pinebook is that it’s portable — that SD card can move around. So let’s talk about multiboot in the sense of “booting the same OS storage medium in different hardware units” rather than “booting different OS from a medium in a single hardware unit”. On these little ARM boards, u-boot does all the heavy lifting early in the boot process. So to re-use the KDE neon Pinebook image on another ARM board, the u-boot blocks need to be replaced.
\nI have the u-boot from a Pine64 image (I forget what) lying around, 1015 blocks of 1024 bytes, which I can dd over the u-boot blocks on the SD card, dd bs=1k conv=notrunc,sync if=uboot.img of=/dev/da0 seek=8, and then the same SD card, with the filesystem and data from the Pinebook, will boot on the Pine64 board. Of course, to move the SD card back again, I need to restore the Pinebook u-boot blocks.
\nHere’s a picture of my Pineboard (the base is a piece of the garden fence, it’s Douglas pine, with 4mm threaded rods acting as the corner posts for my Pine64 mini-rack), with power and network and a serial console attached, along with the serial console output of the same.
\nThe nice thing here is that the same software stack runs on the Pine64 but then has a wired network — which in turn means that if I switch on the other boards in that mini-rack, I’ve got a distcc-capable cluster for fast development, and vast NFS storage (served from ZFS on my FreeBSD machines) for source. I can develop in a high(er) powered environment, and then swap the card around into the Pinebook for testing-on-the-go.
\nSo to sum up: you can multiboot the KDE neon Pinebook image on other Pine64 hardware (i.e. the Pine64 board). To do so, you need to swap around u-boot blocks. The blocks can be picked out of an image built for each board, and then a particular image (e.g. the latest KDE neon Pinebook) can be run on either board.
##Beastie Bits
\n\n##Feedback/Questions
\n\nWe report from our experiences at EuroBSDcon, disenchant software, LLVM 7.0.0 has been released, Thinkpad BIOS update options, HardenedBSD Foundation announced, and ZFS send vs. rsync.
\n\n##Headlines
\n\n###[FreeBSD DevSummit & EuroBSDcon 2018 in Romania]
\n\n\n\n\nSelfhosting as an alternative to the public cloud (by Albert Dengg)
\n
\nUsing Boot Environments at Scale (by Allan Jude)
\nLivepatching FreeBSD kernel (by Maciej Grochowski)
\nFreeBSD: What to (Not) Monitor (by Andrew Fengler)
\nFreeBSD Graphics (by Niclas Zeising)
\n\nHacking together a FreeBSD presentation streaming box – For as little as possible (by Tom Jones)
\n
\nIntroduction of FreeBSD in new environments (by Baptiste Daroussin)
\nKeynote: Some computing and networking historical perspectives (by Ron Broersma)
\nLivepatching FreeBSD kernel (by Maciej Grochowski)
\nFreeBSD: What to (Not) Monitor (by Andrew Fengler)
\nBeing a BSD user (by Roller Angel)
\nFrom “Hello World” to the VFS Layer: building a beadm for DragonFly BSD (by Michael Voight)
\n\n\nI’ve been programming for 15 years now. Recently our industry’s lack of care for efficiency, simplicity, and excellence started really getting to me, to the point of me getting depressed by my own career and the IT in general.
\n
\nModern cars work, let’s say for the sake of argument, at 98% of what’s physically possible with the current engine design. Modern buildings use just enough material to fulfill their function and stay safe under the given conditions. All planes converged to the optimal size/form/load and basically look the same.
\nOnly in software, it’s fine if a program runs at 1% or even 0.01% of the possible performance. Everybody just seems to be ok with it. People are often even proud about how much inefficient it is, as in “why should we worry, computers are fast enough”:
\n@tveastman: I have a Python program I run every day, it takes 1.5 seconds. I spent six hours re-writing it in rust, now it takes 0.06 seconds. That efficiency improvement means I’ll make my time back in 41 years, 24 days :-)
\nYou’ve probably heard this mantra: “programmer time is more expensive than computer time”. What it means basically is that we’re wasting computers at an unprecedented scale. Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time.
\n\n\nLook around: our portable computers are thousands of times more powerful than the ones that brought man to the moon. Yet every other webpage struggles to maintain a smooth 60fps scroll on the latest top-of-the-line MacBook Pro. I can comfortably play games, watch 4K videos but not scroll web pages? How is it ok?
\n
\nGoogle Inbox, a web app written by Google, running in Chrome browser also by Google, takes 13 seconds to open moderately-sized emails:
\nIt also animates empty white boxes instead of showing their content because it’s the only way anything can be animated on a webpage with decent performance. No, decent doesn’t mean 60fps, it’s rather “as fast as this web page could possibly go”. I’m dying to see web community answer when 120Hz displays become mainstream. Shit barely hits 60Hz already.
\nWindows 10 takes 30 minutes to update. What could it possibly be doing for that long? That much time is enough to fully format my SSD drive, download a fresh build and install it like 5 times in a row.
\nPavel Fatin: Typing in editor is a relatively simple process, so even 286 PCs were able to provide a rather fluid typing experience.
\nModern text editors have higher latency than 42-year-old Emacs. Text editors! What can be simpler? On each keystroke, all you have to do is update tiny rectangular region and modern text editors can’t do that in 16ms. It’s a lot of time. A LOT. A 3D game can fill the whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come?
\nAs a general trend, we’re not getting faster software with more features. We’re getting faster hardware that runs slower software with the same features. Everything works way below the possible speed. Ever wonder why your phone needs 30 to 60 seconds to boot? Why can’t it boot, say, in one second? There are no physical limitations to that. I would love to see that. I would love to see limits reached and explored, utilizing every last bit of performance we can get for something meaningful in a meaningful way.
\n\n\nAnd then there’s bloat. Web apps could open up to 10× faster if you just simply block all ads. Google begs everyone to stop shooting themselves in their feet with AMP initiative—a technology solution to a problem that doesn’t need any technology, just a little bit of common sense. If you remove bloat, the web becomes crazy fast. How smart do you have to be to understand that?
\n
\nAndroid system with no apps takes almost 6 Gb. Just think for a second how obscenely HUGE that number is. What’s in there, HD movies? I guess it’s basically code: kernel, drivers. Some string and resources too, sure, but those can’t be big. So, how many drivers do you need for a phone?
\nWindows 95 was 30Mb. Today we have web pages heavier than that! Windows 10 is 4Gb, which is 133 times as big. But is it 133 times as superior? I mean, functionally they are basically the same. Yes, we have Cortana, but I doubt it takes 3970 Mb. But whatever Windows 10 is, is Android really 150% of that?
\nGoogle keyboard app routinely eats 150 Mb. Is an app that draws 30 keys on a screen really five times more complex than the whole Windows 95? Google app, which is basically just a package for Google Web Search, is 350 Mb! Google Play Services, which I do not use (I don’t buy books, music or videos there)—300 Mb that just sit there and which I’m unable to delete.
\nAll that leaves me around 1 Gb for my photos after I install all the essential (social, chats, maps, taxi, banks etc) apps. And that’s with no games and no music at all! Remember times when an OS, apps and all your data fit on a floppy?
\nYour desktop todo app is probably written in Electron and thus has userland driver for Xbox 360 controller in it, can render 3d graphics and play audio and take photos with your web camera.
\nA simple text chat is notorious for its load speed and memory consumption. Yes, you really have to count Slack in as a resource-heavy application. I mean, chatroom and barebones text editor, those are supposed to be two of the less demanding apps in the whole world. Welcome to 2018.
\nAt least it works, you might say. Well, bigger doesn’t imply better. Bigger means someone has lost control. Bigger means we don’t know what’s going on. Bigger means complexity tax, performance tax, reliability tax. This is not the norm and should not become the norm. Overweight apps should mean a red flag. They should mean run away scared.
\n\n\nI want to see progress. I want change. I want state-of-the-art in software engineering to improve, not just stand still. I don’t want to reinvent the same stuff over and over, less performant and more bloated each time. I want something to believe in, a worthy end goal, a future better than what we have today, and I want a community of engineers who share that vision.
\n
\nWhat we have today is not progress. We barely meet business goals with poor tools applied over the top. We’re stuck in local optima and nobody wants to move out. It’s not even a good place, it’s bloated and inefficient. We just somehow got used to it.
\nSo I want to call it out: where we are today is bullshit. As engineers, we can, and should, and will do better. We can have better tools, we can build better apps, faster, more predictable, more reliable, using fewer resources (orders of magnitude fewer!). We need to understand deeply what are we doing and why. We need to deliver: reliably, predictably, with topmost quality. We can—and should–take pride in our work. Not just “given what we had…”—no buts!
\nI hope I’m not alone at this. I hope there are people out there who want to do the same. I’d appreciate if we at least start talking about how absurdly bad our current situation in the software industry is. And then we maybe figure out how to get out.
##News Roundup
\n###[llvm-announce] LLVM 7.0.0 Release
I am pleased to announce that LLVM 7 is now available.\n\nGet it here: https://llvm.org/releases/download.html#7.0.0\n\nThe release contains the work on trunk up to SVN revision 338536 plus\nwork on the release branch. It is the result of the community's work\nover the past six months, including: function multiversioning in Clang\nwith the 'target' attribute for ELF-based x86/x86_64 targets, improved\nPCH support in clang-cl, preliminary DWARF v5 support, basic support\nfor OpenMP 4.5 offloading to NVPTX, OpenCL C++ support, MSan, X-Ray\nand libFuzzer support for FreeBSD, early UBSan, X-Ray and libFuzzer\nsupport for OpenBSD, UBSan checks for implicit conversions, many\nlong-tail compatibility issues fixed in lld which is now production\nready for ELF, COFF and MinGW, new tools llvm-exegesis, llvm-mca and\ndiagtool. And as usual, many optimizations, improved diagnostics, and\nbug fixes.\n\nFor more details, see the release notes:\nhttps://llvm.org/releases/7.0.0/docs/ReleaseNotes.html\nhttps://llvm.org/releases/7.0.0/tools/clang/docs/ReleaseNotes.html\nhttps://llvm.org/releases/7.0.0/tools/clang/tools/extra/docs/ReleaseNotes.html\nhttps://llvm.org/releases/7.0.0/tools/lld/docs/ReleaseNotes.html\n\nThanks to everyone who helped with filing, fixing, and code reviewing\nfor the release-blocking bugs!\n\nSpecial thanks to the release testers and packagers: Bero\nRosenkränzer, Brian Cain, Dimitry Andric, Jonas Hahnfeld, Lei Huang\nMichał Górny, Sylvestre Ledru, Takumi Nakamura, and Vedant Kumar.\n\nFor questions or comments about the release, please contact the\ncommunity on the mailing lists. Onwards to LLVM 8!\n\nCheers,\nHans\n
\n\n###Update your Thinkpad’s bios with Linux or OpenBSD
\n\n\n\n\nAt first, go to the Lenovo website and download your new bios:
\n
\n\n\nFor me the file is called like this : r0iuj25wd.iso
\n
\n\n\nNow you will need to install geteltorito.
\n
$ doas pkg_add geteltorito
\nquirks-3.7 signed on 2018-09-09T13:15:19Z
\ngeteltorito-0.6: ok
$ sudo apt-get install genisoimage
$ geteltorito -o bios_update.img r0iuj25wd.iso
\nBooting catalog starts at sector: 20
\nManufacturer of CD: NERO BURNING ROM VER 12
\nImage architecture: x86
\nBoot media type is: harddisk
\nEl Torito image starts at sector 27 and has 43008 sector(s) of 512 Bytes
\n
\nImage has been written to file "bios_update.img".
\nThis will create a file called bios_update.img.
\n\n\nPlease check twice on your computer the name of your USB key.
\n
$ doas dd if=bios_update.img of=/dev/rsd1c
$ sudo dd if=bios_update.img of=/dev/sda
\n\n\nNow all you need is to reboot, to boot on your USB key and follow the instructions. Enjoy 😉
\n
###Announcing The HardenedBSD Foundation
\n\n\n\n\nIn June of 2018, we announced our intent to become a not-for-profit, tax-exempt 501©(3) organization in the United States. It took a dedicated team months of work behind-the-scenes to make that happen. On 06 September 2018, HardenedBSD Foundation Corp was granted 501©(3) status, from which point all US-based persons making donations can deduct the donation from their taxes.
\n
\nWe are grateful for those who contribute to HardenedBSD in whatever way they can. Thank you for making HardenedBSD possible. We look forward to a bright future, driven by a helpful and positive community.
###How you migrate ZFS filesystems matters
\n\n\n\n\nIf you want to move a ZFS filesystem around from one host to another, you have two general approaches; you can use ‘zfs send’ and ‘zfs receive’, or you can use a user level copying tool such as rsync (or ‘tar -cf | tar -xf’, or any number of similar options). Until recently, I had considered these two approaches to be more or less equivalent apart from their convenience and speed (which generally tilted in favour of ‘zfs send’). It turns out that this is not necessarily the case and there are situations where you will want one instead of the other.
\n
\nWe have had two generations of ZFS fileservers so far, the Solaris ones and the OmniOS ones. When we moved from the first generation to the second generation, we migrated filesystems across using ‘zfs send’, including the filesystem with my home directory in it (we did this for various reasons). Recently I discovered that some old things in my filesystem didn’t have file type information in their directory entries. ZFS has been adding file type information to directories for a long time, but not quite as long as my home directory has been on ZFS.
\nThis illustrates an important difference between the ‘zfs send’ approach and the rsync approach, which is that zfs send doesn’t update or change at least some ZFS on-disk data structures, in the way that re-writing them from scratch from user level does. There are both positives and negatives to this, and a certain amount of rewriting does happen even in the ‘zfs send’ case (for example, all of the block pointers get changed, and ZFS will re-compress your data as applicable).
\nI knew that in theory you had to copy things at the user level if you wanted to make sure that your ZFS filesystem and everything in it was fully up to date with the latest ZFS features. But I didn’t expect to hit a situation where it mattered in practice until, well, I did. Now I suspect that old files on our old filesystems may be partially missing a number of things, and I’m wondering how much of the various changes in ‘zfs upgrade -v’ apply even to old data.
\n(I’d run into this sort of general thing before when I looked into ext3 to ext4 conversion on Linux.)
\nWith all that said, I doubt this will change our plans for migrating our ZFS filesystems in the future (to our third generation fileservers). ZFS sending and receiving is just too convenient, too fast and too reliable to give up. Rsync isn’t bad, but it’s not the same, and so we only use it when we have to (when we’re moving only some of the people in a filesystem instead of all of them, for example).
\nPS: I was going to try to say something about what ‘zfs send’ did and didn’t update, but having looked briefly at the code I’ve concluded that I need to do more research before running my keyboard off. In the mean time, you can read the OpenZFS wiki page on ZFS send and receive, which has plenty of juicy technical details.
\nPPS: Since eliminating all-zero blocks is a form of compression, you can turn zero-filled files into sparse files through a ZFS send/receive if the destination has compression enabled. As far as I know, genuine sparse files on the source will stay sparse through a ZFS send/receive even if they’re sent to a destination with compression off.
##Beastie Bits
\n\n##Feedback/Questions
\n\nFreeBSD and DragonflyBSD benchmarks on AMD’s Threadripper, NetBSD 7.2 has been released, optimized out DTrace kernel symbols, stuck UEFI bootloaders, why ed is not a good editor today, tell your BSD story, and more.
\n\n##Headlines
\n###FreeBSD & DragonFlyBSD Put Up A Strong Fight On AMD’s Threadripper 2990WX, Benchmarks Against Linux
\n\n\nThe past two weeks I have been delivering a great deal of AMD Threadripper 2990WX benchmarks on Linux as well as some against Windows and Windows Server. But recently I got around to trying out some of the BSD operating systems on this 32-core / 64-thread processor to see how they would run and to see whether they would have similar scaling issues or not like we’ve seen on the Windows side against Linux. In this article are FreeBSD and DragonFlyBSD benchmarks with the X399 + 2990WX compared to a few Linux distributions.
\n
\nThe BSDs I focused my testing on were FreeBSD 11.2-STABLE and 12.0-CURRENT/ALPHA1 (the version in development) as well as iX System’s TrueOS that is tracking FreeBSD 12.0-CURRENT. Also included were DragonFlyBSD, with FreeBSD and DragonFlyBSD being tied as my favorite operating systems when it comes to the BSDs. When it came to FreeBSD 11.2-STABLE and 12.0-ALPHA1 on the Threadripper 2990WX, it worked out surprisingly well. I encountered no real issues during my two days of benchmarking on FreeBSD (and TrueOS). It was a great experience and FreeBSD was happy to exploit the 64 threads on the system.
\nDragonFlyBSD was a bit of a different story… Last week when I started this BSD testing I tried DragonFly 5.2.2 as the latest stable release as well as a DragonFlyBSD 5.3 development snapshot from last week: both failed to boot in either BIOS or UEFI modes.
\nBut then a few days ago DragonFlyBSD lead developer Matthew Dillon bought himself a 2990WX platform. He made the necessary changes to get DragonFlyBSD 5.3 working and he ended up finding really great performance and potential out of the platform. So I tried the latest DragonFlyBSD 5.3 daily ISO on 22 August and indeed it now booted successfully and we were off to the races. Thus there are some DragonFlyBSD 5.3 benchmarks included in this article too.
\nJust hours ago, Matthew Dillon landed some 2990WX topology and scheduler enhancements but that fell out of the scope of when DragonFly was installed on this system. But over the weekend or so I plan to re-test DragonFlyBSD 5.3 and see how those optimizations affect the overall 2990WX performance now on that BSD. DragonFlyBSD 5.4 stable should certainly be an interesting release on several fronts!
\nWith FreeBSD 11.2-STABLE and 12.0-ALPHA1 I ran benchmarks when using their stock compiler (LLVM Clang 6.0) as well as GCC 7.3 obtained via GCC 7.3. That was done to rule out compiler differences in benchmarking against the GCC-based Linux distributions. On DragonFlyBSD 5.3 it defaults to the GCC 5.4.1 but via pkg I also did a secondary run when upgraded to GCC 7.3.
\nThe hardware and BIOS/UEFI settings were maintained the same throughout the entire benchmarking process. The system was made up of the AMD Ryzen Threadripper 2990WX at stock speeds, the ASUS ROG ZENITH EXTREME motherboard, 4 x 8GB DDR4-3200MHz memory, Samsung 970 EVO 500GB NVMe SSD, and Radeon RX Vega 56 graphics card.
\nAll of these Linux vs. BSD benchmarks were carried out in a fully-automated and reproducible manner using the open-source Phoronix Test Suite benchmarking framework.
\nWhile for the last of today’s BSD vs. Linux benchmarking on the Threadripper 2990WX, the Linux distributions came out slightly ahead of FreeBSD and DragonFlyBSD with GCC (another test having issues with Clang 6.0 on the BSDs).
\nOverall, I was quite taken away by the BSD performance on the Threadripper 2990WX – particularly FreeBSD. In a surprising number of benchmarks, the BSDs were outperforming the tested Linux distributions though often by incredibly thin margins. Still, quite an accomplishment for these BSD operating systems and considering how much better Linux is already doing than Windows 10 / Windows Server on this 32-core / 64-thread processor. Then again, the BSDs like Linux have a long history of running on high core/thread-count systems, super computers, and other HPC environments.
\nIt will be interesting to see how much faster DragonFlyBSD can run given today’s commit to its kernel with scheduler and topology improvements for the 2990WX. Those additional DragonFlyBSD benchmarks will be published in the coming days once they are completed.
\n\n\nThe NetBSD Project is pleased to announce NetBSD 7.2, the second feature update of the NetBSD 7 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.
\n
The NetBSD 7.2 release is a maintenance release of the netbsd-7 branch, which had it's first major release, NetBSD 7.0 in September 2015. A lot of security features have been added to later NetBSD versions, and for new installations we highly recommend using our latest release, NetBSD 8.0 instead.
\n\n\nComplete source and binaries for NetBSD 7.2 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, SUP, and other services may be found at https://www.NetBSD.org/mirrors/. We encourage users who wish to install via ISO or USB disk images to download via BitTorrent by using the torrent files supplied in the images area. A list of hashes for the NetBSD 7.2 distribution has been signed with the well-connected PGP key for the NetBSD Security Officer: https://cdn.NetBSD.org/pub/NetBSD/security/hashes/NetBSD-7.2_hashes.asc
\n
\nNetBSD is free. All of the code is under non-restrictive licenses, and may be used without paying royalties to anyone. Free support services are available via our mailing lists and website. Commercial support is available from a variety of sources. More extensive information on NetBSD is available from our website:
##News Roundup
\n###Including optimized-out kernel symbols in dtrace on FreeBSD
\n\n\nHave you ever had dtrace(1) on FreeBSD fail to list a probe that should exist in the kernel? This is because Clang will optimize-out some functions. The result is ctfconvert(1) will not generate debugging symbols that dtrace(1) uses to identify probes. I have a quick solution to getting those probes visible to dtrace(1).
\n
\n\n\nIn my case, I was trying to instrument on ieee80211_ioctl_get80211, whose sister function ieee80211_ioctl_set80211 has a dtrace(1) probe in the generic FreeBSD 11 and 12 kernels. Both functions are located in /usr/src/sys/net80211/ieee80211_ioctl.c.
\n
\n\n\nMy first attempt was to add to /etc/make.conf as follows and recompile the kernel.
\n
CFLAGS+=-O0 and -fno-inline-functions
\n\n\nThis failed to produce the dtrace(1) probe. Several other attempts failed and I was getting inconsistent compilation results (Is it me or is ieee80211_ioctl.c compiled with different flags if NO_CLEAN=1 is set?). When I manually compiled the object file by copying the compilation line for the object file and adding -O0 -fno-inline-functions, nm(1) on both the object file and kernel demonstrated that the symbol was present. I installed the kernel, rebooted and it was listed as a dtrace probe. Great!
\n
\n\n\nBut as I continued to debug my WiFi driver (oh yeah, I’m very slowly extending rtwn(4)), I found myself rebuilding the kernel several times and frequently rebooting. Why not do this across the entire kernel?
\n
\n\n\nAfter hacking around, my solution was to modify the build scripts. My solution was to edit /usr/src/sys/conf/kern.pre.mk and modify all optimization level 2 to optimization level 0. The following is my diff(1) on FreeBSD 12.0-CURRENT.
\n
\n\n\nThis seems like a hack rather than a long-term solution. Either the problem is with the hard-coded optimization flags, or the inability to overwrite them in all places in make.conf.
\n
\nRemoving optimizations is only something I would do in a non-production kernel, so its as if I have to choose between optimizations for a production kernel or having dtrace probes. But dtrace explicitly markets itself as not impactful on production.
\nUsing the dtrace pony as your featured image on WordPress does not render properly and must be rotated and modified. Blame Bryan Cantrill.
\nIf you have a better solution, please let me know and I will update the article, but this works for me!
###FreeBSD: UEFI Bootloader stuck on BootCurrent/BootOrder/BootInfo on Asus Motherboards (and fix!)
\n\n\n\n\nStarting with FreeBSD CURRENT from about a few weeks of posting date, but including FreeBSD 12 alpha releases (not related to DEC Alpha), I noticed one thing: When I boot FreeBSD from UEFI on a homebuilt desktop with a Asus H87M-E motherboard, and have Root on ZFS, the bootloader gets stuck on lines like BootCurrent, BootOrder, and BootInfo. This issue occurs when I try to boot directly to efi\\boot\\bootx64.efi.
\n
\n\n\nOne person had a similar issue on a Asus H87I-PLUS motherboard. This issue may or may not exist on other Asus motherboards, desktops, or laptops. This may be specific to Asus motherboards for Intel’s Haswell, but may also exist on newer systems (e.g. Skylake) or older (e.g. Ivy Bridge) with Asus motherboards, as well as Asus desktops or laptops.
\n
\n\n\nKeep in mind that I am not going to talk about this issue and third-party UEFI boot managers such as rEFInd here.
\n
\nThe first option is rather straightforward: you need to make sure your computer has “Secure Boot” disabled and “Legacy Boot” or “CSM” enabled. Then, you need to make sure FreeBSD is installed in BIOS mode. However, this solution is (in my opinion) suboptimal. Why? Because:
\nYou won’t be able to use hard drives bigger than 2TB
\nYou are limited to MBR Partitioning on Asus motherboards with UEFI as Asus motherboards refuse to boot GPT partitioned disks in BIOS mode
\nLegacy BIOS mode may not exist on future computers or motherboards (although those systems may not have this issue, and this issue may get fixed by then)
\nThe second option, however, is less straightforward, but will let you keep UEFI. Many UEFI systems, including affected Asus motherboards described here, include a boot manager built into the UEFI. FreeBSD includes a tool called efibootmgr to manage this, similar to the similarly-named tool in Linux, but with a different syntax.
###Why ed(1) is not a good editor today
\n\n\n\n\nI’ll start with my tweet:
\n
Heretical Unix opinion time: ed(1) may be the 'standard Unix editor', but it is not a particularly good editor outside of a limited environment that almost never applies today.
\n\n\nThere is a certain portion of Unixdom that really likes ed(1), the ‘standard Unix editor’. Having actually used ed for a not insignificant amount of time (although it was the friendlier ‘UofT ed’ variant), I have some reactions to what I feel is sometimes overzealous praise of it. One of these is what I tweeted.
\n
\nThe fundamental limitation of ed is that it is what I call an indirect manipulation interface, in contrast to the explicit manipulation interfaces of screen editors like vi and graphical editors like sam (which are generally lumped together as ‘visual’ editors, so called because they actually show you the text you’re editing). When you edit text in ed, you have some problems that you don’t have in visual editors; you have to maintain in your head the context of what the text looks like (and where you are in it), you have to figure out how to address portions of that text in order to modify them, and finally you have to think about how your edit commands will change the context. Copious use of ed’s p command can help with the first problem, but nothing really deals with the other two. In order to use ed, you basically have to simulate parts of ed in your head.
\nEd is a great editor in situations where the editor explicitly presenting this context is a very expensive or outright impossible operation. Ed works great on real teletypes, for example, or over extremely slow links where you want to send and receive as little data as possible (and on real teletypes you have some amount of context in the form of an actual printout that you can look back at). Back in the old days of Unix, this described a fairly large number of situations; you had actual teletypes, you had slow dialup links (and later slow, high latency network links), and you had slow and heavily overloaded systems.
\nHowever, that’s no longer the situation today (at least almost all of the time). Modern systems and links can easily support visual editors that continually show you the context of the text and generally let you more or less directly manipulate it (whether that is through cursoring around it or using a mouse). Such editors are easier and faster to use, and they leave you with more brainpower free to think about things like the program you’re writing (which is the important thing).
\nIf you can use a visual editor, ed is not a particularly good editor to use instead; you will probably spend a lot of effort (and some amount of time) on doing by hand something that the visual editor will do for you. If you are very practiced at ed, maybe this partly goes away, but I maintain that you are still working harder than you need to be.
\nThe people who say that ed is a quite powerful editor are correct; ed is quite capable (although sadly limited by only editing a single file). It’s just that it’s also a pain to use.
\n(They’re also correct that ed is the foundation of many other things in Unix, including sed and vi. But that doesn’t mean that the best way to learn or understand those things is to learn and use ed.)
\nThis doesn’t make ed a useless, vestigial thing on modern Unix, though. There are uses for ed in non-interactive editing, for example. But on modern Unix, ed is a specialized tool, much like dc. It’s worth knowing that ed is there and roughly what it can do, but it’s probably not worth learning how to use it before you need it. And you’re unlikely to ever be in a situation where it’s the best choice for interactive editing (and if you are, something has generally gone wrong).
\n(But if you enjoy exploring the obscure corners of Unix, sure, go for it. Learn dc too, because it’s interesting in its own way and, like ed, it’s one of those classical old Unix programs.)
##Beastie Bits
\n\n##Feedback/Questions
\n\nMitigating Spectre/Meltdown on HP Proliant servers, omniOS installation setup, debugging a memory corruption issue on OpenBSD, CfT for OpenZFS native encryption, Asigra TrueNAS backup appliance shown at VMworld, NetBSD 6 EoL, and more.
\n
##Headlines
\n###How to mitigate Spectre and Meltdown on an HP Proliant server with FreeBSD
\n\n\nAs recently announced in a previous article I wanted to write a couple of guides on how to mitigate Spectre and Meltdown vulnerabilities in GNU/Linux and UNIX environments. It is always a good and I hope a standard practice to have your systems patched and if they aren’t for whatever the reason (that legacy thing you’re carrying on for ages) you may take the necessary extra steps to protect your environment. I never planned to do any article on patching anything. Nowadays it’s a no brainer and operating systems have provided the necessary tools for this to be easy and as smooth as possible. So why this article?
\n
\nSpectre and Meltdown are both hardware vulnerabilities. Major ones. They are meaningful for several reasons among them the world wide impact since they affect Intel and AMD systems which are ubiquitous. And second because patching hardware is not as easy, for the manufacturer and for the users or administrators in charge of the systems. There is still no known exploit around left out in the open hitting servers or desktops anywhere. The question is not if it will ever happen. The question is when will it happen. And it may be sooner than later. This is why big companies, governments and people in charge of big deployments are patching or have already patched their systems. But have you done it to your system? I know you have a firewall. Have you thought about CVE-2018-3639? This particular one could make your browser being a vector to get into your system. So, no, there is no reason to skip this.
\nPatching these set of vulnerabilities implies some more steps and concerns than updating the operating system. If you are a regular Windows user I find rare you to be here and many of the things you will read may be foreign to you. I am not planning to do a guide on Windows systems since I believe someone else has or will do it and will do it better than me since I am not a pro Windows user. However there is one basic and common thing for all OS’s when dealing with Spectre and Meltdown and that is a microcode update is necessary for the OS patches to effectively work.
\nWhat is microcode? You can read the Wikipedia article but in short it is basically a layer of code that allows chip manufacturers to deal with modifications on the hardware they’ve produced and the operating systems that will manage that hardware. Since there’s been some issues (namely Spectre and Meltdown) Intel and AMD respectively have released a series of microcode updates to address those problems. First series did come with serious problems and some regressions, to the point GNU/Linux producers stopped releasing the microcode updates through their release channels for updates and placed the ball on Intel’s roof. Patching fast does always include risks, specially when dealing with hardware. OS vendors have resumed their microcode update releases so all seems to be fine now.
\nIn order to update the microcode we’re faced with two options. Download the most recent BIOS release from our vendor, provided it patches the Spectre and Meltdown vulnerabilities, or patch it from the OS. If your hardware vendor has decided not to provide support on your hardware you are forced to use the latter solution. Yes, you can still keep your hardware. They usually come accompanied with a “release notes” file where there are some explanatory notes on what is fixed, what is new, etc. To make the search easy for you a news site collected the vendors list and linked the right support pages for anyone to look. In some scenarios it would be desirable not to replace the whole BIOS but just update the microcode from the OS side. In my case I should update an HP Proliant ML110 G7 box and the download link for that would be this.
\nInstead of using the full blown BIOS update path we’ll use the inner utilities to patch Spectre and Meltdown on FreeBSD. So let’s put our hands on it
###A look beyond the BSD teacup: OmniOS installation
\n\n\n\n\nFive years ago I wrote a post about taking a look beyond the Linux teacup. I was an Arch Linux user back then and since there were projects like ArchBSD (called PacBSD today) and Arch Hurd, I decided to take a look at and write about them. Things have changed. Today I’m a happy FreeBSD user, but it’s time again to take a look beyond the teacup of operating systems that I’m familiar with.
\n
\n\n\nThere are a couple of reasons. The Solaris derivatives are the other big community in the *nix family besides Linux and the BSDs and we hadn’t met so far. Working with ZFS on FreeBSD, I now and then I read messages that contain a reference to Illumos which certainly helps to keep up the awareness. Of course there has also been a bit of curiosity – what might the OS be like that grew ZFS?
\n
\nAlso the Ravenports project that I participate in planned to support Solaris/Illumos right from the beginning. I wanted to at least be somewhat “prepared” when support for that platform would finally land. So I did a little research on the various derivatives available and settled on the one that I had heard a talk about at last year’s conference of the German Unix Users Group: “OmniOS – Solaris for the Rest of Us”. I would have chosen SmartOS as I admire what Bryan Cantrill does but for getting to know Illumos I prefer a traditional installation over a run-from-RAM system.
\nOf course FreeBSD is not run by corporations, especially when compared to the state of Linux. And when it comes to sponsoring, OpenBSD also takes the money… When it comes to FreeBSD developers, there’s probably some truth to the claim that some of them are using macOS as their desktop systems while OpenBSD devs are more likely to develop on their OS of choice. But then there’s the statement that “every innovation in the past decade comes from Solaris”. Bhyve alone proves this wrong. But let’s be honest: Two of the major technologies that make FreeBSD a great platform today – ZFS and DTrace – actually do come from Solaris. PAM originates there and a more modern way of managing services as well. Also you hear good things about their zones and a lot of small utilities in general.
\nIn the end it was a lack of time that made me cheat and go down the easiest road: Create a Vagrantfile and just pull a VM image of the net that someone else had prepared… This worked to just make sure that the Raven packages work on OmniOS. I was determined to return, though – someday. You know how things go: “someday” is a pretty common alias for “probably never, actually.”
\nBut then I heard about a forum post on the BSDNow! podcast. The title “Initial OmniOS impressions by a BSD user” caught my attention. I read that it was written by somebody who had used FreeBSD for years but loathed the new Code of Conduct enough to leave. I also oppose the Conduct and have made that pretty clear in my February post [ ! -z ${COC} ] && exit 1. As stated there, I have stayed with my favorite OS and continue to advocate it. I decided to stop reading the post and try things out on my own instead. Now I’ve finally found the time to do so.
\n\n\nThat’s it for part one. In part two I’ll try to make the system useful. So far I have run into a problem that I haven’t been able to solve. But I have some time now to figure things out for the next post. Let’s see if I manage to get it working or if I have to report failure!
\n
###What are all these types of memory in top(1)?
\n\n\n\n\nActive - Contains memory “actively” (recently) being used by applications
\n
\nInactive - Contains memory that has not been touched recently, or was released from the Buffer Cache
\nLaundry - Contains memory that Inactive but still potentially contains useful data that needs to be stored before this memory can be used again
\nWired - Memory that cannot be swapped out, including the kernel, network stack, and the ZFS ARC
\nBuf - Buffer Cache, used my UFS and most filesystems except ZFS (which uses the ARC)
\nFree - Memory that is immediately available for use by the rest of the system
##News Roundup
\n###OpenBSD saves me again! — Debug a memory corruption issue
\n\n\nYesterday, I came across a third-part library issue, which crashes at allocating memory:
\n
Program terminated with signal SIGSEGV, Segmentation fault.
\n#0 0x00007f594a5a9b6b in _int_malloc () from /usr/lib/libc.so.6
\n(gdb) bt
\n#0 0x00007f594a5a9b6b in _int_malloc () from /usr/lib/libc.so.6
\n#1 0x00007f594a5ab503 in malloc () from /usr/lib/libc.so.6
\n#2 0x00007f594b13f159 in operator new (sz=5767168) at /build/gcc/src/gcc/libstdc++-v3/libsupc++/new_op.cc:50
\n\n\nIt is obvious that the memory tags are corrupted, but who is the murder? Since the library involves a lot of maths computation, it is not an easy task to grasp the code quickly. So I need to find another way:
\n
\n(1) Open all warnings during compilation: -Wall. Nothing found.
\n(2) Use valgrind, but unfortunately, valgrind crashes itself:
valgrind: the 'impossible' happened:
\nKilled by fatal signal
\n
\nhost stacktrace:
\n==43326== at 0x58053139: get_bszB_as_is (m_mallocfree.c:303)
\n==43326== by 0x58053139: get_bszB (m_mallocfree.c:315)
\n==43326== by 0x58053139: vgPlain_arena_malloc (m_mallocfree.c:1799)
\n==43326== by 0x5800BA84: vgMemCheck_new_block (mc_malloc_wrappers.c:372)
\n==43326== by 0x5800BD39: vgMemCheck___builtin_vec_new (mc_malloc_wrappers.c:427)
\n==43326== by 0x5809F785: do_client_request (scheduler.c:1866)
\n==43326== by 0x5809F785: vgPlain_scheduler (scheduler.c:1433)
\n==43326== by 0x580AED50: thread_wrapper (syswrap-linux.c:103)
\n==43326== by 0x580AED50: run_a_thread_NORETURN (syswrap-linux.c:156)
\n
\nsched status:
\nrunning_tid=1
\n\n\n(3) Change compiler, use clang instead of gcc, and hope it can give me some clues. Still no effect.
\n
\n(4) Switch Operating System from Linux to OpenBSD, the program crashes again. But this time, it tells me where the memory corruption occurs:
Program terminated with signal SIGSEGV, Segmentation fault.
\n#0 0x000014b07f01e52d in addMod (r=<error reading variable>, a=4693443247995522, b=28622907746665631,
\n\n\nI figure out the issue quickly, and not bother to understand the whole code. OpenBSD saves me again, thanks!
\n
###Native Encryption for ZFS on FreeBSD (Call for Testing)
\n\n\n\n\nTo anyone with an interest in native encryption in ZFS please test the projects/zfs-crypto-merge-0820 branch in my freebsd repo: https://github.com/mattmacy/networking.git
\n
git clone https://github.com/mattmacy/networking.git -b projects/zfs-crypto-merge-0820
\n\n\n\n\nThe UI is quite close to the Oracle Solaris ZFS crypto with minor differences for specifying key location.
\n
\nPlease note that once a feature is enabled on a pool it can’t be disabled. This means that if you enable encryption support on a pool you will never be able to import it in to a ZFS without encryption support. For this reason I would strongly advise against using this on any pool that can’t be easily replaced until this change has made its way in to HEAD after the freeze has been lifted.
\nBy way of background the original ZoL commit can be found at:
###VMworld 2018: Showcasing Hybrid Cloud, Persistent Memory and the Asigra TrueNAS Backup Appliance
\n\n\n\n\nDuring its last year in Las Vegas before moving back to San Francisco, VMworld was abuzz with all the popular buzzwords, but the key focus was on supporting a more agile approach to hybrid cloud.
\n
\nSurveys of IT stakeholders and analysts agree that most businesses have multiple clouds spanning both public cloud providers and private data centers. While the exact numbers vary, well over half of businesses have a hybrid cloud strategy consisting of at least three different clouds.
\nThis focus on hybrid cloud provided the perfect timing for our announcement that iXsystems and Asigra are partnering to deliver the Asigra TrueNAS Backup Appliance, which combines Asigra Cloud Backup software backed by TrueNAS storage. Asigra TrueNAS Backup Appliances provide a self-healing and ransomware-resistent OpenZFS backup repository in your private cloud. The appliance can simultaneously be used as general-purpose file, block, and object storage. How does this tie in with the hybrid cloud? The Asigra Cloud Backup software can backup data from public cloud repositories – G Suite, Office 365, Salesforce, etc. – as well as intelligently move backed-up data to the public cloud for long-term retention.
\nAnother major theme at the technical sessions was persistent memory, as vSphere 6.7 added support for persistent memory – either as a storage tier or virtualized and presented to a guest OS. As detailed in our blog post from SNIA’s Persistent Memory Summit 2018, persistent memory is rapidly becoming mainstream. Persistent memory bridges the gap between memory and flash storage – providing near-memory latency storage that persists across reboots or power loss. vSphere allows both legacy and persistent memory-aware applications to leverage this ultra-fast storage tier. We were excited to show off our newly-introduced TrueNAS M-Series at VMworld, as all TrueNAS M40 and M50 models leverage NVDIMM persistent memory technology to provide a super-fast write cache, or SLOG, without any of the limitations of Flash technology.
\nThe iXsystems booth’s theme was “Enterprise Storage, Open Source Economics”. iXsystems leverages the power of Open Source software, combined with our enterprise-class hardware and support, to provide incredibly low TCO storage for virtualization environments. Our TrueNAS unified storage and server offerings are an ideal solution for your organization’s private cloud infrastructure. Combined with VMware NSX Hybrid Connect – formerly known as VMware Hybrid Cloud Extension – you can seamlessly shift running systems into a public cloud environment for a true hybrid cloud solution.
\nAnother special treat at this year’s booth was iXsystems Vice President of Engineering Kris Moore giving demos of an early version of “Project TrueView”, a single-pane of glass management solution for administration of multiple FreeNAS and TrueNAS systems. In addition to simplified administration and enhanced monitoring, Project TrueView will also provide Role-Based Access Control for finer-grained permissions management. A beta version of Project TrueView is expected to be available at the end of this year.
\nOverall, we had a great week at VMworld 2018 with lots of good conversations with customers, press, analysts, and future customers about TrueNAS, the Asigra TrueNAS Backup Appliance, iXsystems servers, Project TrueView, and more – our booth was more popular than ever!
\n\n\nIn keeping with NetBSD’s policy of supporting only the latest (8.x) and next most recent (7.x) major branches, the recent release of NetBSD 8.0 marks the end of life for NetBSD 6.x. As in the past, a month of overlapping support has been provided in order to ease the migration to newer releases.
\n
As of now, the following branches are no longer maintained:
\nnetbsd-6-1
\nnetbsd-6-0
\nnetbsd-6
\nThis means:
\nThere will be no more pullups to those branches (even for security issues)
\nThere will be no security advisories made for any those branches
\nThe existing 6.x releases on ftp.NetBSD.org will be moved into /pub/NetBSD-archive/
\nMay NetBSD 8.0 serve you well! (And if it doesn’t, please submit a PR!)
\n##Beastie Bits
\n\n##Feedback/Questions
\n\nOpenBSD on Microsoft Surface Go, FreeBSD Foundation August Update, What’s taking so long with Project Trident, pkgsrc config file versioning, and MacOS remnants in ZFS code.
\n\n##Headlines
\n###OpenBSD on the Microsoft Surface Go
\n\n\nFor some reason I like small laptops and the constraints they place on me (as long as they’re still usable). I used a Dell Mini 9 for a long time back in the netbook days and was recently using an 11" MacBook Air as my primary development machine for many years. Recently Microsoft announced a smaller, cheaper version of its Surface tablets called Surface Go which piqued my interest.
\n
\n\n\nThe Surface Go is available in two hardware configurations: one with 4Gb of RAM and a 64Gb eMMC, and another with 8Gb of RAM with a 128Gb NVMe SSD. (I went with the latter.) Both ship with an Intel Pentium Gold 4415Y processor which is not very fast, but it’s certainly usable.
\n
\nThe tablet measures 9.65" across, 6.9" tall, and 0.3" thick. Its 10" diagonal 3:2 touchscreen is covered with Gorilla Glass and has a resolution of 1800x1200. The bezel is quite large, especially for such a small screen, but it makes sense on a device that is meant to be held, to avoid accidental screen touches.
\nThe keyboard and touchpad are located on a separate, removable slab called the Surface Go Signature Type Cover which is sold separately. I opted for the “cobalt blue” cover which has a soft, cloth-like alcantara material. The cover attaches magnetically along the bottom edge of the device and presents USB-attached keyboard and touchpad devices. When the cover is folded up against the screen, it sends an ACPI sleep signal and is held to the screen magnetically. During normal use, the cover can be positioned flat on a surface or slightly raised up about 3/4" near the screen for better ergonomics. When using the device as a tablet, the cover can be rotated behind the screen which causes it to automatically stop sending keyboard and touchpad events until it is rotated back around.
\nThe keyboard has a decent amount of key travel and a good layout, with Home/End/Page Up/Page Down being accessible via Fn+Left/Right/Up/Down but also dedicated Home/End/Page Up/Page Down keys on the F9-F12 keys which I find quite useful since the keyboard layout is somewhat small. By default, the F1-F12 keys do not send F1-F12 key codes and Fn must be used, either held down temporarily or Fn pressed by itself to enable Fn-lock which annoyingly keeps the bright Fn LED illuminated. The keys are backlit with three levels of adjustment, handled by the keyboard itself with the F7 key.
\nThe touchpad on the Type Cover is a Windows Precision Touchpad connected via USB HID. It has a decent click feel but when the cover is angled up instead of flat on a surface, it sounds a bit hollow and cheap.
\n\n\nThe touchscreen is powered by an Elantech chip connected via HID-over-i2c, which also supports pen input. A Surface Pen digitizer is available separately from Microsoft and comes in the same colors as the Type Covers. The pen works without any pairing necessary, though the top button on it works over Bluetooth so it requires pairing to use. Either way, the pen requires an AAAA battery inside it to operate. The Surface Pen can attach magnetically to the left side of the screen when not in use.
\n
\nA kickstand can swing out behind the display to use the tablet in a laptop form factor, which can adjust to any angle up to about 170 degrees. The kickstand stays firmly in place wherever it is positioned, which also means it requires a bit of force to pull it out when initially placing the Surface Go on a desk.
\nAlong the top of the display are a power button and physical volume rocker buttons. Along the right side are the 3.5mm headphone jack, USB-C port, power port, and microSD card slot located behind the kickstand.
\nCharging can be done via USB-C or the dedicated charge port, which accommodates a magnetically-attached, thin barrel similar to Apple’s first generation MagSafe adapter. The charging cable has a white LED that glows when connected, which is kind of annoying since it’s near the mid-line of the screen rather than down by the keyboard. Unlike Apple’s MagSafe, the indicator light does not indicate whether the battery is charged or not. The barrel charger plug can be placed up or down, but in either direction I find it puts an awkward strain on the power cable coming out of it due to the vertical position of the port.
\nWireless connectivity is provided by a Qualcomm Atheros QCA6174 802.11ac chip which also provides Bluetooth connectivity.
\nMost of the sensors on the device such as the gyroscope and ambient light sensor are connected behind an Intel Sensor Hub PCI device, which provides some power savings as the host CPU doesn’t have to poll the sensors all the time.
\n\n\nThe Surface Go’s BIOS/firmware menu can be entered by holding down the Volume Up button, then pressing and releasing the Power button, and releasing Volume Up when the menu appears. Secure Boot as well as various hardware components can be disabled in this menu. Boot order can also be adjusted. A temporary boot menu can be brought up the same way but using Volume Down instead.
\n
###FreeBSD Foundation Update, August 2018
\n\n\n\n\nDear FreeBSD Community Member,
\n
\nIt’s been a busy summer for the Foundation. From traveling around the globe spreading the word about FreeBSD to bringing on new team members to improve the Project’s Continuous Integration work, we’re very excited about what we’ve accomplished. Take a minute to check out the latest updates within our Foundation sponsored projects; read more about our advocacy efforts in Bangladesh and community building in Cambridge; don’t miss upcoming Travel Grant deadlines, and new Developer Summits; and be sure to find out how your support will ensure our progress continues into 2019.
\nWe can’t do this without you! Happy reading!! Deb
##News Roundup
\n###Project Trident: What’s taking so long?
\n\n\nThe short answer is that it’s complicated.
\n
\nProject Trident is quite literally a test of the new TrueOS build system. As expected, there have been quite a few bugs, undocumented features, and other optional bits that we discovered we needed that were not initially present. All of these things have to be addressed and retested in a constant back and forth process.
\nWhile Ken and JT are both experienced developers, neither has done this kind of release engineering before. JT has done some release engineering back in his Linux days, but the TrueOS and FreeBSD build system is very different. Both Ken and JT are learning a completely new way of building a FreeBSD/TrueOS distribution. Please keep in mind that no one has used this new TrueOS build system before, so Ken and JT want to not only provide a good Trident release, but also provide a model or template for other potential TrueOS distributions too!
\n\n\nThrough perseverance, trial and error, and a lot of head-scratching we have reached the point of having successful builds. It took a while to get there, but now we are simply working out a few bugs with the new installer that Ken wrote as well as finding and fixing all the new Xorg configuration options which recently landed in FreeBSD. We also found that a number of services have been removed or replaced between TrueOS 18.03 and 18.06 so we are needing to adjust what we consider the “base” services for the desktop. All of these issues are being resolved and we are continually rebuilding and pulling in new patches from TrueOS as soon as they are committed.
\n
\nIn the meantime we have made an early BETA release of Trident available to the users in our Telegram Channel for those who want to help out in testing these early versions.
\n\n\nAt the moment we are doing many iterations of testing and tweaking the install ISO and package configurations in order to ensure that all the critical functionality works out-of-box (networking, sound, video, basic apps, etc). While we do not foresee any other major delays, sometimes things happen that our outside of our control. For an example, one of the recent delays that hit recently was completely unexpected: we had a hard drive failure on our build server. Up until recently, The aptly named “Poseidon” build server was running a Micron m500dc drive, but that drive is now constantly reporting errors. Despite ordering a replacement Western Digital Blue SSD several weeks ago, we just received it this past week. The drive is now installed with the builder back to full functionality, but we did lose many precious days with the delay.
\n
\nThe build server for Project Trident is very similar to the one that JT donated to the TrueOS project. JT had another DL580 G7, so he donated one to the Trident Project for their build server. Poseidon also has 256GB RAM (64 x 4GB sticks) which is a smidge higher than what the TrueOS builder has.
\nSince we are talking about hardware, we probably should address another question we get often, “What Hardware are the devs testing on?” So let’s go ahead and answer that one now.
Developer Hardware
\nJT: His main test box is a custom-built Intel i7 7700K system running 32GB RAM, dual Intel Optane 900P drives, and an Nvidia 1070 GTX with four 4K Acer Monitors. He also uses a Lenovo x250 ThinkPad alongside a desk full of x230t and x220 ThinkPads. One of which he gave away at SouthEast LinuxFest this year, which you can read about here. However it’s not done there, being a complete hardware hoarder, JT also tests on several Intel NUCs and his second laptop a Fujitsu t904, not to mention a Plethora of HP DL580 servers, a DL980 server, and a stack of BL485c, BL460c, and BL490c Blades in his HP c7000 and c3000 Bladecenter chassis. (Maybe it’s time for an intervention for his hardware collecting habits)
\nKen: For a laptop, he primarily uses a 3rd generation X1 Carbon, but also has an old Eee PC T101MT Netbook (dual core 1GHz, 2GB of memory) which he uses for verifying how well Trident works on low-end hardware. As far as workstations go, his office computer is an Intel i7 with an NVIDIA Geforce GTX 960 running three 4K monitors and he has a couple other custom-built workstations (1 AMD, 1 Intel+NVIDIA) at his home. Generally he assembled random workstations based on hardware that was given to him or that he could acquire cheap.
\nTim: is using a third gen X1 Carbon and a custom built desktop with an Intel Core i5-4440 CPU, 16 GiB RAM, Nvidia GeForce GTX 750 Ti, and a RealTek 8168 / 8111 network card.
\nRod: Rod uses… No one knows what Rod uses, It’s kinda like how many licks does it take to get to the center of a Tootsie-Roll Tootsie-Pop… the world may just never know.
\n###NetBSD GSoC: pkgsrc config file versioning
\n\n\n\n\n\n\nPackages may install code (both machine executable code and interpreted programs), documentation and manual pages, source headers, shared libraries and other resources such as graphic elements, sounds, fonts, document templates, translations and configuration files, or a combination of them.
\n
\nConfiguration files are usually the means through which the behaviour of software without a user interface is specified. This covers parts of the operating systems, network daemons and programs in general that don’t come with an interactive graphical or textual interface as the principal mean for setting options.
\nSystem wide configuration for operating system software tends to be kept under /etc, while configuration for software installed via pkgsrc ends up under LOCALBASE/etc (e.g., /usr/pkg/etc).
\nSoftware packaged as part of pkgsrc provides example configuration files, if any, which usually get extracted to LOCALBASE/share/examples/PKGBASE/.
\nDon’t worry: automatic merging is disabled by default, set $VCSAUTOMERGE to enable it.
\nIn order to avoid breakage, installed configuration is backed up first in the VCS, separating user-modified files from files that have been already automatically merged in the past, in order to allow the administrator to easily restore the last manually edited file in case of breakage.
\nVCS functionality only applies to configuration files, not to rc.d scripts, and only if the environment variable $NOVCS is unset.
\nThe version control system to be used as a backend can be set through $VCS. It default to RCS, the Revision Control System, which works only locally and doesn’t support atomic transactions.
\nOther backends such as CVS are supported and more will come; these, being used at the explicit request of the administrator, need to be already installed and placed in a directory part of $PATH.
\n\n\npkgsrc is now able to deploy configuration from packages being installed from a remote, site-specific vcs repository.
\n
\nUser modified files are always tracked even if automerge functionality is not enabled, and a new tool, pkgconftrack(1), exists to manually store user changes made outside of package upgrade time.
\nVersion Control software is executed as the same user running pkg_add or make install, unless the user is “root”. In this case, a separate, unprivileged user, pkgvcsconf, gets created with its own home directory and a working login shell (but no password). The home directory is not strictly necessary, it exists to facilitate migrations betweens repositories and vcs changes; it also serves to store keys used to access remote repositories.
\nUsing git instead of rcs is simply done by setting VCS=git in pkg_install.conf
\n\n\nSupport for configuration tracking is in scripts, pkginstall scripts, that get built into binary packages and are run by pkg_add upon installation. The idea behind the proposal suggested that users of the new feature should be able to store revisions of their installed configuration files, and of package-provided default, both in local or remote repositories. With this capability in place, it doesn’t take much to make the scripts “pull” configuration from a VCS repository at installation time.
\n
\nThat’s what setting VCSCONFPULL=yes in pkg_install.conf after having enabled VCSTRACK_CONF does: You are free to use official, third party prebuilt packages that have no customization in them, enable these options, and point pkgsrc to a private conf repository. If it contains custom configuration for the software you are installing, an attempt will be made to use it and install it on your system. If it fails, pkginstall will fall back to using the defaults that come inside the package. RC scripts are always deployed from the binary package, if existing and PKG_RCD_SCRIPTS=yes in pkg_install.conf or the environment.
\nThis will be part of packages, not a separate solution like configuration management tools. It doesn’t support running scripts on the target system to customize the installation, it doesn’t come with its domain-specific language, it won’t run as a daemon or require remote logins to work. It’s quite limited in scope, but you can define a ROLE for your system in pkg_install.conf or in the environment, and pkgsrc will look for configuration you or your organization crafted for such a role (e.g., public, standalone webserver vs reverse proxy or node in a database cluster)
###A little bit of the one-time MacOS version still lingers in ZFS
\n\n\n\n\nOnce upon a time, Apple came very close to releasing ZFS as part of MacOS. Apple did this work in its own copy of the ZFS source base (as far as I know), but the people in Sun knew about it and it turns out that even today there is one little lingering sign of this hoped-for and perhaps prepared-for ZFS port in the ZFS source code. Well, sort of, because it’s not quite in code.
\n
\nLurking in the function that reads ZFS directories to turn (ZFS) directory entries into the filesystem independent format that the kernel wants is the following comment:
objnum = ZFS_DIRENT_OBJ(zap.za_first_integer);
\n/*
\n* MacOS X can extract the object type here such as:
\n* uint8_t type = ZFS_DIRENT_TYPE(zap.za_first_integer);
\n*/
\n\n\nZFS maintains file type information in directories. This information can’t be used on Solaris (and thus Illumos), where the overall kernel doesn’t have this in its filesystem independent directory entry format, but it could have been on MacOS (‘Darwin’), because MacOS is among the Unixes that support d_type. The comment itself dates all the way back to this 2007 commit, which includes the change ‘reserve bits in directory entry for file type’, which created the whole setup for this.
\n
\nI don’t know if this file type support was added specifically to help out Apple’s MacOS X port of ZFS, but it’s certainly possible, and in 2007 it seems likely that this port was at least on the minds of ZFS developers. It’s interesting but understandable that FreeBSD didn’t seem to have influenced them in the same way, at least as far as comments in the source code go; this file type support is equally useful for FreeBSD, and the FreeBSD ZFS port dates to 2007 too (per this announcement).
\nRegardless of the exact reason that ZFS picked up maintaining file type information in directory entries, it’s quite useful for people on both FreeBSD and Linux that it does so. File type information is useful for any number of things and ZFS filesystems can (and do) provide this information on those Unixes, which helps make ZFS feel like a truly first class filesystem, one that supports all of the expected general system features.
##Beastie Bits
\n\n##Feedback/Questions
\n\nInsight into TrueOS and Trident, stop evildoers with pf-badhost, Flashback to FreeBSDcon ‘99, OpenBSD’s measures against TLBleed, play Morrowind on OpenBSD in 5 steps, DragonflyBSD developers shocked at Threadripper performance, and more.
\n\n##Headlines
\n###An Insight into the Future of TrueOS BSD and Project Trident
\n\n\nLast month, TrueOS announced that they would be spinning off their desktop offering. The team behind the new project, named Project Trident, have been working furiously towards their first release. They did take a few minutes to answer some of our question about Project Trident and TrueOS. I would like to thank JT and Ken for taking the time to compile these answers.
\n
\n\n\nProject Trident: Project Trident is the continuation of the TrueOS Desktop. Essentially, it is the continuation of the primary “TrueOS software” that people have been using for the past 2 years. The continuing evolution of the entire TrueOS project has reached a stage where it became necessary to reorganize the project. To understand this change, it is important to know the history of the TrueOS project.
\n
\n\n\nOriginally, Kris Moore created PC-BSD. This was a Desktop release of FreeBSD focused on providing a simple and user-friendly graphical experience for FreeBSD. PC-BSD grew and matured over many years. During the evolution of PC-BSD, many users began asking for a server focused version of the software. Kris agreed, and TrueOS was born as a scaled down server version of PC-BSD. In late 2016, more contributors and growth resulted in significant changes to the PC-BSD codebase. Because the new development was so markedly different from the original PC-BSD design, it was decided to rebrand the project.
\n
\n\n\nTrueOS was chosen as the name for this new direction for PC-BSD as the project had grown beyond providing only a graphical front to FreeBSD and was beginning to make fundamental changes to the FreeBSD operating system. One of these changes was moving PC-BSD from being based on each FreeBSD Release to TrueOS being based on the active and less outdated FreeBSD Current. Other major changes are using OpenRC for service management and being more aggressive about addressing long-standing issues with the FreeBSD release process. TrueOS moved toward a rolling release cycle, twice a year, which tested and merged FreeBSD changes directly from the developer instead of waiting months or even years for the FreeBSD review process to finish. TrueOS also deprecated and removed obsolete technology much more regularly.
\n
\n\n\nAs the TrueOS Project grew, the developers found these changes were needed by other FreeBSD-based projects. These projects began expressing interest in using TrueOS rather than FreeBSD as the base for their project. This demonstrated that TrueOS needed to again evolve into a distribution framework for any BSD project to use. This allows port maintainers and source developers from any BSD project to pool their resources and use the same source repositories while allowing every distribution to still customize, build, and release their own self-contained project. The result is a natural split of the traditional TrueOS team. There were now naturally two teams in the TrueOS project: those working on the build infrastructure and FreeBSD enhancements – the “core” part of the project, and those working on end-user experience and utility – the “desktop” part of the project.
\n
\n\n\nWhen the decision was made to formally split the projects, the obvious question that arose was what to call the “Desktop” project. As TrueOS was already positioned to be a BSD distribution platform, the developers agreed the desktop side should pick a new name. There were other considerations too, one notable being that we were concerned that if we continued to call the desktop project “TrueOS Desktop”, it would prevent people from considering TrueOS as the basis for their distribution because of misconceptions that TrueOS was a desktop-focused OS. It also helps to “level the playing field” for other desktop distributions like GhostBSD so that TrueOS is not viewed as having a single “blessed” desktop version.
\n
\n\n\nProject Trident: TrueOS has already added a number of features to FreeBSD:
\n
\nOpenRC replaces rc.d for service management
\nLibreSSL in base
\nRoot NSS certificates out-of-box
\nScriptable installations (pc-sysinstall)
\nThe full list of changes can be seen on the TrueOS repository (https://github.com/trueos/trueos/blob/trueos-master/README.md). This list does change quite regularly as FreeBSD development itself changes.
\n\n\nProject Trident: Historically, one of the biggest hurdles for creating a desktop version of FreeBSD is that the build options for packages are tuned for servers rather than desktops. This means a desktop distribution cannot use the pre-built packages from FreeBSD and must build, use, and maintain a custom package repository. Maintaining a fork of the FreeBSD ports tree is no trivial task. TrueOS has created a full distribution framework so now all it takes to create a custom build of FreeBSD is a single JSON manifest file. There is now a single “source of truth” for the source and ports repositories that is maintained by the TrueOS team and regularly tagged with “stable” build markers. All projects can use this framework, which makes updates trivial.
\n
\n\n\nProject Trident: That is the hope. Historically, creating a desktop-centered BSD has required a lot of specialized knowledge. Not only do most people not have this knowledge, but many do not even know what they need to learn until they start troubleshooting. TrueOS is trying to drastically simplify this process to enable the wider Open Source community to experiment, contribute, and enjoy BSD-based projects.
\n
\n\n\nProject Trident: Project Trident will be dependent on TrueOS for ARM support. The developers have talked about the possibility of supporting ARM64 and RISC-V architectures, but it is not possible at the current time. If more Open Source contributors want to help develop ARM and RISC-V support, the TrueOS project is definitely willing to help test and integrate that code.
\n
\n\n\nProject Trident: Long-term, almost nothing. Lumina is still the desktop environment for Project Trident and will continue to be developed and enhanced alongside Project Trident just as it was for TrueOS. Short-term, we will be delaying the release of Lumina 2.0 and will release an updated version of the 1.x branch (1.5.0) instead. This is simply due to all the extra overhead to get Project Trident up and running. When things settle down into a rhythm, the development of Lumina will pick up once again.
\n
\n\n\nProject Trident: While Lumina is included by default, all of the other popular desktop environments will be available in the package repo exactly as they had been before.
\n
\n\n\nProject Trident: Steam is still unavailable natively on FreeBSD, so we do not have any plans to ship it out of the box currently. In the meantime, we highly recommend installing the Windows version of Steam through the PlayOnBSD utility.
\n
\n\n\nProject Trident: The AppCafe is the name of the graphical interface for the “pkg” utility integrated into the SysAdm client created by TrueOS. This hasn’t changed. SysAdm, the graphical client, and by extension AppCafe are still available for all TrueOS-based distributions to use.
\n
\n\n\nProject Trident: iXsystems is the first corporate sponsor of Project Trident and we are always open to other sponsorships as well. We would prefer smaller individual contributions from the community, but we understand that larger project needs or special-purpose goals are much more difficult to achieve without allowing larger corporate sponsorships as well. In either case, Project Trident is always looking out for the best interests of the community and will not allow intrusive or harmful code to enter the project even if a company or individual tries to make that code part of a sponsorship deal.
\n
\n\n\nProject Trident: Yes! That was a primary reason for TrueOS to start tracking the CURRENT branch of FreeBSD in 2016. This allows for the changes that FreeBSD developers are making, including new hardware support, to be available much sooner than if we followed the FreeBSD release cycle.
\n
\n\n\nProject Trident: Right now we are targeting a late August release date. This is because Project Trident is “kicking the wheels” on the new TrueOS distribution system. We want to ensure everything is working smoothly before we release. Going forward, we plan on having regular package updates every week or two for the end-user packages and a new release of Trident with an updated OS version every 6 months. This will follow the TrueOS release schedule with a small time offset.
\n
###pf-badhost: Stop the evil doers in their tracks!
\n\n\n\n\npf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet’s biggest irritants. Annoyances such as ssh bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.
\n
\nFiltering performance is exceptional, as the badhost list is stored in a pf table. To quote the OpenBSD FAQ page regarding tables: “the lookup time on a table holding 50,000 addresses is only slightly more than for one holding 50 addresses.”
\npf-badhost is simple and powerful. The blocklists are pulled from quality, trusted sources. The ‘Firehol’, ‘Emerging Threats’ and ‘Binary Defense’ block lists are used as they are popular, regularly updated lists of the internet’s most egregious offenders. The pf-badhost.sh script can easily be expanded to use additional or alternate blocklists.
\npf-badhost works best when used in conjunction with unbound-adblock for the ultimate badhost blocking.
DigitalOcean
\nhttps://do.co/bsdnow
###FLASHBACK: FreeBSDCon’99: Fans of Linux’s lesser-known sibling gather for the first time
\n\n\n\n\nFreeBSD, a port of BSD Unix to Intel, has been around almost as long as Linux has – but without the media hype. Its developer and user community recently got a chance to get together for the first time, and they did it in the city where BSD – the Berkeley Software Distribution – was born some 25 years ago.
\n
\nOctober 17, 1999 marked a milestone in the history of FreeBSD – the first FreeBSD conference was held in the city where it all began, Berkeley, CA. Over 300 developers, users, and interested parties attended from around the globe.
\nThis was easily 50 percent more people than the conference organizers had expected. This first conference was meant to be a gathering mostly for developers and FreeBSD advocates. The turnout was surprisingly (and gratifyingly) large.
\nIn fact, attendance exceeded expectations so much that, for instance, Kirk McKusick had to add a second, identical tutorial on FreeBSD internals, because it was impossible for everyone to attend the first!
\nBut for a first-ever conference, I was impressed by how smoothly everything seemed to go. Sessions started on time, and the sessions I attended were well-run; nothing seemed to be too cold, dark, loud, late, or off-center.
\nOf course, the best part about a conference such as this one is the opportunity to meet with other people who share similar interests. Lunches and breaks were a good time to meet people, as was the Tuesday night beer bash.
\nThe Wednesday night reception was of a type unusual for the technical conferences I usually attend – a three-hour Hornblower dinner cruise on San Francisco Bay. Not only did we all enjoy excellent food and company, but we all got to go up on deck and watch the lights of San Francisco and Berkeley as we drifted by. Although it’s nice when a conference attracts thousands of attendees, there are some things that can only be done with smaller groups of people; this was one of them.
\nIn short, this was a tiny conference, but a well-run one.
\n\n\nAlthough it was a relatively small conference, the number and quality of the sessions belied the size. Each of the three days of the conference featured a different keynote speaker. In addition to Jordan Hubbard, Jeremy Allison spoke on “Samba Futures” on day two, and Brian Behlendorf gave a talk on “FreeBSD and Apache: A Perfect Combo” to start off the third day.
\n
\nThe conference sessions themselves were divided into six tracks: advocacy, business, development, networking, security, and panels. The panels track featured three different panels, made up of three different slices of the community: the FreeBSD core team, a press panel, and a prominent user panel with representatives from such prominent commercial users as Yahoo! and USWest.
\nI was especially interested in Apple Computer’s talk in the development track. Wilfredo Sanchez, technical lead for open source projects at Apple (no, that’s not an oxymoron!) spoke about Apple’s Darwin project, the company’s operating system road map, and the role of BSD (and, specifically, FreeBSD) in Apple’s plans.
\nApple and Unix have had a long and uneasy history, from the Lisa through the A/UX project to today. Personally, I’m very optimistic about the chances for the Darwin project to succeed. Apple’s core OS kernel team has chosen FreeBSD as its reference platform. I’m looking forward to what this partnership will bring to both sides.
\nOther development track sessions included in-depth tutorials on writing device drivers, basics of the Vinum Volume Manager, Fibre Channel, development models (the open repository model), and the FreeBSD Documentation Project (FDP). If you’re interested in contributing to the FreeBSD project, the FDP is a good place to start.
\nAdvocacy sessions included “How One Person Can Make a Difference” (a timeless topic that would find a home at any technical conference!) and “Starting and Managing A User Group” (trials and tribulations as well as rewards).
\nThe business track featured speakers from three commercial users of FreeBSD: Cybernet, USWest, and Applix. Applix presented its port of Applixware Office for FreeBSD and explained how Applix has taken the core services of Applixware into open source.
\nCommercial applications and open source were once a rare combination; we can only hope the trend away from that state of affairs will continue.
\n\n\nThe use of FreeBSD in embedded applications is increasing as well – and it is increasing at the same rate that hardware power is. These days, even inexpensive systems are able to run a BSD kernel.
\n
\nThe BSD license and the solid TCP/IP stack prove significant enticements to this market as well. (Unlike the GNU Public License, the BSD license does not require that vendors make derivative works open source.)
\nCompanies such as USWest and Verio use FreeBSD for a wide variety of different Internet services.
\nYahoo! and Hotmail are examples of companies that use FreeBSD extensively for more specific purposes. Yahoo!, for example, has many hundreds of FreeBSD boxes, and Hotmail has almost 2000 FreeBSD machines at its data center in the San Francisco Bay area.
\nHotmail is owned by Microsoft, so the fact that it runs FreeBSD is a secret. Don’t tell anyone…
\nWhen asked to comment on the increasing commercial interest in BSD, Hubbard said that FreeBSD is learning the Red Hat lesson. “Walnut Creek and others with business interests in FreeBSD have learned a few things from the Red Hat IPO,” he said, “and nobody is just sitting around now, content with business as usual. It’s clearly business as unusual in the open source world today.”
\nHubbard had also singled out some of BSD’s commercial partners, such as Whistle Communications, for praise in his opening day keynote. These partners play a key role in moving the project forward, he said, by contributing various enhancements and major new systems, such as Netgraph, as well as by contributing paid employee time spent on FreeBSD.
\nEven short FreeBSD-related contacts can yield good results, Hubbard said. An example of this is the new jail() security code introduced in FreeBSD 3.x and 4.0, which was contributed by R & D Associates. A number of ISPs are also now donating the hardware and bandwidth that allows the project to provide more resource mirrors and experimental development sites.
\n\n\nAnd speaking of corporate sponsors, thanks go to Walnut Creek for sponsoring the conference, and to Yahoo! for covering all the expenses involved in bringing the entire FreeBSD core team to Berkeley.
\n
\nAs a fan of FreeBSD, I’m happy to see that the project has finally produced a conference. It was time: many of the 16 core team members had been working together on a regular basis for nearly seven years without actually meeting face to face.
\nIt’s been an interesting year for open source projects. I’m looking forward to the next year – and the next BSD conference – to be even better.
##News Roundup
\n###OpenBSD Recommends: Disable SMT/Hyperthreading in all Intel BIOSes
Two recently disclosed hardware bugs affected Intel cpus:\n\n - TLBleed\n\n - T1TF (the name "Foreshadow" refers to 1 of 3 aspects of this\n bug, more aspects are surely on the way)\n\nSolving these bugs requires new cpu microcode, a coding workaround,\n*AND* the disabling of SMT / Hyperthreading.\n\nSMT is fundamentally broken because it shares resources between the two\ncpu instances and those shared resources lack security differentiators.\nSome of these side channel attacks aren't trivial, but we can expect\nmost of them to eventually work and leak kernel or cross-VM memory in\ncommon usage circumstances, even such as javascript directly in a\nbrowser.\n\nThere will be more hardware bugs and artifacts disclosed. Due to the\nway SMT interacts with speculative execution on Intel cpus, I expect SMT\nto exacerbate most of the future problems.\n\nA few months back, I urged people to disable hyperthreading on all\nIntel cpus. I need to repeat that:\n\n DISABLE HYPERTHREADING ON ALL YOUR INTEL MACHINES IN THE BIOS.\n\nAlso, update your BIOS firmware, if you can.\n\nOpenBSD -current (and therefore 6.4) will not use hyperthreading if it\nis enabled, and will update the cpu microcode if possible.\n\nBut what about 6.2 and 6.3?\n\nThe situation is very complex, continually evolving, and is taking too\nmuch manpower away from other tasks. Furthermore, Intel isn't telling\nus what is coming next, and are doing a terrible job by not publically\ndocumenting what operating systems must do to resolve the problems. We\nare having to do research by reading other operating systems. There is\nno time left to backport the changes -- we will not be issuing a\ncomplete set of errata and syspatches against 6.2 and 6.3 because it is\nturning into a distraction.\n\nRather than working on every required patch for 6.2/6.3, we will\nre-focus manpower and make sure 6.4 contains the best solutions\npossible.\n\nSo please try take responsibility for your own machines: Disable SMT in\nthe BIOS menu, and upgrade your BIOS if you can.\n\nI'm going to spend my money at a more trustworthy vendor in the future.\n
\n\n###Get Morrowind running on OpenBSD in 5 simple steps
\n\n\n\n\nThis article contains brief instructions on how to get one of the greatest Western RPGs of all time, The Elder Scrolls III: Morrowind, running on OpenBSD using the OpenMW open source engine recreation. These instructions were tested on a ThinkPad X1 Carbon Gen 3. The information was adapted from this OpenMW forum thread: https://forum.openmw.org/viewtopic.php?t=3510
\n
pkg_add openmw innoextract
innoextract setup_tes_morrowind_goty_2.0.0.7.exe
iXsystems
\nhttps://twitter.com/allanjude/status/1034647571124367360
\n\n\nPart of the role of being a packager is compiling lots (and lots) of packages. That means compiling lots of code from interesting places and in a variety of styles. In my opinion, being a good packager also means providing feedback to upstream when things are bad. That means filing upstream bugs when possible, and upstreaming patches.
\n
\nOne of the “exciting” moments in packaging is when tools change. So each and every major CMake update is an exercise in recompiling 2400 or more packages and adjusting bits and pieces. When a software project was last released in 2013, adjusting it to modern tools can become quite a chore (e.g. Squid Report Generator). CMake is excellent for maintaining backwards compatibility, generally accommodating old software with new policies. The most recent 3.12 release candidate had three issues filed from the FreeBSD side, all from fallout with older software. I consider the hours put into good bug reports, part of being a good citizen of the Free Software world.
\nMy most interesting bug this week, though, came from one line of code somewhere in Kleopatra: Q_UNUSED(gpgagent_data);
\nThat one line triggered a really peculiar link error in KDE’s FreeBSD CI system. Yup … telling the compiler something is unused made it fall over. Commenting out that line got rid of the link error, but introduced a warning about an unused function. Working with KDE-PIM’s Volker Krause, we whittled the problem down to a six-line example program — two lines if you don’t care much for coding style. I’m glad, at that point, that I could throw it over the hedge to the LLVM team with some explanatory text. Watching the process on their side reminds me ever-so-strongly of how things work in KDE (or FreeBSD for that matter): Bugzilla, Phabricator, and git combine to be an effective workflow for developers (perhaps less so for end-users).
\nToday I got a note saying that the issue had been resolved. So brief a time for a bug. Live fast. Get squashed young.
###DragonFlyBSD Now Runs On The Threadripper 2990WX, Developer Shocked At Performance
\n\n\n\n\nLast week I carried out some tests of BSD vs. Linux on the new 32-core / 64-thread Threadripper 2990WX. I tested FreeBSD 11, FreeBSD 12, and TrueOS – those benchmarks will be published in the next few days. I tried DragonFlyBSD, but at the time it wouldn’t boot with this AMD HEDT processor. But now the latest DragonFlyBSD development kernel can handle the 2990WX and the lead DragonFly developer calls this new processor “a real beast” and is stunned by its performance potential.
\n
\n\n\nWhen I tried last week, the DragonFlyBSD 5.2.2 stable release nor DragonFlyBSD 5.3 daily snapshot would boot on the 2990WX. But it turns out Matthew Dillon, the lead developer of DragonFlyBSD, picked up a rig and has it running now. So in time for the next 5.4 stable release or those using the daily snapshots can have this 32-core / 64-thread Zen+ CPU running on this operating system long ago forked from FreeBSD.
\n
\n\n\nIn announcing his success in bringing up the 2990WX under DragonFlyBSD, which required a few minor changes, he shared his performance thoughts and hopes for the rig. “The cpu is a real beast, packing 32 cores and 64 threads. It blows away our dual-core Xeon to the tune of being +50% faster in concurrent compile tests, and it also blows away our older 4-socket Opteron (which we call ‘Monster’) by about the same margin. It’s an impressive CPU. For now the new beast is going to be used to help us improve I/O performance through the filesystem, further SMP work (but DFly scales pretty well to 64 threads already), and perhaps some driver to work to support the 10gbe on the mobo.”
\n
\n\n\nDillon shared some results on the system as well. " The Threadripper 2990WX is a beast. It is at least 50% faster than both the quad socket opteron and the dual socket Xeon system I tested against. The primary limitation for the 2990WX is likely its 4 channels of DDR4 memory, and like all Zen and Zen+ CPUs, memory performance matters more than CPU frequency (and costs almost no power to pump up the performance). That said, it still blow away a dual-socket Xeon with 3x the number of memory channels. That is impressive!"
\n
\n\n\nThe well known BSD developer also added, “This puts the 2990WX at par efficiency vs a dual-socket Xeon system, and better than the dual-socket Xeon with slower memory and a power cap. This is VERY impressive. I should note that the 2990WX is more specialized with its asymetric NUMA architecture and 32 cores. I think the sweet spot in terms of CPU pricing and efficiency is likely going to be with the 2950X (16-cores/32-threads). It is clear that the 2990WX (32-cores/64-threads) will max out 4-channel memory bandwidth for many workloads, making it a more specialized part. But still awesome…This thing is an incredible beast, I’m glad I got it.”
\n
\n\n\nWhile I have the FreeBSD vs. Linux benchmarks from a few days ago, it looks like now on my ever growing TODO list will be re-trying out the newest DragonFlyBSD daily snapshot for seeing how the performance compares in the mix. Stay tuned for the numbers that should be in the next day or two.
\n
##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nTrip reports from the Essen Hackathon and BSDCam, CfT: ZFS native encryption and UFS trim consolidation, ZFS performance benchmarks on a FreeBSD server, how to port your OS to EC2, Vint Cerf about traceability, Remote Access console to an RPi3 running FreeBSD, and more.
\n\n##Headlines
\n###Essen Hackathon & BSDCam 2018 trip report
###Call for Testing: ZFS Native Encryption for FreeBSD
\n\niXsystems
\n\n###Call for Testing: UFS TRIM Consolidation
\n\n\n\n\nWhen deleting files on filesystems that are stored on flash-memory (solid-state) disk drives, the filesystem notifies the underlying disk of the blocks that it is no longer using. The notification allows the drive to avoid saving these blocks when it needs to flash (zero out) one of its flash pages. These notifications of no-longer-being-used blocks are referred to as TRIM notifications. In FreeBSD these TRIM notifications are sent from the filesystem to the drive using the BIO_DELETE command.
\n
\nUntil now, the filesystem would send a separate message to the drive for each block of the file that was deleted. Each Gigabyte of file size resulted in over 3000 TRIM messages being sent to the drive. This burst of messages can overwhelm the drive’s task queue causing multiple second delays for read and write requests.
\nThis implementation collects runs of contiguous blocks in the file and then consolodates them into a single BIO_DELETE command to the drive. The BIO_DELETE command describes the run of blocks as a single large block being deleted. Each Gigabyte of file size can result in as few as two BIO_DELETE commands and is typically less than ten. Though these larger BIO_DELETE commands take longer to run, they do not clog the drive task queue, so read and write commands can intersperse effectively with them.
\nThough this new feature has been throughly reviewed and tested, it is being added disabled by default so as to minimize the possibility of disrupting the upcoming 12.0 release. It can be enabled by running ``sysctl vfs.ffs.dotrimcons=1’’. Users are encouraged to test it. If no problems arise, we will consider requesting that it be enabled by default for 12.0.
\nThis support is off by default, but I am hoping that I can get enough testing to ensure that it (a) works, and (b) is helpful that it will be reasonable to have it turned on by default in 12.0. The cutoff for turning it on by default in 12.0 is September 19th. So I am requesting your testing feedback in the near-term. Please let me know if you have managed to use it successfully (or not) and also if it provided any performance difference (good or bad).
gstat
with the -d flag##News Roundup
\n###ZFS performance
\n\n\nThis is NOT an all-in post about ZFS performance. I built a FreeBSD+ZFS file server recently at work to serve as an offsite backup server. I wanted to run a few synthetic workloads on it and look at how it fares from performance perspective. Mostly for curiosity and learning purposes.
\n
\nAs stated in the notes about building this server, performance was not one of the priorities, as this server will never face our active workload. What I care about from this server is its ability to work with rsync and keep the data synchronised with our primary storage server. With that context, I ran a few write tests to see how good our solution is and what to expect from it in terms of performance.
\n\n\nWrite Performance: Incompressible: 1600-2600 MB/s, Compressible: 2500-6600 MB/s
\n
\nAnother over 1200 MB/s is enough to keep your 10 gigabit network saturated
\n\n\nI’ve been the maintainer of the FreeBSD/EC2 platform for about 7.5 years now, and as far as “running things in virtual machines” goes, that remains the only operating system and the only cloud which I work on. That said, from time to time I get questions from people who want to port other operating systems into EC2, and being a member of the open source community, I do my best to help them. I realized a few days ago that rather than replying to emails one by one it would be more efficient to post something publicly; so — for the benefit of the dozen or so people who want to port operating systems to run in EC2, and the curiosity of maybe a thousand more people who use EC2 but will never build AMIs themselves — here’s a rough guide to building EC2 images.
\n
\nBefore we can talk about building images, there are some things you need:
\nYour OS needs to run on x86 hardware. 64-bit (“amd64”, “x86-64”) is ideal, but I’ve managed to run 32-bit FreeBSD on “64-bit” EC2 instances so at least in some cases that’s not strictly necessary.
\nYou almost certainly want to have drivers for Xen block devices (for all of the pre-Nitro EC2 instances) or for NVMe disks (for the most recent EC2 instances). Theoretically you could make do without these since there’s some ATA emulation available for bootstrapping, but if you want to do any disk I/O after the kernel finishes booting you’ll want to have a disk driver.
\nSimilarly, you need support for the Xen network interface (older instances), Intel 10 GbE SR-IOV networking (some newer but pre-Nitro instances), or Amazon’s “ENA” network adapters (on Nitro instances), unless you plan on having instances which don’t communicate over the network. The ENA driver is probably the hardest thing to port, since as far as I know there’s no way to get your hands on the hardware directly, and it’s very difficult to do any debugging in EC2 without having a working network.
\nFinally, the obvious: You need to have an AWS account, and appropriate API access keys.
\nBuilding a disk imageBuilding an AMI
\n
\nI wrote a simple tool for converting disk images into EC2 instances: bsdec2-image-upload. It uploads a disk image to Amazon S3; makes an API call to import that disk image into an EBS volume; creates a snapshot of that volume; then registers an EC2 AMI using that snapshot.
\nTo use bsdec2-image-upload, you’ll first need to create an S3 bucket for it to use as a staging area. You can call it anything you like, but I recommend that you
\n\n\nCreate it in a “nearby” region (for performance reasons), and
\n
\nSet an S3 “lifecycle policy” which deletes objects automatically after 1 day (since bsdec2-image-upload doesn’t clean up the S3 bucket, and those objects are useless once you’ve finished creating an AMI).
\n\n\nBoot configuration
\n
\nOdds are that your instance started booting and got as far as the boot loader launching the kernel, but at some point after that things went sideways. Now we start the iterative process of building disk images, turning them into AMIs, launching said AMIs, and seeing where they break. Some things you’ll probably run into here:
\nEC2 instances have two types of console available to them: A serial console and an VGA console. (Or rather, emulated serial and emulated VGA.) If you can have your kernel output go to both consoles, I recommend doing that. If you have to pick one, the serial console (which shows up as the “System Log” in EC2) is probably more useful than the VGA console (which shows up as “instance screenshot”) since it lets you see more than one screen of logs at once; but there’s a catch: Due to some bizarre breakage in EC2 — which I’ve been complaining about for ten years — the serial console is very “laggy”. If you find that you’re not getting any output, wait five minutes and try again.
\nYou may need to tell your kernel where to find the root filesystem. On FreeBSD we build our disk images using GPT labels, so we simply need to specify in /etc/fstab that the root filesystem is on /dev/gpt/rootfs; but if you can’t do this, you’ll probably need to have different AMIs for Nitro instances vs. non-Nitro instances since Xen block devices will typically show up with different device names from NVMe disks. On FreeBSD, I also needed to set the vfs.root.mountfrom kernel environment variable for a while; this also is no longer needed on FreeBSD but something similar may be needed on other systems.
\nYou’ll need to enable networking, using DHCP. On FreeBSD, this means placing ifconfig_DEFAULT=“SYNCDHCP” into /etc/rc.conf; other systems will have other ways of specifying network parameters, and it may be necessary to specify a setting for the Xen network device, Intel SR-IOV network, and the Amazon ENA interface so that you’ll have the necessary configuration across all EC2 instance types. (On FreeBSD, ifconfig_DEFAULT takes care of specifying the network settings which should apply for whatever network interface the kernel finds at boot time.)
\nYou’ll almost certainly want to turn on SSH, so that you can connect into newly launched instances and make use of them. Don’t worry about setting a password or creating a user to SSH into yet — we’ll take care of that later.
\nEC2 configuration
\nNow it’s time to make the AMI behave like an EC2 instance. To this end, I prepared a set of rc.d scripts for FreeBSD. Most importantly, they
\nPrint the SSH host keys to the console, so that you can veriy that they are correct when you first SSH in. (Remember, Verifying SSH host keys is more important than flossing every day.)
\nDownload the SSH public key you want to use for logging in, and create an account (by default, “ec2-user”) with that key set up for you.
\nFetch EC2 user-data and process it via configinit to allow you to configure the system as part of the process of launching it.
\nIf your OS has an rc system derived from NetBSD’s rc.d, you may be able to use these scripts without any changes by simply installing them and enabling them in /etc/rc.conf; otherwise you may need to write your own scripts using mine as a model.
\nFirstboot scripts
\nA feature I added to FreeBSD a few years ago is the concept of “firstboot” scripts: These startup scripts are only run the first time a system boots. The aforementioned configinit and SSH key fetching scripts are flagged this way — so if your OS doesn’t support the “firstboot” keyword on rc.d scripts you’ll need to hack around that — but EC2 instances also ship with other scripts set to run on the first boot:
\nFreeBSD Update will fetch and install security and critical errata updates, and then reboot the system if necessary.
\nThe UFS filesystem on the “boot disk” will be automatically expanded to the full size of the disk — this makes it possible to specify a larger size of disk at EC2 instance launch time.
\nThird-party packages will be automatically fetched and installed, according to a list in /etc/rc.conf. This is most useful if configinit is used to edit /etc/rc.conf, since it allows you to specify packages to install via the EC2 user-data.
\nWhile none of these are strictly necessary, I find them to be extremely useful and highly recommend implementing similar functionality in your systems.
\nSupport my work!
\nI hope you find this useful, or at very least interesting. Please consider supporting my work in this area; while I’m happy to contribute my time to supporting open source software, it would be nice if I had money coming in which I could use to cover incidental expenses (e.g., conference travel) so that I didn’t end up paying to contribute to FreeBSD.
Digital Ocean
\nhttps://do.co/bsdnow
\n\n\nAt a recent workshop on cybersecurity in the U.K., a primary topic of consideration was how to preserve the freedom and openness of the Internet while protecting against the harmful behaviors that have emerged in this global medium. That this is a significant challenge cannot be overstated. The bad behaviors range from social network bullying and misinformation to email spam, distributed denial of service attacks, direct cyberattacks against infrastructure, malware propagation, identity theft, and a host of other ills requiring a wide range of technical and legal considerations. That these harmful behaviors can and do cross international boundaries only makes it more difficult to fashion effective responses.
\n
\nIn other columns, I have argued for better software development tools to reduce the common mistakes that lead to vulnerabilities that are exploited. Here, I want to focus on another aspect of response related to law enforcement and tracking down perpetrators. Of course, not all harms are (or perhaps are not yet) illegal, but discovering those who cause them may still be warranted. The recent adoption and implementation of the General Data Protection Regulation (GDPR) in the European Union creates an interesting tension because it highlights the importance and value of privacy while those who do direct or indirect harm must be tracked down and their identities discovered.
\nIn passing, I mention that cryptography has sometimes been blamed for protecting the identity or actions of criminals but it is also a tool for protecting privacy. Arguments have been made for “back doors” to cryptographic systems but I am of the opinion that such proposals carry extremely high risk to privacy and safety. It is not my intent to argue this question in this column.
\nWhat is of interest to me is a concept to which I was introduced at the Ditchley workshop, specifically, differential traceability. The ability to trace bad actors to bring them to justice seems to me an important goal in a civilized society. The tension with privacy protection leads to the idea that only under appropriate conditions can privacy be violated. By way of example, consider license plates on cars. They are usually arbitrary identifiers and special authority is needed to match them with the car owners (unless, of course, they are vanity plates like mine: “Cerfsup”). This is an example of differential traceability; the police department has the authority to demand ownership information from the Department of Motor Vehicles that issues the license plates. Ordinary citizens do not have this authority.
\nIn the Internet environment there are a variety of identifiers associated with users (including corporate users). Domain names, IP addresses, email addresses, and public cryptography keys are examples among many others. Some of these identifiers are dynamic and thus ambiguous. For example, IP addresses are not always permanent and may change (for example, temporary IP addresses assigned at Wi-Fi hotspots) or may be ambiguous in the case of Network Address Translation. Information about the time of assignment and the party to whom an IP address was assigned may be needed to identify an individual user. There has been considerable debate and even a recent court case regarding requirements to register users in domain name WHOIS databases in the context of the adoption of GDPR. If we are to accomplish the simultaneous objectives of protecting privacy while apprehending those engaged in harmful or criminal behavior on the Internet, we must find some balance between conflicting but desirable outcomes.
\nThis suggests to me that the notion of traceability under (internationally?) agreed circumstances (that is, differential traceability) might be a fruitful concept to explore. In most societies today, it is accepted that we must be identifiable to appropriate authorities under certain conditions (consider border crossings, traffic violation stops as examples). While there are conditions under which apparent anonymity is desirable and even justifiable (whistle-blowing, for example) absolute anonymity is actually quite difficult to achieve (another point made at the Ditchley workshop) and might not be absolutely desirable given the misbehaviors apparent anonymity invites. I expect this is a controversial conclusion and I look forward to subsequent discussion.
###Remote Access Console using FreeBSD on an RPi3
\n\n\n\n\nFor the software I ended up using conserver. Below is a very brief tutorial on how to set everything up. I assume you have basic unix skills.
\n
\n\n\nA small bonus script I wrote to turn on the 2nd LED on the rPI once the system is booted, it will then blink the LED if someone is connected to any of the consoles.
\n
##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\nWe need more feedback emails. Please write to feedback@bsdnow.tv
Additionally, we are considering a new segment to be added to the end of the show (to make it skippable), where we have a ~15 minute deep dive on a topic. Some initial ideas are on the Virtual Memory subsystem, the Scheduler, Capsicum, and GEOM. What topics would you like to get very detailed explanations of? Many of the explanations may have accompanying graphics, and not be very suitable for audio only listeners, that is why we are planning to put it at the very end of the episode.
\n\nThe strange birth and long life of Unix, FreeBSD jail with a single public IP, EuroBSDcon 2018 talks and schedule, OpenBSD on G4 iBook, PAM template user, ZFS file server, and reflections on one year of OpenBSD use.
\n\n##Headlines
\n###The Strange Birth and Long Life of Unix
\n\n\nThey say that when one door closes on you, another opens. People generally offer this bit of wisdom just to lend some solace after a misfortune. But sometimes it’s actually true. It certainly was for Ken Thompson and the late Dennis Ritchie, two of the greats of 20th-century information technology, when they created the Unix operating system, now considered one of the most inspiring and influential pieces of software ever written.
\n
\nA door had slammed shut for Thompson and Ritchie in March of 1969, when their employer, the American Telephone & Telegraph Co., withdrew from a collaborative project with the Massachusetts Institute of Technology and General Electric to create an interactive time-sharing system called Multics, which stood for “Multiplexed Information and Computing Service.” Time-sharing, a technique that lets multiple people use a single computer simultaneously, had been invented only a decade earlier. Multics was to combine time-sharing with other technological advances of the era, allowing users to phone a computer from remote terminals and then read e-mail, edit documents, run calculations, and so forth. It was to be a great leap forward from the way computers were mostly being used, with people tediously preparing and submitting batch jobs on punch cards to be run one by one.
\nOver five years, AT&T invested millions in the Multics project, purchasing a GE-645 mainframe computer and dedicating to the effort many of the top researchers at the company’s renowned Bell Telephone Laboratories—including Thompson and Ritchie, Joseph F. Ossanna, Stuart Feldman, M. Douglas McIlroy, and the late Robert Morris. But the new system was too ambitious, and it fell troublingly behind schedule. In the end, AT&T’s corporate leaders decided to pull the plug.
\nAfter AT&T’s departure from the Multics project, managers at Bell Labs, in Murray Hill, N.J., became reluctant to allow any further work on computer operating systems, leaving some researchers there very frustrated. Although Multics hadn’t met many of its objectives, it had, as Ritchie later recalled, provided them with a “convenient interactive computing service, a good environment in which to do programming, [and] a system around which a fellowship could form.” Suddenly, it was gone.
\nWith heavy hearts, the researchers returned to using their old batch system. At such an inauspicious moment, with management dead set against the idea, it surely would have seemed foolhardy to continue designing computer operating systems. But that’s exactly what Thompson, Ritchie, and many of their Bell Labs colleagues did. Now, some 40 years later, we should be thankful that these programmers ignored their bosses and continued their labor of love, which gave the world Unix, one of the greatest computer operating systems of all time.
\nThe rogue project began in earnest when Thompson, Ritchie, and a third Bell Labs colleague, Rudd Canaday, began to sketch out on paper the design for a file system. Thompson then wrote the basics of a new operating system for the lab’s GE-645 mainframe. But with the Multics project ended, so too was the need for the GE-645. Thompson realized that any further programming he did on it was likely to go nowhere, so he dropped the effort.
\nThompson had passed some of his time after the demise of Multics writing a computer game called Space Travel, which simulated all the major bodies in the solar system along with a spaceship that could fly around them. Written for the GE-645, Space Travel was clunky to play—and expensive: roughly US $75 a game for the CPU time. Hunting around, Thompson came across a dusty PDP-7, a minicomputer built by Digital Equipment Corp. that some of his Bell Labs colleagues had purchased earlier for a circuit-analysis project. Thompson rewrote Space Travel to run on it.
\nAnd with that little programming exercise, a second door cracked ajar. It was to swing wide open during the summer of 1969 when Thompson’s wife, Bonnie, spent a month visiting his parents to show off their newborn son. Thompson took advantage of his temporary bachelor existence to write a good chunk of what would become the Unix operating system for the discarded PDP‑7. The name Unix stems from a joke one of Thompson’s colleagues made: Because the new operating system supported only one user (Thompson), he saw it as an emasculated version of Multics and dubbed it “Un-multiplexed Information and Computing Service,” or Unics. The name later morphed into Unix.
\nInitially, Thompson used the GE-645 to compose and compile the software, which he then downloaded to the PDP‑7. But he soon weaned himself from the mainframe, and by the end of 1969 he was able to write operating-system code on the PDP-7 itself. That was a step in the right direction. But Thompson and the others helping him knew that the PDP‑7, which was already obsolete, would not be able to sustain their skunkworks for long. They also knew that the lab’s management wasn’t about to allow any more research on operating systems.
\nSo Thompson and Ritchie got creative. They formulated a proposal to their bosses to buy one of DEC’s newer minicomputers, a PDP-11, but couched the request in especially palatable terms. They said they were aiming to create tools for editing and formatting text, what you might call a word-processing system today. The fact that they would also have to write an operating system for the new machine to support the editor and text formatter was almost a footnote.
\nManagement took the bait, and an order for a PDP-11 was placed in May 1970. The machine itself arrived soon after, although the disk drives for it took more than six months to appear. During the interim, Thompson, Ritchie, and others continued to develop Unix on the PDP-7. After the PDP-11’s disks were installed, the researchers moved their increasingly complex operating system over to the new machine. Next they brought over the roff text formatter written by Ossanna and derived from the runoff program, which had been used in an earlier time-sharing system.
\nUnix was put to its first real-world test within Bell Labs when three typists from AT&T’s patents department began using it to write, edit, and format patent applications. It was a hit. The patent department adopted the system wholeheartedly, which gave the researchers enough credibility to convince management to purchase another machine—a newer and more powerful PDP-11 model—allowing their stealth work on Unix to continue.
\nDuring its earliest days, Unix evolved constantly, so the idea of issuing named versions or releases seemed inappropriate. But the researchers did issue new editions of the programmer’s manual periodically, and the early Unix systems were named after each such edition. The first edition of the manual was completed in November 1971.
\nSo what did the first edition of Unix offer that made it so great? For one thing, the system provided a hierarchical file system, which allowed something we all now take for granted: Files could be placed in directories—or equivalently, folders—that in turn could be put within other directories. Each file could contain no more than 64 kilobytes, and its name could be no more than six characters long. These restrictions seem awkwardly limiting now, but at the time they appeared perfectly adequate.
\nAlthough Unix was ostensibly created for word processing, the only editor available in 1971 was the line-oriented ed. Today, ed is still the only editor guaranteed to be present on all Unix systems. Apart from the text-processing and general system applications, the first edition of Unix included games such as blackjack, chess, and tic-tac-toe. For the system administrator, there were tools to dump and restore disk images to magnetic tape, to read and write paper tapes, and to create, check, mount, and unmount removable disk packs.
\nMost important, the system offered an interactive environment that by this time allowed time-sharing, so several people could use a single machine at once. Various programming languages were available to them, including BASIC, Fortran, the scripting of Unix commands, assembly language, and B. The last of these, a descendant of a BCPL (Basic Combined Programming Language), ultimately evolved into the immensely popular C language, which Ritchie created while also working on Unix.
\nThe first edition of Unix let programmers call 34 different low-level routines built into the operating system. It’s a testament to the system’s enduring nature that nearly all of these system calls are still available—and still heavily used—on modern Unix and Linux systems four decades on. For its time, first-edition Unix provided a remarkably powerful environment for software development. Yet it contained just 4200 lines of code at its heart and occupied a measly 16 KB of main memory when it ran.
\nUnix’s great influence can be traced in part to its elegant design, simplicity, portability, and serendipitous timing. But perhaps even more important was the devoted user community that soon grew up around it. And that came about only by an accident of its unique history.
\nThe story goes like this: For years Unix remained nothing more than a Bell Labs research project, but by 1973 its authors felt the system was mature enough for them to present a paper on its design and implementation at a symposium of the Association for Computing Machinery. That paper was published in 1974 in the Communications of the ACM. Its appearance brought a flurry of requests for copies of the software.
\nThis put AT&T in a bind. In 1956, AT&T had agreed to a U.S government consent decree that prevented the company from selling products not directly related to telephones and telecommunications, in return for its legal monopoly status in running the country’s long-distance phone service. So Unix could not be sold as a product. Instead, AT&T released the Unix source code under license to anyone who asked, charging only a nominal fee. The critical wrinkle here was that the consent decree prevented AT&T from supporting Unix. Indeed, for many years Bell Labs researchers proudly displayed their Unix policy at conferences with a slide that read, “No advertising, no support, no bug fixes, payment in advance.”
\nWith no other channels of support available to them, early Unix adopters banded together for mutual assistance, forming a loose network of user groups all over the world. They had the source code, which helped. And they didn’t view Unix as a standard software product, because nobody seemed to be looking after it. So these early Unix users themselves set about fixing bugs, writing new tools, and generally improving the system as they saw fit.
\nThe Usenix user group acted as a clearinghouse for the exchange of Unix software in the United States. People could send in magnetic tapes with new software or fixes to the system and get back tapes with the software and fixes that Usenix had received from others. In Australia, the University of New South Wales and the University of Sydney produced a more robust version of Unix, the Australian Unix Share Accounting Method, which could cope with larger numbers of concurrent users and offered better performance.
\nBy the mid-1970s, the environment of sharing that had sprung up around Unix resembled the open-source movement so prevalent today. Users far and wide were enthusiastically enhancing the system, and many of their improvements were being fed back to Bell Labs for incorporation in future releases. But as Unix became more popular, AT&T’s lawyers began looking harder at what various licensees were doing with their systems.
\nOne person who caught their eye was John Lions, a computer scientist then teaching at the University of New South Wales, in Australia. In 1977, he published what was probably the most famous computing book of the time, A Commentary on the Unix Operating System, which contained an annotated listing of the central source code for Unix.
\nUnix’s licensing conditions allowed for the exchange of source code, and initially, Lions’s book was sold to licensees. But by 1979, AT&T’s lawyers had clamped down on the book’s distribution and use in academic classes. The antiauthoritarian Unix community reacted as you might expect, and samizdat copies of the book spread like wildfire. Many of us have nearly unreadable nth-generation photocopies of the original book.
\nEnd runs around AT&T’s lawyers indeed became the norm—even at Bell Labs. For example, between the release of the sixth edition of Unix in 1975 and the seventh edition in 1979, Thompson collected dozens of important bug fixes to the system, coming both from within and outside of Bell Labs. He wanted these to filter out to the existing Unix user base, but the company’s lawyers felt that this would constitute a form of support and balked at their release. Nevertheless, those bug fixes soon became widely distributed through unofficial channels. For instance, Lou Katz, the founding president of Usenix, received a phone call one day telling him that if he went down to a certain spot on Mountain Avenue (where Bell Labs was located) at 2 p.m., he would find something of interest. Sure enough, Katz found a magnetic tape with the bug fixes, which were rapidly in the hands of countless users.
\nBy the end of the 1970s, Unix, which had started a decade earlier as a reaction against the loss of a comfortable programming environment, was growing like a weed throughout academia and the IT industry. Unix would flower in the early 1980s before reaching the height of its popularity in the early 1990s.
\nFor many reasons, Unix has since given way to other commercial and noncommercial systems. But its legacy, that of an elegant, well-designed, comfortable environment for software development, lives on. In recognition of their accomplishment, Thompson and Ritchie were given the Japan Prize earlier this year, adding to a collection of honors that includes the United States’ National Medal of Technology and Innovation and the Association of Computing Machinery’s Turing Award. Many other, often very personal, tributes to Ritchie and his enormous influence on computing were widely shared after his death this past October.
\nUnix is indeed one of the most influential operating systems ever invented. Its direct descendants now number in the hundreds. On one side of the family tree are various versions of Unix proper, which began to be commercialized in the 1980s after the Bell System monopoly was broken up, freeing AT&T from the stipulations of the 1956 consent decree. On the other side are various Unix-like operating systems derived from the version of Unix developed at the University of California, Berkeley, including the one Apple uses today on its computers, OS X. I say “Unix-like” because the developers of the Berkeley Software Distribution (BSD) Unix on which these systems were based worked hard to remove all the original AT&T code so that their software and its descendants would be freely distributable.
\nThe effectiveness of those efforts were, however, called into question when the AT&T subsidiary Unix System Laboratories filed suit against Berkeley Software Design and the Regents of the University of California in 1992 over intellectual property rights to this software. The university in turn filed a counterclaim against AT&T for breaches to the license it provided AT&T for the use of code developed at Berkeley. The ensuing legal quagmire slowed the development of free Unix-like clones, including 386BSD, which was designed for the Intel 386 chip, the CPU then found in many IBM PCs.
\nHad this operating system been available at the time, Linus Torvalds says he probably wouldn’t have created Linux, an open-source Unix-like operating system he developed from scratch for PCs in the early 1990s. Linux has carried the Unix baton forward into the 21st century, powering a wide range of digital gadgets including wireless routers, televisions, desktop PCs, and Android smartphones. It even runs some supercomputers.
\nAlthough AT&T quickly settled its legal disputes with Berkeley Software Design and the University of California, legal wrangling over intellectual property claims to various parts of Unix and Linux have continued over the years, often involving byzantine corporate relations. By 2004, no fewer than five major lawsuits had been filed. Just this past August, a software company called the TSG Group (formerly known as the SCO Group), lost a bid in court to claim ownership of Unix copyrights that Novell had acquired when it purchased the Unix System Laboratories from AT&T in 1993.
\nAs a programmer and Unix historian, I can’t help but find all this legal sparring a bit sad. From the very start, the authors and users of Unix worked as best they could to build and share, even if that meant defying authority. That outpouring of selflessness stands in sharp contrast to the greed that has driven subsequent legal battles over the ownership of Unix.
\nThe world of computer hardware and software moves forward startlingly fast. For IT professionals, the rapid pace of change is typically a wonderful thing. But it makes us susceptible to the loss of our own history, including important lessons from the past. To address this issue in a small way, in 1995 I started a mailing list of old-time Unix aficionados. That effort morphed into the Unix Heritage Society. Our goal is not only to save the history of Unix but also to collect and curate these old systems and, where possible, bring them back to life. With help from many talented members of this society, I was able to restore much of the old Unix software to working order, including Ritchie’s first C compiler from 1972 and the first Unix system to be written in C, dating from 1973.
\nOne holy grail that eluded us for a long time was the first edition of Unix in any form, electronic or otherwise. Then, in 2006, Al Kossow from the Computer History Museum, in Mountain View, Calif., unearthed a printed study of Unix dated 1972, which not only covered the internal workings of Unix but also included a complete assembly listing of the kernel, the main component of this operating system. This was an amazing find—like discovering an old Ford Model T collecting dust in a corner of a barn. But we didn’t just want to admire the chrome work from afar. We wanted to see the thing run again.
\nIn 2008, Tim Newsham, an independent programmer in Hawaii, and I assembled a team of like-minded Unix enthusiasts and set out to bring this ancient system back from the dead. The work was technically arduous and often frustrating, but in the end, we had a copy of the first edition of Unix running on an emulated PDP-11/20. We sent out messages announcing our success to all those we thought would be interested. Thompson, always succinct, simply replied, “Amazing.” Indeed, his brainchild was amazing, and I’ve been happy to do what I can to make it, and the story behind it, better known.
Digital Ocean
\nhttp://do.co/bsdnow
###FreeBSD jails with a single public IP address
\n\n\n\n\nJails in FreeBSD provide a simple yet flexible way to set up a proper server layout. In the most setups the actual server only acts as the host system for the jails while the applications themselves run within those independent containers. Traditionally every jail has it’s own IP for the user to be able to address the individual services. But if you’re still using IPv4 this might get you in trouble as the most hosters don’t offer more than one single public IP address per server.
\n
\n\n\nIn this case NAT (“Network Address Translation”) is a good way to expose services in different jails using the same IP address.
\n
\nFirst, let’s create an internal network (“NAT network”) at 192.168.0.0/24. You could generally use any private IPv4 address space as specified in RFC 1918. Here’s an overview: https://en.wikipedia.org/wiki/Private_network. Using pf, FreeBSD’s firewall, we will map requests on different ports of the same public IP address to our individual jails as well as provide network access to the jails themselves.
\nFirst let’s check which network devices are available. In my case there’s em0 which provides connectivity to the internet and lo0, the local loopback device.
options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>\n [...]\n inet 172.31.1.100 netmask 0xffffff00 broadcast 172.31.1.255\n nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>\n media: Ethernet autoselect (1000baseT <full-duplex>)\n status: active\n\nlo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384\n options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>\n inet6 ::1 prefixlen 128\n inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2\n inet 127.0.0.1 netmask 0xff000000\n nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>```\n\n> For our internal network, we create a cloned loopback device called lo1. Therefore we need to customize the /etc/rc.conf file, adding the following two lines:\n\n```cloned_interfaces="lo1"\nipv4_addrs_lo1="192.168.0.1-9/29"```\n\n> This defines a /29 network, offering IP addresses for a maximum of 6 jails:\n\n```ipcalc 192.168.0.1/29\nAddress: 192.168.0.1 11000000.10101000.00000000.00000 001\nNetmask: 255.255.255.248 = 29 11111111.11111111.11111111.11111 000\nWildcard: 0.0.0.7 00000000.00000000.00000000.00000 111\n=>\nNetwork: 192.168.0.0/29 11000000.10101000.00000000.00000 000\nHostMin: 192.168.0.1 11000000.10101000.00000000.00000 001\nHostMax: 192.168.0.6 11000000.10101000.00000000.00000 110\nBroadcast: 192.168.0.7 11000000.10101000.00000000.00000 111\nHosts/Net: 6 Class C, Private Internet```\n\n> Then we need to restart the network. Please be aware of currently active SSH sessions as they might be dropped during restart. It’s a good moment to ensure you have KVM access to that server ;-)\n\n```service netif restart```\n\n> After reconnecting, our newly created loopback device is active:\n\n```lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384\n options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>\n inet 192.168.0.1 netmask 0xfffffff8\n inet 192.168.0.2 netmask 0xffffffff\n inet 192.168.0.3 netmask 0xffffffff\n inet 192.168.0.4 netmask 0xffffffff\n inet 192.168.0.5 netmask 0xffffffff\n inet 192.168.0.6 netmask 0xffffffff\n inet 192.168.0.7 netmask 0xffffffff\n inet 192.168.0.8 netmask 0xffffffff\n inet 192.168.0.9 netmask 0xffffffff\n nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>```\n\n+ Setting up\n\n> pf part of the FreeBSD base system, so we only have to configure and enable it. By this moment you should already have a clue of which services you want to expose. If this is not the case, just fix that file later on. In my example configuration, I have a jail running a webserver and another jail running a mailserver:\n\n + Public IP address\n```IP_PUB="1.2.3.4"```\n\n + Packet normalization\n```scrub in all```\n\n + Allow outbound connections from within the jails\n```nat on em0 from lo1:network to any -> (em0)```\n\n + webserver jail at 192.168.0.2\n```rdr on em0 proto tcp from any to $IP_PUB port 443 -> 192.168.0.2```\n\n + just an example in case you want to redirect to another port within your jail\n```rdr on em0 proto tcp from any to $IP_PUB port 80 -> 192.168.0.2 port 8080```\n\n + mailserver jail at 192.168.0.3\n```rdr on em0 proto tcp from any to $IP_PUB port 25 -> 192.168.0.3```\n```rdr on em0 proto tcp from any to $IP_PUB port 587 -> 192.168.0.3```\n```rdr on em0 proto tcp from any to $IP_PUB port 143 -> 192.168.0.3```\n```rdr on em0 proto tcp from any to $IP_PUB port 993 -> 192.168.0.3```\n\n> Now just enable pf like this (which is the equivalent of adding pf_enable=YES to /etc/rc.conf):\n\n```sysrc pf_enable="YES"```\n\n> and start it:\n\n```service pf start```\n\n+ Install ezjail\n\n> Ezjail is a collection of scripts by erdgeist that allow you to easily manage your jails.\n\n```pkg install ezjail```\n\n> As an alternative, you could install ezjail from the ports tree. Now we need to set up the basejail which contains the shared base system for our jails. In fact, every jail that you create get’s will use that basejail to symlink directories related to the base system like /bin and /sbin. This can be accomplished by running\n\n```ezjail-admin install```\n\n> In the next step, we’ll copy the /etc/resolv.conf file from our host to the newjail, which is the template for newly created jails (the parts that are not provided by basejail), to ensure that domain resolution will work properly within our jails later on:\n\n```cp /etc/resolv.conf /usr/jails/newjail/etc/```\n\n> Last but not least, we enable ezjail and start it:\n\n```sysrc ezjail_enable="YES"```\n```service ezjail start```\n\n+ Create a jail\n\n> Creating a jail is as easy as it could probably be:\n\n```ezjail-admin create webserver 192.168.0.2```\n```ezjail-admin start webserver```\n\n> Now you can access your jail using:\n\n```ezjail-admin console webserver```\n\n> Each jail contains a vanilla FreeBSD installation.\n\n+ Deploy services\n\n> Now you can spin up as many jails as you want to set up your services like web, mail or file shares. You should take care not to enable sshd within your jails, because that would cause problems with the service’s IP bindings. But this is not a problem, just SSH to the host and enter your jail using ezjail-admin console.\n***\n\n###[EuroBSDcon 2018 Talks & Schedule](https://2018.eurobsdcon.org/talks-schedule/)\n***\n\n\n\n\n##News Roundup\n###[OpenBSD on an iBook G4](https://bobstechsite.com/openbsd-on-an-ibook-g4/)\n> I've mentioned on social media and on the BTS podcast a few times that I wanted to try installing OpenBSD onto an old "snow white" iBook G4 I acquired last summer to see if I could make it a useful machine again in the year 2018. This particular eBay purchase came with a 14" 1024x768 TFT screen, 1.07GHz PowerPC G4 processor, 1.5GB RAM, 100GB of HDD space and an ATI Radeon 9200 graphics card with 32 MB of SDRAM. The optical drive, ethernet port, battery & USB slots are also fully-functional. The only thing that doesn't work is the CMOS battery, but that's not unexpected for a device that was originally released in 2004.\n\n+ Initial experiments\n\n> This iBook originally arrived at my door running Apple Mac OSX Leopard and came with the original install disk, the iLife & iWork suites for 2008, various instruction manuals, a working power cable and a spare keyboard. As you'll see in the pictures I took for this post the characters on the buttons have started to wear away from 14 years of intensive use, but the replacement needs a very good clean before I decide to swap it in!\n\n> After spending some time exploring the last version of OSX to support the IBM PowerPC processor architecture I tried to see if the hardware was capable of modern computing with Linux. Something I knew ahead of trying this was that the WiFi adapter was unlikely to work because it's a highly proprietary component designed by Apple to work specifically with OSX and nothing else, but I figured I could probably use a wireless USB dongle later to get around this limitation.\n\n> Unfortunately I found that no recent versions of mainstream Linux distributions would boot off this machine. Debian has dropped support 32-bit PowerPC architectures and the PowerPC variants of Ubuntu 16.04 LTS (vanilla, MATE and Lubuntu) wouldn't even boot the installer! The only distribution I could reliably install on the hardware was Lubuntu 14.04 LTS.\n\n> Unfortunately I'm not the biggest fan of the LXDE desktop for regular work and a lot of ported applications were old and broken because it clearly wasn't being maintained by people that use the hardware anymore. Ubuntu 14.04 is also approaching the end of its support life in early 2019, so this limited solution also has a limited shelf-life.\n\n+ Over to BSD\n\n> I discussed this problem with a few people on Mastodon and it was pointed out to me that OSX is built on the Darwin kernel, which happens to be a variant of BSD. NetBSD and OpenBSD fans in particular convinced me that their communities still saw the value of supporting these old pieces of kit and that I should give BSD a try.\n\n> So yesterday evening I finally downloaded the "macppc" version of OpenBSD 6.3 with no idea what to expect. I hoped for the best but feared the worst because my last experience with this operating system was trying out PC-BSD in 2008 and discovering with disappointment that it didn't support any of the hardware on my Toshiba laptop.\n\n> When I initially booted OpenBSD I was a little surprised to find the login screen provided no visual feedback when I typed in my password, but I can understand the security reasons for doing that. The initial desktop environment that was loaded was very basic. All I could see was a console output window, a terminal and a desktop switcher in the X11 environment the system had loaded.\n\n> After a little Googling I found this blog post had some fantastic instructions to follow for the post-installation steps: https://sohcahtoa.org.uk/openbsd.html. I did have to adjust them slightly though because my iBook only has 1.5GB RAM and not every package that page suggests is available on macppc by default. You can see a full list here: https://ftp.openbsd.org/pub/OpenBSD/6.3/packages/powerpc/.\n\n+ Final thoughts\n\n> I was really impressed with the performance of OpenBSD's "macppc" port. It boots much faster than OSX Leopard on the same hardware and unlike Lubuntu 14.04 it doesn't randomly hang for no reason or crash if you launch something demanding like the GIMP.\n\n> I was pleased to see that the command line tools I'm used to using on Linux have been ported across too. OpenBSD also had no issues with me performing basic desktop tasks on XFCE like browsing the web with NetSurf, playing audio files with VLC and editing images with the GIMP. Limited gaming is also theoretically possible if you're willing to build them (or an emulator) from source with SDL support.\n\n> If I wanted to use this system for heavy duty work then I'd probably be inclined to run key applications like LibreOffice on a Raspberry Pi and then connect my iBook G4 to those using VNC or an SSH connection with X11 forwarding. BSD is UNIX after all, so using my ancient laptop as a dumb terminal should work reasonably well.\n\n> In summary I was impressed with OpenBSD and its ability to breathe new life into this old Apple Mac. I'm genuinely excited about the idea of trying BSD with other devices on my network such as an old Asus Eee PC 900 netbook and at least one of the many Raspberry Pi devices I use. Whether I go the whole hog and replace Fedora on my main production laptop though remains to be seen!\n\n***\n\n###[The template user with PAM and login(1)](http://oshogbo.vexillium.org/blog/48)\n> When you build a new service (or an appliance) you need your users to be able to configure it from the command line. To accomplish this you can create system accounts for all registered users in your service and assign them a special login shell which provides such limited functionality. This can be painful if you have a dynamic user database.\n> Another challenge is authentication via remote services such as RADIUS. How can we implement services when we authenticate through it and log into it as a different user? Furthermore, imagine a scenario when RADIUS decides on which account we have the right to access by sending an additional attribute.\n> To address these two problems we can use a "template" user. Any of the PAM modules can set the value of the PAM_USER item. The value of this item will be used to determine which account we want to login. Only the "template" user must exist on the local password database, but the credential check can be omitted by the module.\n> This functionality exists in the login(1) used by FreeBSD, HardenedBSD, DragonFlyBSD and illumos. The functionality doesn't exist in the login(1) used in NetBSD, and OpenBSD doesn't support PAM modules at all. In addition what is also noteworthy is that such functionality was also in the OpenSSH but they decided to remove it and call it a security vulnerability (CVE 2015-6563). I can see how some people may have seen it that way, that’s why I recommend reading this article from an OpenPAM author and a FreeBSD security officer at the time.\n> Knowing the background let's take a look at an example.\n\n```PAM_EXTERN int\npam_sm_authenticate(pam_handle_t *pamh, int flags __unused,\n int argc __unused, const char *argv[] __unused)\n{\n const char *user, *password;\n int err;\n\n err = pam_get_user(pamh, &user, NULL);\n if (err != PAM_SUCCESS)\n return (err);\n\n err = pam_get_authtok(pamh, PAM_AUTHTOK, &password, NULL);\n if (err == PAM_CONV_ERR)\n return (err);\n if (err != PAM_SUCCESS)\n return (PAM_AUTH_ERR);\n\n err = authenticate(user, password);\n if (err != PAM_SUCCESS) {\n return (err);\n }\n\n return (pam_set_item(pamh, PAM_USER, "template"));\n}\n
\n\n\n\n\nIn the listing above we have an example of a PAM module. The pam_get_user(3) provides a username. The pam_get_authtok(3) shows us a secret given by the user. Both functions allow us to give an optional prompt which should be shown to the user. The authenticate function is our crafted function which authenticates the user. In our first scenario we wanted to keep all users in an external database. If authentication is successful we then switch to a template user which has a shell set up for a script allowing us to configure the machine. In our second scenario the authenticate function authenticates the user in RADIUS.
\n
\n\n\nAnother step is to add our PAM module to the /etc/pam.d/system or to the /etc/pam.d/login configuration:
\n
auth sufficient pam_template.so no_warn allow_local
\n\n\nUnfortunately the description of all these options goes beyond this article - if you would like to know more about it you can find them in the PAM manual. The last thing we need to do is to add our template user to the system which you can do by the adduser(8) command or just simply modifying the /etc/master.passwd file and use pwd_mkdb(8) program:
\n
$ tail -n /etc/master.passwd
\ntemplate:*:1000:1000::0:0:User &:/:/usr/local/bin/templatesh
\n$ sudo pwd_mkdb /etc/master.passwd
\n\n\nAs you can see,the template user can be locked and we still can use it in our PAM module (the * character after login).
\n
\nI would like to thank Dag-Erling Smørgrav for pointing this functionality out to me when I was looking for it some time ago.
iXsystems
\niXsystems @ VMWorld
\n\n\nAt work, we run a compute cluster that uses an Isilon cluster as primary NAS storage. Excluding snapshots, we have about 200TB of research data, some of them in compressed formats, and others not. We needed an offsite backup file server that would constantly mirror our primary NAS and serve as a quick recovery source in case of a data loss in the the primary NAS. This offsite file server would be passive - will never face the wrath of the primary cluster workload.
\n
\nIn addition to the role of a passive backup server, this solution would take on some passive report generation workloads as an ideal way of offloading some work from the primary NAS. The passive work is read-only.
\nThe backup server would keep snapshots in a best effort basis dating back to 10 years. However, this data on this backup server would be archived to tapes periodically.
A simple guidance of priorities:
\nData integrity > Cost of solution > Storage capacity > Performance.
\nWhy not enterprise NAS? NetApp FAS or EMC Isilon or the like?
\n\n\n\nWe decided that enterprise grade NAS like NetAPP FAS or EMC Isilon are prohibitively expensive and an overkill for our needs.
\n
\nAn open source & cheaper alternative to enterprise grade filesystem with the level of durability we expect turned up to be ZFS. We’re already spoilt from using snapshots by a clever Copy-on-Write Filesystem(WAFL) by NetApp. ZFS providing snapshots in almost identical way was a big influence in the choice. This is also why we did not consider just a CentOS box with the default XFS filesystem.
\n\n\nThis is a backup server, a long-term solution. Stability and reliability are key requirements. ZFS on Linux may be popular at this time, but there is a lot of churn around its development, which means there is a higher probability of bugs like this to occur. We’re not looking for cutting edge features here. Perhaps, Linux would be considered in the future.
\n
\n\n\nWe already utilize FreeBSD and OpenBSD for infrastructure services and we have nothing but praises for the stability that the BSDs have provided us. We’d gladly use FreeBSD and OpenBSD wherever possible.
\n
\n\n\nIMHO, FreeNAS provides a integrated GUI management tool over FreeBSD for a novice user to setup and configure FreeBSD, ZFS, Jails and many other features. But, this user facing abstraction adds an extra layer of complexity to maintain that is just not worth it in simpler use cases like ours. For someone that appreciates the commandline interface, and understands FreeBSD enough to administer it, plain FreeBSD + ZFS is simpler and more robust than FreeNAS.
\n
###Reflection on one-year usage of OpenBSD
\n\n\n\n\nI have used OpenBSD for more than one year, and it is time to give a summary of the experience:
\n
\n\n\na) A good UNIX tutorial. When I am curious about some UNIXcommands’ implementation, I will refer to OpenBSD source code, and I actually gain something every time. E.g., refresh socket programming skills from nc; know how to process file efficiently from cat.
\n
\n\n\nb) A better test bed. Although my work focus on developing programs on Linux, I will try to compile and run applications on OpenBSD if it is possible. One reason is OpenBSD usually gives more helpful warnings. E.g., hint like this:
\n
......
\nwarning: sprintf() is often misused, please use snprintf()
\n......
\n\n\nOr you can refer this post which I wrote before. The other is sometimes program run well on Linux may crash on OpenBSD, and OpenBSD can help you find hidden bugs.
\n
\n\n\nc) Some handy tools. E.g. I find tcpbench is useful, so I ported it into Linux for my own usage (project is here).
\n
\n\n\na) Patches. Although most of them are trivial modifications, they are still my contributions.
\n
\n\n\nb) Write blog posts to share experience about using OpenBSD.
\n
\n\n\nc) Develop programs for OpenBSD/*BSD: lscpu and free.
\n
\n\n\nd) Porting programs into OpenBSD: E.g., I find google/benchmark is a nifty tool, but lacks OpenBSD support, I submitted PR and it is accepted. So you can use google/benchmark on OpenBSD now.
\n
##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nFreeBSD Foundation July Newsletter, a bunch of BSDCan trip reports, HardenedBSD Foundation status, FreeBSD and OSPFd, ZFS disk structure overview, and more Spectre mitigations in OpenBSD.
\n\n##Headlines
\n###FreeBSD Foundation Update, July 2018
\n\n\nWe’re in the middle of summer here, in Boulder, CO. While the days are typically hot, they can also be quite unpredictable. Thanks to the Rocky Mountains, waking up to 50-degree (~10 C) foggy weather is not surprising. In spite of the unpredictable weather, many of us took some vacation this month. Whether it was extending the Fourth of July celebration, spending time with family, or relaxing and enjoying the summer weather, we appreciated our time off, while still managing to accomplish a lot!
\n
\nIn this newsletter, Glen Barber enlightens us about the upcoming 12.0 release. I gave a recap of OSCON, that Ed Maste and I attended, and Mark Johnston explains the work on his improved microcode loading project, that we are funding. Finally, Anne Dickison gives us a rundown on upcoming events and information on submitting a talk for MeetBSD.
\nYour support helps us continue this work. Please consider making a donation today. We can’t do it without you. Happy reading!!
iXsystems
\n\n###BSDCan Trip Reports
\n\n##News Roundup
\n###FreeBSD and OSPFd
\n\n\nWith FreeBSD jails deployed around the world, static routing was getting a bit out of hand. Plus, when I needed to move a jail from one data center to another, I would have to update routing tables across multiple sites. Not ideal. Enter dynamic routing…
\n
\n\n\nOSPF (open shortest path first) is an internal dynamic routing protocol that provides the autonomy that I needed and it’s fairly easy to setup. This article does not cover configuration of VPN links, ZFS, or Freebsd jails, however it’s recommended that you use seperate ZFS datasets per jail so that migration between hosts can be done with zfs send & receive.
\n
\n\n\nIn this scenario, we have five FreeBSD servers in two different data centers. Each physical server runs anywhere between three to ten jails. When jails are deployed, they are assigned a /32 IP on lo2. From here, pf handles inbound port forwarding and outbound NAT. Links between each server are provided by OpenVPN TAP interfaces. (I used TAP to pass layer 2 traffic. I seem to remember that I needed TAP interfaces due to needing GRE tunnels on top of TUN interfaces to get OSPF to communicate. I’ve heard TAP is slower than TUN so I may revisit this.)
\n
\n\n\nIn this example, we will use 172.16.2.0/24 as the range for OpenVPN P2P links and 172.16.3.0/24 as the range of IPs available for assignment to each jail. Previously, when deploying a jail, I assigned IPs based on the following groups:
\n
Server 1: 172.16.3.0/28
\nServer 2: 172.16.3.16/28
\nServer 3: 172.16.3.32/28
\nServer 4: 172.16.3.48/28
\nServer 5: 172.16.3.64/28
\n\n\nWhen statically routing, this made routing tables a bit smaller and easier to manage. However, when I needed to migrate a jail to a new host, I had to add a new /32 to all routing tables. Now, with OSPF, this is no longer an issue, nor is it required.
\n
To get started, first we install the Quagga package.
\nThe two configuration files needed to get OSPFv2 running are /usr/local/etc/quagga/zebra.conf and /usr/local/etc/quagga/ospfd.conf.
\nStarting with zebra.conf, we’ll define the hostname and a management password.
\nSecond, we will populate the ospfd.conf file.
\nTo break this down:
\nservice advanced-vty allows you to skip the en or enable command. Since I’m the only one who uses this service, it’s one less command to type.
\nip ospf authentication message-digest and ip ospf message-diget-key… ignores non-authenticated OSPF communication. This is useful when communicating over the WAN and to prevent a replay attack. Since I’m using a VPN to communicate, I could exclude these.
\npassive-interface default turns off the active communication of OSPF messages on all interfaces except for the interfaces listed as no passive-interface [interface name]. Since my ospf communication needs to leverage the VPNs, this prevents the servers from trying to send ospf data out the WAN interface (a firewall would work too).
\nnetwork 172.16.2.0/23 area 0.0.0.0 lists a supernet of both 172.16.2.0/24 and 172.16.3.0/24. This ensures routes for the jails are advertised along with the P2P links used by OpenVPN. The OpenVPN links are not required but can provide another IP to access your server if one of the links goes down. (See the suggested tasks below).
\nAt this point, we can enable the services in rc.conf.local and start them.
\nWe bind the management interface to 127.0.0.1 so that it’s only accessable to local telnet sessions. If you want to access this service remotely, you can bind to a remotely accessable IP. Remember telnet is not secure. If you need remote access, use a VPN.
\nTo manage the services, you can telnet to your host’s localhost address.
\nUse 2604 for the ospf service.
\nRemember, this is accessible by non-root users so set a good password.
\n###A broad overview of how ZFS is structured on disk
\n\n\n\n\nWhen I wrote yesterday’s entry, it became clear that I didn’t understand as much about how ZFS is structured on disk (and that this matters, since I thought that ZFS copy on write updates updated a lot more than they do). So today I want to write down my new broad understanding of how this works. (All of this can be dug out of the old, draft ZFS on-disk format specification, but that spec is written in a very detailed way and things aren’t always immediately clear from it.)
\n
\n\n\nAlmost everything in ZFS is in DMU object. All objects are defined by a dnode, and object dnodes are almost always grouped together in an object set. Object sets are themselves DMU objects; they store dnodes as basically a giant array in a ‘file’, which uses data blocks and indirect blocks and so on, just like anything else. Within a single object set, dnodes have an object number, which is the index of their position in the object set’s array of dnodes. (Because an object number is just the index of the object’s dnode in its object set’s array of dnodes, object numbers are basically always going to be duplicated between object sets (and they’re always relative to an object set). For instance, pretty much every object set is going to have an object number ten, although not all object sets may have enough objects that they have an object number ten thousand. One corollary of this is that if you ask zdb to tell you about a given object number, you have to tell zdb what object set you’re talking about. Usually you do this by telling zdb which ZFS filesystem or dataset you mean.)
\n
\n\n\nEach ZFS filesystem has its own object set for objects (and thus dnodes) used in the filesystem. As I discovered yesterday, every ZFS filesystem has a directory hierarchy and it may go many levels deep, but all of this directory hierarchy refers to directories and files using their object number.
\n
\n\n\nZFS organizes and keeps track of filesystems, clones, and snapshots through the DSL (Dataset and Snapshot Layer). The DSL has all sorts of things; DSL directories, DSL datasets, and so on, all of which are objects and many of which refer to object sets (for example, every ZFS filesystem must refer to its current object set somehow). All of these DSL objects are themselves stored as dnodes in another object set, the Meta Object Set, which the uberblock points to. To my surprise, object sets are not stored in the MOS (and as a result do not have ‘object numbers’). Object sets are always referred to directly, without indirection, using a block pointer to the object set’s dnode. (I think object sets are referred to directly so that snapshots can freeze their object set very simply.)
\n
\n\n\nThe DSL directories and datasets for your pool’s set of filesystems form a tree themselves (each filesystem has a DSL directory and at least one DSL dataset). However, just like in ZFS filesystems, all of the objects in this second tree refer to each other indirectly, by their MOS object number. Just as with files in ZFS filesystems, this level of indirection limits the amount of copy on write updates that ZFS had to do when something changes.
\n
\n\n\nPS: If you want to examine MOS objects with zdb, I think you do it with something like ‘zdb -vvv -d ssddata 1’, which will get you object number 1 of the MOS, which is the MOS object directory. If you want to ask zdb about an object in the pool’s root filesystem, use ‘zdb -vvv -d ssddata/ 1’. You can tell which one you’re getting depending on what zdb prints out. If it says ‘Dataset mos [META]’ you’re looking at objects from the MOS; if it says ‘Dataset ssddata [ZPL]’, you’re looking at the pool’s root filesystem (where object number 1 is the ZFS master node).
\n
\n\n\nPPS: I was going to write up what changed on a filesystem write, but then I realized that I didn’t know how blocks being allocated and freed are reflected in pool structures. So I’ll just say that I think that ignoring free space management, only four DMU objects get updated; the file itself, the filesystem’s object set, the filesystem’s DSL dataset object, and the MOS.
\n
Digital Ocean
\n\n###HardenedBSD Foundation Status
\n\n\n\n\nOn 09 July 2018, the HardenedBSD Foundation Board of Directors held the kick-off meeting to start organizing the Foundation. The following people attended the kick-off meeting:
\n
\n\n\nWe discussed the very first steps that need to be taken to organize the HardenedBSD Foundation as a 501©(3) not-for-profit organization in the US. We determined we could file a 1023EZ instead of the full-blown 1023. This will help speed the process up drastically.
\n
\n\n\nWe added Christian Severt, who is on Emerald Onion’s Board of Directors, to the HardenedBSD Foundation Board of Directors as an advisor. He was foundational in getting Emerald Onion their 501©(3) tax-exempt, not-for-profit status and has really good insight. Additionally, he’s going to help HardenedBSD coordinate hosting services, figuring out the best deals for us.
\n
\n\n\nWe promoted George Saylor to Vice President and changed Shawn Webb’s title to President and Director. This is to help resolve potential concerns both the state and federal agencies might have with an organization having only a single President role.
\n
\n\n\nWe hope to be granted our 501©(3) status before the end of the year, though that may be subject to change. We are excited for the formation of the HardenedBSD Foundation, which will open up new opportunities not otherwise available to HardenedBSD.
\n
###More mitigations against speculative execution vulnerabilities
\n\n\n\n\nPhilip Guenther (guenther@) and Bryan Steele (brynet@) have added more mitigations against speculative execution CPU vulnerabilities on the amd64 platform.
\n
\nCVSROOT: /cvs\nModule name: src\nChanges by: guenther@cvs.openbsd.org 2018/07/23 11:54:04\n\nModified files:\n sys/arch/amd64/amd64: locore.S \n sys/arch/amd64/include: asm.h cpufunc.h frameasm.h \n\nLog message:\nDo "Return stack refilling", based on the "Return stack underflow" discussion\nand its associated appendix at https://support.google.com/faqs/answer/7625886\nThis should address at least some cases of "SpectreRSB" and earlier\nSpectre variants; more commits to follow.\n\nThe refilling is done in the enter-kernel-from-userspace and\nreturn-to-userspace-from-kernel paths, making sure to do it before\nunblocking interrupts so that a successive interrupt can't get the\nCPU to C code without doing this refill. Per the link above, it\nalso does it immediately after mwait, apparently in case the low-power\nCPU states of idle-via-mwait flush the RSB.\n\nok mlarkin@ deraadt@```\n\n+ and:\n\n```CVSROOT: /cvs\nModule name: src\nChanges by: guenther@cvs.openbsd.org 2018/07/23 20:42:25\n\nModified files:\n sys/arch/amd64/amd64: locore.S vector.S vmm_support.S \n sys/arch/amd64/include: asm.h cpufunc.h \n\nLog message:\nAlso do RSB refilling when context switching, after vmexits, and\nwhen vmlaunch or vmresume fails.\n\nFollow the lead of clang and the intel recommendation and do an lfence\nafter the pause in the speculation-stop path for retpoline, RSB refill,\nand meltover ASM bits.\n\nok kettenis@ deraadt@```\n\n+ "Mitigation G-2" for AMD processors:\n\n```CVSROOT: /cvs\nModule name: src\nChanges by: brynet@cvs.openbsd.org 2018/07/23 17:25:03\n\nModified files:\n sys/arch/amd64/amd64: identcpu.c \n sys/arch/amd64/include: specialreg.h \n\nLog message:\nAdd "Mitigation G-2" per AMD's Whitepaper "Software Techniques for\nManaging Speculation on AMD Processors"\n\nBy setting MSR C001_1029[1]=1, LFENCE becomes a dispatch serializing\ninstruction.\n\nTested on AMD FX-4100 "Bulldozer", and Linux guest in SVM vmd(8)\n\nok deraadt@ mlarkin@```\n***\n\n\n##Beastie Bits\n+ [HardenedBSD will stop supporting 10-STABLE on 10 August 2018](https://groups.google.com/a/hardenedbsd.org/forum/#!topic/users/xvU0g-g1l5U)\n+ [GSoC 2018 Reports: Integrate libFuzzer with the Basesystem, Part 2](https://blog.netbsd.org/tnf/entry/gsoc_2018_reports_integrate_libfuzzer1)\n+ [ZFS Boot Environments at PBUG](https://vermaden.wordpress.com/2018/07/30/zfs-boot-environments-at-pbug/)\n+ [Second Editions versus the Publishing Business](https://blather.michaelwlucas.com/archives/3229)\n+ [Theo de Raadt on "unveil(2) usage in base"](https://undeadly.org/cgi?action=article;sid=20180728063716)\n+ [rtadvd(8) has been replaced by rad(8)](https://undeadly.org/cgi?action=article;sid=20180724072205)\n+ [BSD Users Stockholm Meetup #3](https://www.meetup.com/BSD-Users-Stockholm/events/253447019/)\n+ [Changes to NetBSD release support policy](https://blog.netbsd.org/tnf/entry/changes_to_netbsd_release_support)\n+ [The future of HAMMER1](http://lists.dragonflybsd.org/pipermail/users/2018-July/357832.html)\n***\n\n**Tarsnap**\n\n##Feedback/Questions\n+ Rodriguez - [A Question](http://dpaste.com/0Y1B75Q#wrap)\n+ Shane - [About ZFS Mostly](http://dpaste.com/32YGNBY#wrap)\n+ Leif - [ZFS less than 8gb](http://dpaste.com/2GY6HHC#wrap)\n+ Wayne - [ZFS vs EMC](http://dpaste.com/17PSCXC#wrap)\n***\n\n- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [feedback@bsdnow.tv](mailto:feedback@bsdnow.tv)\n
","summary":"FreeBSD Foundation July Newsletter, a bunch of BSDCan trip reports, HardenedBSD Foundation status, FreeBSD and OSPFd, ZFS disk structure overview, and more Spectre mitigations in OpenBSD.","date_published":"2018-08-08T01:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/2975f51c-21d4-41df-bae9-4e3616147a50.mp3","mime_type":"audio/mp3","size_in_bytes":52903277,"duration_in_seconds":5272}]},{"id":"http://feed.jupiter.zone/bsdnow#entry-2354","title":"Episode 257: Great NetBSD 8 | BSD Now 257","url":"https://www.bsdnow.tv/257","content_text":"NetBSD 8.0 available, FreeBSD on Scaleway’s ARM64 VPS, encrypted backups with OpenBSD, Dragonfly server storage upgrade, zpool checkpoints, g2k18 hackathon reports, and more.\n\n\n##Headlines\n###NetBSD v8.0 Released\n\n\nThe NetBSD Project is pleased to announce NetBSD 8.0, the sixteenth major release of the NetBSD operating system.\n\n\n\nThis release brings stability improvements, hundreds of bug fixes, and many new features.\n\n\n\n\nSome highlights of the NetBSD 8.0 release are:\n\n\nUSB stack rework, USB3 support added.\n\n\nIn-kernel audio mixer (audio_system(9)).\n\n\nReproducible builds (MKREPRO, see mk.conf(5)).\n\n\nFull userland debug information (MKDEBUG, see mk.conf(5)) available. While most install media do not come with them (for size reasons), the debug and xdebug sets can be downloaded and extracted as needed later. They provide full symbol information for all base system and X binaries and libraries and allow better error reporting and (userland) crash analysis.\n\n\nPaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk.\n\n\nPaX ASLR (Address Space Layout Randomization) enabled by default on: i386, amd64, evbarm, landisk, sparc64.\n\n\nPosition independent executables by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64.\n\n\nA new socket layer can(4) has been added for communication of devices on a CAN bus.\n\n\nA special pseudo interface ipsecif(4) for route-based VPNs has been added.\n\n\nParts of the network stack have been made MP-safe. The kernel option NET_MPSAFE is required to enable this.\n\n\nHardening of the network stack in general.\n\n\nVarious WAPBL (the NetBSD file system “log” option) stability and performance improvements.\n\n\nSpecific to i386 and amd64 CPUs:\n\n\nMeltdown mitigation: SVS (Separate Virtual Space), enabled by default.\n\n\nSpectreV2 mitigation: retpoline (support in gcc), used by default for kernels. Other hardware mitigations are also available.\n\n\nSpectreV4 mitigations available for Intel and AMD.\n\n\nPopSS workaround: user access to debug registers is turned off by default.\n\n\nLazy FPU saving disabled on vulnerable Intel CPUs (“eagerfpu”).\n\n\nSMAP support.\n\n\nImprovement and hardening of the memory layout: W^X, fewer writable pages, better consistency, better performance.\n\n\n(U)EFI bootloader.\n\n\nMany evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the number of boards has vastly increased.\n\n\nLots of updates to 3rd party software included:\n\n\nGCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer\n\n\nGDB 7.12\n\n\nGNU binutils 2.27\n\n\nClang/LLVM 3.8.1\n\n\nOpenSSH 7.6\n\n\nOpenSSL 1.0.2k\n\n\nmdocml 1.14.1\n\n\nacpica 20170303\n\n\nntp 4.2.8p11-o\n\n\ndhcpcd 7.0.6\n\n\nLua 5.3.4\n\n\n\n\n\n###Running FreeBSD on the ARM64 VPS from Scaleway\n\n\nI’ve been thinking about this 6 since 2017, but only yesterday signed up for an account and played around with the ARM64 offering.\nTurns out it’s pretty great! KVM boots into UEFI, there’s a local VirtIO disk attached, no NBD junk required. So we can definitely run FreeBSD.\nI managed to “depenguinate” a running instance, the notes are below. Would be great if Scaleway offered an official image instead :wink:\nFor some reason, unlike on x86 4, mounting additional volumes is not allowed 4 on ARM64 instances. So we’ll have to move the running Linux to a ramdisk using pivot_root and then we can do whatever to our one and only disk.\nSpin up an instance with Ubuntu Zesty and ssh in.\n\n\n\nPrepare the system and change the root to a tmpfs:\n\n\napt install gdisk\nmount -t tmpfs tmpfs /tmp\ncp -r /bin /sbin /etc /dev /root /home /lib /run /usr /var /tmp\nmkdir /tmp/proc /tmp/sys /tmp/oldroot\nmount /dev/vda /tmp/oldroot\nmount --make-rprivate /\npivot_root /tmp /tmp/oldroot\nfor i in dev proc sys run; do mount --move /oldroot/$i /$i; done\nsystemctl daemon-reload\nsystemctl restart sshd\n\n\n\nNow reconnect to ssh from a second terminal (note: rm the connection file if you use ControlPersist in ssh config), then exit the old session. Kill the old sshd process, restart or stop the rest of the stuff using the old disk:\n\n\npkill -f notty\nsed -ibak 's/RefuseManualStart.*$//g' /lib/systemd/system/dbus.service\nsystemctl daemon-reload\nsystemctl restart dbus\nsystemctl daemon-reexec\nsystemctl stop user@0 ntp cron systemd-logind\nsystemctl restart systemd-journald systemd-udevd\npkill agetty\npkill rsyslogd\n\n\n\nCheck that nothing is touching /oldroot:\n\n\nlsof | grep oldroot\n\n\n\nThere will probably be an old dbus-daemon, kill it.\nAnd finally, unmount the old root and overwrite the hard disk with a memstick image:\n\n\numount -R /oldroot\nwget https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/12.0/FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz\nxzcat FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz | dd if=/dev/stdin of=/dev/vda bs=1M\n\n\n\n(Look for the newest snapshot, don’t copy paste the July 19 link above if you’re reading this in the future. Actually maybe use a release instead of CURRENT…)\nNow, fix the GPT: move the secondary table to the end of the disk and resize the table.\nIt’s important to resize here, as FreeBSD does not do that and silently creates partitions that won’t persist across reboots\n\n\ngdisk /dev/vda\nx\ne\ns\n4\nw\ny\n\n\nAnd reboot. (You might actually want to hard reboot here: for some reason on the first reboot from Linux, pressing the any-key to enter the prompt in the loader hangs the console for me.)\n\nI didn’t have to go into the ESC menu and choose the local disk in the boot manager, it seems to boot from disk automatically.\n\nNow we’re in the FreeBSD EFI loader.\nFor some reason, the (recently fixed? 2) serial autodetection from EFI is not working correctly. Or something.\nSo you don’t get console output by default.\nTo fix, you have to run these commands in the boot loader command prompt:\n\nset console=comconsole,efi\nboot\n\n\nIgnore the warning about comconsole not being a valid console.\nSince there’s at least one (efi) that the loader thinks is valid, it sets the whole variable.)\n\n(UPD: shouldn’t be necessary in the next snapshot)\n\nNow it’s a regular installation process!\nWhen asked about partitioning, choose Shell, and manually add a partition and set up a root filesystem:\n\ngpart add -t freebsd-zfs -a 4k -l zroot vtbd0\nzpool create -R /mnt -O mountpoint=none -O atime=off zroot /dev/gpt/zroot\nzfs create -o canmount=off -o mountpoint=none zroot/ROOT\nzfs create -o mountpoint=/ zroot/ROOT/default\nzfs create -o mountpoint=/usr zroot/ROOT/default/usr\nzfs create -o mountpoint=/var zroot/ROOT/default/var\nzfs create -o mountpoint=/var/log zroot/ROOT/default/var/log\nzfs create -o mountpoint=/usr/home zroot/home\nzpool set bootfs=zroot/ROOT/default zroot\nexit\n\n\n(In this example, I set up ZFS with a beadm-compatible layout which allows me to use Boot Environments.)\n\nIn the post-install chroot shell, fix some configs like so:\n\necho 'zfs_load=\"YES\"' >> /boot/loader.conf\necho 'console=\"comconsole,efi\"' >> /boot/loader.conf\necho 'vfs.zfs.arc_max=\"512M\"' >> /boot/loader.conf\nsysrc zfs_enable=YES\nexit\n\n\n(Yeah, for some reason, the loader does not load zfs.ko’s dependency opensolaris.ko automatically here. idk what even. It does on my desktop and laptop.)\n\nNow you can reboot into the installed system!!\n\nHere’s how you can set up IPv6 (and root’s ssh key) auto configuration on boot:\n\nPkg bootstrap\npkg install curl\ncurl https://raw.githubusercontent.com/scaleway/image-tools/master/bases/overlay-common/usr/local/bin/scw-metadata > /usr/local/bin/scw-metadata\nchmod +x /usr/local/bin/scw-metadata\necho '#\\!/bin/sh' > /etc/rc.local\necho 'PATH=/usr/local/bin:$PATH' >> /etc/rc.local\necho 'eval $(scw-metadata)' >> /etc/rc.local\necho 'echo $SSH_PUBLIC_KEYS_0_KEY > /root/.ssh/authorized_keys' >> /etc/rc.local\necho 'chmod 0400 /root/.ssh/authorized_keys' >> /etc/rc.local\necho 'ifconfig vtnet0 inet6 $IPV6_ADDRESS/$IPV6_NETMASK' >> /etc/rc.local\necho 'route -6 add default $IPV6_GATEWAY' >> /etc/rc.local\nmkdir /run\nmkdir /root/.ssh\nsh /etc/rc.local\n\n\n\nAnd to fix incoming TCP connections, configure the DHCP client to change the broadcast address:\n\n\necho 'interface \"vtnet0\" { supersede broadcast-address 255.255.255.255; }' >> /etc/dhclient.conf\nkillall dhclient\ndhclient vtnet0\n\n\nOther random notes:\nkeep in mind that -CURRENT snapshots come with a debugging kernel by default, which limits syscall performance by a lot, you might want to build your own 2 with config GENERIC-NODEBUG\nalso disable heavy malloc debugging features by running ln -s ‘abort:false,junk:false’ /etc/malloc.conf (yes that’s storing config in a symlink)\nyou can reuse the installer’s partition for swap\n\n\n\n\n** Digital Ocean **\nhttp://do.co/bsdnow\n\n###Easy encrypted backups on OpenBSD with base tools\n\n\nToday’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).\nWhat we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy.\nDump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page.\nWhat is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup.\n\n\n\nImportant features of this backup solution:\nsave files with attributes, permissions and flags\ncan recreate a partition from a dump, restore files interactively, from a list or from its inode number (useful when you have files in lost+found)\none dump = one file\n\n\n\nMy process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.\nLet me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc… since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday.\nOne could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is.\nHistory note: dump was designed to be used with magnetic tapes.\n\n\n\nSee the article for the remainder of the article\n\n\n\n\n##News Roundup\n###Status of DFly server storage upgrades (Matt Dillon)\n\n\nLast month we did some storage upgrades, particularly of internet-facing machines for package and OS distribution. Yesterday we did a number of additional upgrades, described below. All using funds generously donated by everyone!\n\n\n\nThe main repository server received a 2TB SSD to replace the HDDs it was using before. This will improve access to a number of things maintained by this server, including the mail archives, and gives the main repo server more breathing room for repository expansion. Space was at a premium before. Now there’s plenty.\n\n\n\nMonster, the quad socket opteron which we currently use as the database builder and repository that we export to our public grok service (grok.dragonflybsd.org) received a 512G SSD to add swap space for swapcache, to help cache the grok meta-data. It now has 600GB of swapcache configured. Over the next few weeks we will also be changing the grok updates to ping-pong between the two 4TB data drives it received in the last upgrade so we can do concurrent updates and web accesses without them tripping over each other performance-wise.\n\n\n\nThe main developer box, Leaf, received a 2TB SSD and we are currently in the midst of migrating all the developer accounts in /home and /build from its old HDDs to its new SSD. This machine serves developer repos, developer web stuff, our home page and wiki, etc, so those will become snappier as well.\n\n\n\nHard drives are becoming real dinosaurs. We still have a few left from the old days but in terms of active use the only HDDs we feel we really need to keep now are the ones we use for backups and grok data, owing to the amount of storage needed for those functions.\n\n\n\nFive years ago when we received the blade server that now sits in the colo, we had a small 256G SSD for root on every blade, and everything else used HDDs. To make things operate smoothly, most of that 256G root SSD was assigned to swapcache (200G of it, in fact, in most cases). Even just 2 years ago replacing all those HDDs with SSDs, even just the ones being used to actively serve data and support developers, would have been cost prohibitive. But today it isn’t and the only HDDs we really need anywhere are for backups or certain very large bits of bulk data (aka the grok source repository and index). The way things are going, even the backup drives will probably become SSDs over the next two years.\n\n\n\n\n###iX ad spot\nOSCON 2018 Recap\n\n\n\n###zpool checkpoints\n\n\nIn March, to FreeBSD landed a very interesting feature called ‘zpool checkpoints’. Before we jump straight into the topic, let’s take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data.\n\n\n\nA very good example of how to use ZFS snapshot is during an upgrade of database schema. Let us consider a situation where we have a few scripts which change our schema. Sometimes we are unable to upgrade in one transaction (for example, when we attempt to alter a table and then update it in single transaction). If our database is on dataset, we can just snapshot it, and if something goes wrong, simply rollback the file system to its previous state.\n\n\n\nThe problem with snapshot is that it works only on a single dataset. If we added some dataset, we wouldn’t then be able to create the snapshot which would rollback that operation. The same with changing the attributes of a dataset. If we change the compression on the dataset, we cannot rollback it. We would need to change that manually.\n\n\n\nAnother interesting problem involves upgrading the whole operating system when we upgrade system with a new ZFS version. What if we start upgrading our dataset and our kernel begins to crash? (If you use FreeBSD, I doubt you will ever have had that experience but still…). If we rollback to the old kernel, there is a chance the dataset will stop working because the new kernel doesn’t know how to use the new features.\n\n\n\nZpool checkpoints is the solution to all those problems. Instead of taking a single snapshot of the dataset, we can now take a snapshot of the whole pool. That means we will not only rollback the data but also all the metadata. If we rewind to the checkpoint, all our ZFS properties will be rolled back; the upgrade will be rolledback, and even the creation/deletion of the dataset, and the snapshot, will be rolledback.\n\n\n\nZpool Checkpoint has introduced a few simple functions:\nFor a creating checkpoint:\n\n\nzpool checkpoint <pool>\n\n\nRollbacks state to checkpoint and remove the checkpoint:\n\n\nzpool import -- rewind-to-checkpoint <pool>\n\n\nMount the pool read only - this does not rollback the data:\n\n\nzpool import --read-only=on --rewind-to-checkpoint\n\n\nRemove the checkpoint\n\n\nzpool checkpoint --discard <pool> or zpool checkpoint -d <pool>\n\n\nWith this powerful feature we need to remember some safety rules:\nScrub will work only on data that isn’t in checkpool.\nYou can’t remove vdev if you have a checkpoint.\nYou can’t split mirror.\nReguid will not work either.\nCreate a checkpoint when one of the disks is removed…\n\n\n\nFor me, this feature is incredibly useful, especially when upgrading an operating system, or when I need to experiment with additional data sets. If you speak Polish, I have some additional information for you. During the first Polish BSD user group meeting, I had the opportunity to give a short talk about this feature. Here you find the video of that talk, and here is the slideshow.\n\n\n\nI would like to offer my thanks to Serapheim Dimitropoulos for developing this feature, and for being so kind in sharing with me so many of its intricacies. If you are interested in knowing more about the technical details of this feature, you should check out Serapheim’s blog, and his video about checkpoints.\n\n\n\n\n###g2k18 Reports\n\n\ng2k18 hackathon report: Ingo Schwarze on sed(1) bugfixing with Martijn van Duren, and about other small userland stuff\ng2k18 hackathon report: Kenneth Westerback on dhcpd(8) fixes, disklabel(8) refactoring and more\ng2k18 Hackathon Report: Marc Espie on ports and packages progress\ng2k18 hackathon report: Antoine Jacoutot on porting\ng2k18 hackathon report: Matthieu Herrb on font caches and xenodm\ng2k18 hackathon report: Florian Obser on rtadvd(8) -> rad(8) progress (actually, rewrite)\ng2k18 Hackathon Report: Klemens Nanni on improvements to route(8), pfctl(8), and mount(2)\ng2k18 hackathon report: Carlos Cardenas on vmm/vmd progress, LACP\ng2k18 hackathon report: Claudio Jeker on OpenBGPD developments\nPicture of the last day of the g2k18 hackathon in Ljubljana, Slovenia\n\n\n\n\n##Beastie Bits\n\n\nSomething blogged (on pkgsrcCon 2018)\nGSoC 2018 Reports: Configuration files versioning in pkgsrc, Part 1\nThere should be a global ‘awareness’ week for developers\nPolish BSD User Group – Upcoming Meeting: Aug 9th 2018\nLondon BSD User Group – Upcoming Meeting: Aug 14th 2018\nPhillip Smith’s collection of reasons why ZFS is better so that he does not have to repeat\nhimself all the time\nEuroBSDCon 2018: Sept 20-23rd in Romania – Register NOW!\nMeetBSD 2018: Oct 19-20 in Santa Clara, California. Call for Papers closes on Aug 12\n\n\n\n\nTarsnap\n\n##Feedback/Questions\n\n\nDale - L2ARC recommendations & drive age question\nTodd - ZFS & S3\nefraim - License Poem\nHenrick - Yet another ZFS question\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n\n\n","content_html":"NetBSD 8.0 available, FreeBSD on Scaleway’s ARM64 VPS, encrypted backups with OpenBSD, Dragonfly server storage upgrade, zpool checkpoints, g2k18 hackathon reports, and more.
\n
##Headlines
\n###NetBSD v8.0 Released
\n\n\nThe NetBSD Project is pleased to announce NetBSD 8.0, the sixteenth major release of the NetBSD operating system.
\n
\n\n\nThis release brings stability improvements, hundreds of bug fixes, and many new features.
\n
Some highlights of the NetBSD 8.0 release are:
\nUSB stack rework, USB3 support added.
\nIn-kernel audio mixer (audio_system(9)).
\nReproducible builds (MKREPRO, see mk.conf(5)).
\nFull userland debug information (MKDEBUG, see mk.conf(5)) available. While most install media do not come with them (for size reasons), the debug and xdebug sets can be downloaded and extracted as needed later. They provide full symbol information for all base system and X binaries and libraries and allow better error reporting and (userland) crash analysis.
\nPaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk.
\nPaX ASLR (Address Space Layout Randomization) enabled by default on: i386, amd64, evbarm, landisk, sparc64.
\nPosition independent executables by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64.
\nA new socket layer can(4) has been added for communication of devices on a CAN bus.
\nA special pseudo interface ipsecif(4) for route-based VPNs has been added.
\nParts of the network stack have been made MP-safe. The kernel option NET_MPSAFE is required to enable this.
\nHardening of the network stack in general.
\nVarious WAPBL (the NetBSD file system “log” option) stability and performance improvements.
\nSpecific to i386 and amd64 CPUs:
\nMeltdown mitigation: SVS (Separate Virtual Space), enabled by default.
\nSpectreV2 mitigation: retpoline (support in gcc), used by default for kernels. Other hardware mitigations are also available.
\nSpectreV4 mitigations available for Intel and AMD.
\nPopSS workaround: user access to debug registers is turned off by default.
\nLazy FPU saving disabled on vulnerable Intel CPUs (“eagerfpu”).
\nSMAP support.
\nImprovement and hardening of the memory layout: W^X, fewer writable pages, better consistency, better performance.
\n(U)EFI bootloader.
\nMany evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the number of boards has vastly increased.
\nLots of updates to 3rd party software included:
\nGCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer
\nGDB 7.12
\nGNU binutils 2.27
\nClang/LLVM 3.8.1
\nOpenSSH 7.6
\nOpenSSL 1.0.2k
\nmdocml 1.14.1
\nacpica 20170303
\nntp 4.2.8p11-o
\ndhcpcd 7.0.6
\nLua 5.3.4
\n###Running FreeBSD on the ARM64 VPS from Scaleway
\n\n\n\n\nI’ve been thinking about this 6 since 2017, but only yesterday signed up for an account and played around with the ARM64 offering.
\n
\nTurns out it’s pretty great! KVM boots into UEFI, there’s a local VirtIO disk attached, no NBD junk required. So we can definitely run FreeBSD.
\nI managed to “depenguinate” a running instance, the notes are below. Would be great if Scaleway offered an official image instead :wink:
\nFor some reason, unlike on x86 4, mounting additional volumes is not allowed 4 on ARM64 instances. So we’ll have to move the running Linux to a ramdisk using pivot_root and then we can do whatever to our one and only disk.
\nSpin up an instance with Ubuntu Zesty and ssh in.
apt install gdisk\nmount -t tmpfs tmpfs /tmp\ncp -r /bin /sbin /etc /dev /root /home /lib /run /usr /var /tmp\nmkdir /tmp/proc /tmp/sys /tmp/oldroot\nmount /dev/vda /tmp/oldroot\nmount --make-rprivate /\npivot_root /tmp /tmp/oldroot\nfor i in dev proc sys run; do mount --move /oldroot/$i /$i; done\nsystemctl daemon-reload\nsystemctl restart sshd\n
\n\n\n\n\nNow reconnect to ssh from a second terminal (note: rm the connection file if you use ControlPersist in ssh config), then exit the old session. Kill the old sshd process, restart or stop the rest of the stuff using the old disk:
\n
pkill -f notty\nsed -ibak 's/RefuseManualStart.*$//g' /lib/systemd/system/dbus.service\nsystemctl daemon-reload\nsystemctl restart dbus\nsystemctl daemon-reexec\nsystemctl stop user@0 ntp cron systemd-logind\nsystemctl restart systemd-journald systemd-udevd\npkill agetty\npkill rsyslogd\n
\n\n\n\n\nCheck that nothing is touching /oldroot:
\n
lsof | grep oldroot\n
\n\n\n\n\nThere will probably be an old dbus-daemon, kill it.
\n
\nAnd finally, unmount the old root and overwrite the hard disk with a memstick image:
umount -R /oldroot\nwget https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/12.0/FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz\nxzcat FreeBSD-12.0-CURRENT-arm64-aarch64-20180719-r336479-mini-memstick.img.xz | dd if=/dev/stdin of=/dev/vda bs=1M\n
\n\n\n\n\n(Look for the newest snapshot, don’t copy paste the July 19 link above if you’re reading this in the future. Actually maybe use a release instead of CURRENT…)
\n
\nNow, fix the GPT: move the secondary table to the end of the disk and resize the table.
\nIt’s important to resize here, as FreeBSD does not do that and silently creates partitions that won’t persist across reboots
gdisk /dev/vda\nx\ne\ns\n4\nw\ny\n
\n\nAnd reboot. (You might actually want to hard reboot here: for some reason on the first reboot from Linux, pressing the any-key to enter the prompt in the loader hangs the console for me.)
\n\nI didn’t have to go into the ESC menu and choose the local disk in the boot manager, it seems to boot from disk automatically.
\n\nNow we’re in the FreeBSD EFI loader.
\nFor some reason, the (recently fixed? 2) serial autodetection from EFI is not working correctly. Or something.
\nSo you don’t get console output by default.
\nTo fix, you have to run these commands in the boot loader command prompt:
set console=comconsole,efi\nboot\n
\n\nIgnore the warning about comconsole not being a valid console.
\nSince there’s at least one (efi) that the loader thinks is valid, it sets the whole variable.)
(UPD: shouldn’t be necessary in the next snapshot)
\n\nNow it’s a regular installation process!
\nWhen asked about partitioning, choose Shell, and manually add a partition and set up a root filesystem:
gpart add -t freebsd-zfs -a 4k -l zroot vtbd0\nzpool create -R /mnt -O mountpoint=none -O atime=off zroot /dev/gpt/zroot\nzfs create -o canmount=off -o mountpoint=none zroot/ROOT\nzfs create -o mountpoint=/ zroot/ROOT/default\nzfs create -o mountpoint=/usr zroot/ROOT/default/usr\nzfs create -o mountpoint=/var zroot/ROOT/default/var\nzfs create -o mountpoint=/var/log zroot/ROOT/default/var/log\nzfs create -o mountpoint=/usr/home zroot/home\nzpool set bootfs=zroot/ROOT/default zroot\nexit\n
\n\n(In this example, I set up ZFS with a beadm-compatible layout which allows me to use Boot Environments.)
\n\nIn the post-install chroot shell, fix some configs like so:
\n\necho 'zfs_load="YES"' >> /boot/loader.conf\necho 'console="comconsole,efi"' >> /boot/loader.conf\necho 'vfs.zfs.arc_max="512M"' >> /boot/loader.conf\nsysrc zfs_enable=YES\nexit\n
\n\n(Yeah, for some reason, the loader does not load zfs.ko’s dependency opensolaris.ko automatically here. idk what even. It does on my desktop and laptop.)
\n\nNow you can reboot into the installed system!!
\n\nHere’s how you can set up IPv6 (and root’s ssh key) auto configuration on boot:
\n\nPkg bootstrap\npkg install curl\ncurl https://raw.githubusercontent.com/scaleway/image-tools/master/bases/overlay-common/usr/local/bin/scw-metadata > /usr/local/bin/scw-metadata\nchmod +x /usr/local/bin/scw-metadata\necho '#\\!/bin/sh' > /etc/rc.local\necho 'PATH=/usr/local/bin:$PATH' >> /etc/rc.local\necho 'eval $(scw-metadata)' >> /etc/rc.local\necho 'echo $SSH_PUBLIC_KEYS_0_KEY > /root/.ssh/authorized_keys' >> /etc/rc.local\necho 'chmod 0400 /root/.ssh/authorized_keys' >> /etc/rc.local\necho 'ifconfig vtnet0 inet6 $IPV6_ADDRESS/$IPV6_NETMASK' >> /etc/rc.local\necho 'route -6 add default $IPV6_GATEWAY' >> /etc/rc.local\nmkdir /run\nmkdir /root/.ssh\nsh /etc/rc.local\n
\n\n\n\n\nAnd to fix incoming TCP connections, configure the DHCP client to change the broadcast address:
\n
echo 'interface "vtnet0" { supersede broadcast-address 255.255.255.255; }' >> /etc/dhclient.conf
\nkillall dhclient
\ndhclient vtnet0
** Digital Ocean **
\nhttp://do.co/bsdnow
###Easy encrypted backups on OpenBSD with base tools
\n\n\n\n\nToday’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).
\n
\nWhat we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy.
\nDump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page.
\nWhat is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup.
\n\n\nMy process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.
\n
\nLet me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc… since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday.
\nOne could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is.
\nHistory note: dump was designed to be used with magnetic tapes.
##News Roundup
\n###Status of DFly server storage upgrades (Matt Dillon)
\n\n\nLast month we did some storage upgrades, particularly of internet-facing machines for package and OS distribution. Yesterday we did a number of additional upgrades, described below. All using funds generously donated by everyone!
\n
\n\n\nThe main repository server received a 2TB SSD to replace the HDDs it was using before. This will improve access to a number of things maintained by this server, including the mail archives, and gives the main repo server more breathing room for repository expansion. Space was at a premium before. Now there’s plenty.
\n
\n\n\nMonster, the quad socket opteron which we currently use as the database builder and repository that we export to our public grok service (grok.dragonflybsd.org) received a 512G SSD to add swap space for swapcache, to help cache the grok meta-data. It now has 600GB of swapcache configured. Over the next few weeks we will also be changing the grok updates to ping-pong between the two 4TB data drives it received in the last upgrade so we can do concurrent updates and web accesses without them tripping over each other performance-wise.
\n
\n\n\nThe main developer box, Leaf, received a 2TB SSD and we are currently in the midst of migrating all the developer accounts in /home and /build from its old HDDs to its new SSD. This machine serves developer repos, developer web stuff, our home page and wiki, etc, so those will become snappier as well.
\n
\n\n\nHard drives are becoming real dinosaurs. We still have a few left from the old days but in terms of active use the only HDDs we feel we really need to keep now are the ones we use for backups and grok data, owing to the amount of storage needed for those functions.
\n
\n\n\nFive years ago when we received the blade server that now sits in the colo, we had a small 256G SSD for root on every blade, and everything else used HDDs. To make things operate smoothly, most of that 256G root SSD was assigned to swapcache (200G of it, in fact, in most cases). Even just 2 years ago replacing all those HDDs with SSDs, even just the ones being used to actively serve data and support developers, would have been cost prohibitive. But today it isn’t and the only HDDs we really need anywhere are for backups or certain very large bits of bulk data (aka the grok source repository and index). The way things are going, even the backup drives will probably become SSDs over the next two years.
\n
###iX ad spot
\nOSCON 2018 Recap
\n\n\nIn March, to FreeBSD landed a very interesting feature called ‘zpool checkpoints’. Before we jump straight into the topic, let’s take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data.
\n
\n\n\nA very good example of how to use ZFS snapshot is during an upgrade of database schema. Let us consider a situation where we have a few scripts which change our schema. Sometimes we are unable to upgrade in one transaction (for example, when we attempt to alter a table and then update it in single transaction). If our database is on dataset, we can just snapshot it, and if something goes wrong, simply rollback the file system to its previous state.
\n
\n\n\nThe problem with snapshot is that it works only on a single dataset. If we added some dataset, we wouldn’t then be able to create the snapshot which would rollback that operation. The same with changing the attributes of a dataset. If we change the compression on the dataset, we cannot rollback it. We would need to change that manually.
\n
\n\n\nAnother interesting problem involves upgrading the whole operating system when we upgrade system with a new ZFS version. What if we start upgrading our dataset and our kernel begins to crash? (If you use FreeBSD, I doubt you will ever have had that experience but still…). If we rollback to the old kernel, there is a chance the dataset will stop working because the new kernel doesn’t know how to use the new features.
\n
\n\n\nZpool checkpoints is the solution to all those problems. Instead of taking a single snapshot of the dataset, we can now take a snapshot of the whole pool. That means we will not only rollback the data but also all the metadata. If we rewind to the checkpoint, all our ZFS properties will be rolled back; the upgrade will be rolledback, and even the creation/deletion of the dataset, and the snapshot, will be rolledback.
\n
zpool checkpoint <pool>
zpool import -- rewind-to-checkpoint <pool>
zpool import --read-only=on --rewind-to-checkpoint
zpool checkpoint --discard <pool> or zpool checkpoint -d <pool>
\n\n\nFor me, this feature is incredibly useful, especially when upgrading an operating system, or when I need to experiment with additional data sets. If you speak Polish, I have some additional information for you. During the first Polish BSD user group meeting, I had the opportunity to give a short talk about this feature. Here you find the video of that talk, and here is the slideshow.
\n
\n\n\nI would like to offer my thanks to Serapheim Dimitropoulos for developing this feature, and for being so kind in sharing with me so many of its intricacies. If you are interested in knowing more about the technical details of this feature, you should check out Serapheim’s blog, and his video about checkpoints.
\n
###g2k18 Reports
\n\n##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nFreeBSD ULE vs. Linux CFS, OpenBSD on Tuxedo InfinityBook, how zfs diff reports filenames efficiently, why choose FreeBSD over Linux, PS4 double free exploit, OpenBSD’s wifi autojoin, and FreeBSD jails the hard way.
\n\nCelebrate our 256th episode with us. You can win a Mogics Power Bagel (not sponsored).
\n\nTo enter, go find the 4 episodes we did in December of 2017. In the opening, find the 4 letters in the bookshelf behind me. They spell different words in each of the 4 episodes. Send us these words in order to feedback@bsdnow.tv with the subject “bsdnow256” until August 8th, 2018 18:00 UTC and we’ll randomly draw the winner on the live show. We’ll then contact you to ship the item.
\nOnly one item to win. All decisions are final. Better luck next time.
Introduction
\nThis paper analyzes the impact on application performance of the design and implementation choices made in two widely used open-source schedulers: ULE, the default FreeBSD scheduler, and CFS, the default Linux scheduler. We compare ULE and CFS in otherwise identical circumstances. We have ported ULE to Linux, and use it to schedule all threads that are normally scheduled by CFS. We compare the performance of a large suite of applications on the modified kernel running ULE and on the standard Linux kernel running CFS. The observed performance differences are solely the result of scheduling decisions, and do not reflect differences in other subsystems between FreeBSD and Linux. There is no overall winner. On many workloads the two schedulers perform similarly, but for some workloads there are significant and even surprising differences. ULE may cause starvation, even when executing a single application with identical threads, but this starvation may actually lead to better application performance for some workloads. The more complex load balancing mechanism of CFS reacts more quickly to workload changes, but ULE achieves better load balance in the long run.
\nOperating system kernel schedulers are responsible for maintaining high utilization of hardware resources (CPU cores, memory, I/O devices) while providing fast response time to latency-sensitive applications. They have to react to workload changes, and handle large numbers of cores and threads with minimal overhead [12]. This paper provides a comparison between the default schedulers of two of the most widely deployed open-source operating systems: the Completely Fair Scheduler (CFS) used in Linux, and the ULE scheduler used in FreeBSD. Our goal is not to declare an overall winner.
\nIn fact, we find that for some workloads ULE is better and for others CFS is better. Instead, our goal is to illustrate how differences in the design and the implementation of the two schedulers are reflected in application performance under different workloads. ULE and CFS are both designed to schedule large numbers of threads on large multicore machines. Scalability considerations have led both schedulers to adopt per-core run-queues. On a context switch, a core accesses only its local run-queue to find the next thread to run. Periodically and at select times, e.g., when a thread wakes up, both ULE and CFS perform load balancing, i.e., they try to balance the amount of work waiting in the run-queues of different cores.
\nULE and CFS, however, differ greatly in their design and implementation choices. FreeBSD ULE is a simple scheduler (2,950 lines of code in FreeBSD 11.1), while Linux CFS is much more complex (17,900 lines of code in the latest LTS Linux kernel, Linux 4.9). FreeBSD run-queues are FIFO. For load balancing, FreeBSD strives to even out the number of threads per core. In Linux, a core decides which thread to run next based on prior execution time, priority, and perceived cache behavior of the threads in its runqueue. Instead of evening out the number of threads between cores, Linux strives to even out the average amount of pending work.
Performance analysis
\nWe now analyze the impact of the per-core scheduling on the performance of 37 applications. We define “performance” as follows: for database workloads and NAS applications, we compare the number of operations per second, and for the other applications we compare “execution time”. The higher the “performance”, the better a scheduler performs. Figure 5 presents the performance difference between CFS and ULE on a single core, with percentages above 0 meaning that the application executes faster with ULE than CFS.
\nOverall, the scheduler has little influence on most workloads. Indeed, most applications use threads that all perform the same work, thus both CFS and ULE endup scheduling all of the threads in a round-robin fashion. The average performance difference is 1.5%, in favor of ULE. Still, scimark is 36% slower on ULE than CFS, and apache is 40% faster on ULE than CFS. Scimark is a single-threaded Java application. It launches one compute thread, and the Java runtime executes other Java system threads in the background (for the garbage collector, I/O, etc.).
\nWhen the application is executed with ULE, the compute thread can be delayed, because Java system threads are considered interactive and get priority over the computation thread. The apache workload consists of two applications: the main server (httpd) running 100 threads, and ab, a single-threaded load injector.
\nThe performance difference between ULE and CFS is explained by different choices regarding thread preemption. In ULE, full preemption is disabled, while CFS preempts the running thread when the thread that has just been woken up has a vruntime that is much smaller than the vruntime of the currently executing thread (1ms difference in practice). In CFS, ab is preempted 2 million times during the benchmark, while it never preempted with ULE.
\nThis behavior is explained as follows: ab starts by sending 100 requests to the httpd server, and then waits for the server to answer. When ab is woken up, it checks which requests have been processed and sends new requests to the server. Since ab is single-threaded, all requests sent to the server are sent sequentially. In ULE, ab is able to send as many new requests as it has received responses. In CFS, every request sent by ab wakes up a httpd thread, which preempts ab.
Conclusion
\nScheduling threads on a multicore machine is hard. In this paper, we perform a fair comparison of the design choices of two widely used schedulers: the ULE scheduler from FreeBSD and CFS from Linux. We show that they behave differently even on simple workloads, and that no scheduler performs better than the other on all workloads.
Disclaimer:
\nI came across the Tuxedo Computers InfinityBook last year at the Open! Conference where Tuxedo had a small booth. Previously they came to my attention since they’re a member of the OSB Alliance on whose board I’m a member. Furthermore Tuxedo Computers are a sponsor of the OSBAR which I’m part of the organizational team.
OpenBSD on the Tuxedo InfinityBook
\nI’ve asked the guys over at Tuxedo Computers whether they would be interested to have some tests with *BSD done and that I could test drive one of their machines and give feedback on what works and what does not - and possibly look into it.+
Within a few weeks they shipped me a machine and last week the InfinityBook Pro 14” arrived. Awesome. Thanks already to the folks at Tuxedo Computers. The machine arrived accompanied by lot’s of swag :)
\n\nThe InfinityBook is a very nice machine and allows a wide range of configuration. The configuration that was shipped to me:
\n\nIntel Core i7-8550U
\n1x 16GB RAM 2400Mhz Crucial Ballistix Sport LT
\n250 GB Samsung 860 EVO (M.2 SATAIII)
I used a USB-stick to boot install63.fs and re-installed the machine with OpenBSD. Full dmesg.
\n\nThe installation went flawlessly, the needed intel firmware is being installed after installation automatically via fw_update(1).
\n\nOut of the box the graphics works and once installed the machine presents the login.
\n\nVideo
\nWhen X starts the display is turned off for some reason. You will need to hit fn+f12 (the key with the moon on it) then the display will go on. Aside from that little nit, X works just fine and presents one the expected resolution.
External video is working just fine as well. Either via hdmi output or via the mini displayport connector.
\n\nThe buttons for adjusting brightness (fn+f8 and fn+f9) are not working. Instead one has to use wsconsctl(8) to adjust the brightness.
\n\nNetworking
\nThe infinityBook has built-in ethernet, driven by re(4) And for the wireless interface the iwm(4) driver is being used. Both work as expected.
ACPI
\nNeither suspend nor hibernate work. Reporting of battery status is bogus as well. Some of the keyboard function keys work:
LCD on/off works (fn+f2)
\nKeyboard backlight dimming works (fn+f4)
\nVolume (fn+f5 / fn+f6) works
Sound
\nThe azalia chipset is being used for audio processing. Works as expected, volume can be controlled via buttons (fn+f5, fn+f6) or via mixerctl.
Touchpad
\nCan be controlled via wsconsctl(8).
\nSo far I must say, that the InfinityBook makes a nice machine - and I’m enjoying working with it.
iXsystems
\niXsystems - Its all NAS
As a copy on write (file)system, ZFS can use the transaction group (txg) numbers that are embedded in ZFS block pointers to efficiently find the differences between two txgs; this is used in, for example, ZFS bookmarks. However, as I noted at the end of my entry on block pointers, this doesn’t give us a filesystem level difference; instead, it essentially gives us a list of inodes (okay, dnodes) that changed.
\nIn theory, turning an inode or dnode number into the path to a file is an expensive operation; you basically have to search the entire filesystem until you find it. In practice, if you’ve ever run ‘zfs diff’, you’ve likely noticed that it runs pretty fast. Nor is this the only place that ZFS quickly turns dnode numbers into full paths, as it comes up in ‘zpool status’ reports about permanent errors. At one level, zfs diff and zpool status do this so rapidly because they ask the ZFS code in the kernel to do it for them. At another level, the question is how the kernel’s ZFS code can be so fast.
\nThe interesting and surprising answer is that ZFS cheats, in a way that makes things very fast when it works and almost always works in normal filesystems and with normal usage patterns. The cheat is that ZFS dnodes record their parent’s object number.
\nIf you’re familiar with the twists and turns of Unix filesystems, you’re now wondering how ZFS deals with hardlinks, which can cause a file to be in several directories at once and so have several parents (and then it can be removed from some of the directories). The answer is that ZFS doesn’t; a dnode only ever tracks a single parent, and ZFS accepts that this parent information can be inaccurate. I’ll quote the comment in zfs_obj_to_pobj:
\nWhen a link is removed [the file’s] parent pointer is not changed and will be invalid. There are two cases where a link is removed but the file stays around, when it goes to the delete queue and when there are additional links.
\nBefore I get into the details, I want to say that I appreciate the brute force elegance of this cheat. The practical reality is that most Unix files today don’t have extra hardlinks, and when they do most hardlinks are done in ways that won’t break ZFS’s parent stuff. The result is that ZFS has picked an efficient implementation that works almost all of the time; in my opinion, the great benefit we get from having it around are more than worth the infrequent cases where it fails or malfunctions. Both zfs diff and having filenames show up in zpool status permanent error reports are very useful (and there may be other cases where this gets used).
\nThe current details are that any time you hardlink a file to somewhere or rename it, ZFS updates the file’s parent to point to the new directory. Often this will wind up with a correct parent even after all of the dust settles; for example, a common pattern is to write a file to an initial location, hardlink it to its final destination, and then remove the initial location version. In this case, the parent will be correct and you’ll get the right name.
Not too long ago I wondered if and in what situations FreeBSD could be faster than Linux and we received a good amount of informative feedback. So far, Linux rules the desktop space and FreeBSD rules the server space.
\n\nIn the meantime, though, what exactly is FreeBSD? And at what times should you choose it over a GNU/Linux installation? Let’s tackle these questions.
\n\nFreeBSD is a free and open source derivative of BSD (Berkeley Software Distribution) with a focus on speed, stability, security, and consistency, among other features. It has been developed and maintained by a large community ever since its initial release many years ago on November 1, 1993.
\n\nBSD is the version of UNIX® that was developed at the University of California in Berkeley. And being a free and open source version, “Free” being a prefix to BSD is a no-brainer.
\n\nWhat’s FreeBSD Good For?
\n\nFreeBSD offers a plethora of advanced features and even boasts some not available in some commercial Operating Systems. It makes an excellent Internet and Intranet server thanks to its robust network services that allow it to maximize memory and work with heavy loads to deliver and maintain good response times for thousands of simultaneous user processes.
\n\nFreeBSD runs a huge number of applications with ease. At the moment, it has over 32,000 ported applications and libraries with support for desktop, server, and embedded environments. with that being said, let me also add that FreeBSD is excellent for working with advanced embedded platforms. Mail and web appliances, timer servers, routers, MIPS hardware platforms, etc. You name it!
\n\nFreeBSD is available to install in several ways and there are directions to follow for any method you want to use; be it via CD-ROM, over a network using NFS or FTP, or DVD.
\n\nFreeBSD is easy to contribute to and all you have to do is to locate the section of the FreeBSD code base to modify and carefully do a neat job. Potential contributors are also free to improve on its artwork and documentation, among other project aspects.
\n\nFreeBSD is backed by the FreeBSD Foundation, a non-profit organization that you can contribute to financially and all direct contributions are tax deductible.
\n\nFreeBSD’s license allows users to incorporate the use of proprietary software which is ideal for companies interested in generating revenues. Netflix, for example, could cite this as one of the reasons for using FreeBSD servers.
\n\nWhy Should You Choose It over Linux?
\n\nFrom what I’ve gathered about both FreeBSD and Linux, FreeBSD has a better performance on servers than Linux does. Yes, its packaged applications are configured to offer better a performance than Linux and it is usually running fewer services by default, there really isn’t a way to certify which is faster because the answer is dependent on the running hardware and applications and how the system is tuned.
\n\nFreeBSD is reportedly more secure than Linux because of the way the whole project is developed and maintained.
\n\nUnlike with Linux, the FreeBSD project is controlled by a large community of developers around the world who fall into any of these categories; core team, contributors, and committers.
\n\nFreeBSD is much easier to learn and use because there aren’t a thousand and one distros to choose from with different package managers, DEs, etc.
\n\nFreeBSD is more convenient to contribute to because it is the entire OS that is preserved and not just the kernel and a repo as is the case with Linux. You can easily access all of its versions since they are sorted by release numbers.
\n\nApart from the many documentations and guides that you can find online, FreeBSD has a single official documentation wherein you can find the solution to virtually any issue you will come across. So, you’re sure to find it resourceful.
\n\nFreeBSD has close to no software issues compared to Linux because it has Java, is capable of running Windows programs using Wine, and can run .NET programs using Mono.
\n\nFreeBSD’s ports/packages system allows you to compile software with specific configurations, thereby avoiding conflicting dependency and version issues.
\n\nBoth the FreeBSD and GNU/Linux project are always receiving updates. The platform you decide to go with is largely dependent on what you want to use it for, your technical know-how, willingness to learn new stuff, and ultimately your preference.
\nWhat is your take on the topic? For what reasons would you choose FreeBSD over Linux if you would? Let us know what you think about both platforms in the comments section below.
Introduction
\nWelcome to the 5.0x kernel exploit write-up. A few months ago, a kernel vulnerability was discovered by qwertyoruiopz and an exploit was released for BPF which involved crafting an out-of-bounds (OOB) write via use-after-free (UAF) due to the lack of proper locking. It was a fun bug, and a very trivial exploit. Sony then removed the write functionality from BPF, so that exploit was patched. However, the core issue still remained (being the lack of locking). A very similar race condition still exists in BPF past 4.55, which we will go into detail below on. The full source of the exploit can be found here.
\nThis bug is no longer accessible however past 5.05 firmware, because the BPF driver has finally been blocked from unprivileged processes - WebKit can no longer open it. Sony also introduced a new security mitigation in 5.0x firmwares to prevent the stack pointer from pointing into user space, however we’ll go more in detail on this a bit further down.
Assumptions
\nSome assumptions are made of the reader’s knowledge for the writeup. The avid reader should have a basic understanding of how memory allocators work - more specifically, how malloc() and free() allocate and deallocate memory respectively. They should also be aware that devices can be issued commands concurrently, as in, one command could be received while another one is being processed via threading. An understanding of C, x86, and exploitation basics is also very helpful, though not necessarily required.
Background
\nThis section contains some helpful information to those newer to exploitation, or are unfamiliar with device drivers, or various exploit techniques such as heap spraying and race conditions. Feel free to skip to the “A Tale of Two Free()'s” section if you’re already familiar with this material.
What Are Drivers?
\nThere are a few ways that applications can directly communicate with the operating system. One of which is system calls, which there are over 600 of in the PS4 kernel, ~500 of which are FreeBSD - the rest are Sony-implemented. Another method is through something called “Device Drivers”. Drivers are typically used to bridge the gap between software and hardware devices (usb drives, keyboard/mouse, webcams, etc) - though they can also be used just for software purposes.
\nThere are a few operations that a userland application can perform on a driver (if it has sufficient permissions) to interface with it after opening it. In some instances, one can read from it, write to it, or in some cases, issue more complex commands to it via the ioctl() system call. The handlers for these commands are implemented in kernel space - this is important, because any bugs that could be exploited in an ioctl handler can be used as a privilege escalation straight to ring0 - typically the most privileged state.
\nDrivers are often the more weaker points of an operating system for attackers, because sometimes these drivers are written by developers who don’t understand how the kernel works, or the drivers are older and thus not wise to newer attack methods.
The BPF Device Driver
\nIf we take a look around inside of WebKit’s sandbox, we’ll find a /dev directory. While this may seem like the root device driver path, it’s a lie. Many of the drivers that the PS4 has are not exposed to this directory, but rather only ones that are needed for WebKit’s operation (for the most part). For some reason though, BPF (aka. the “Berkely Packet Filter”) device is not only exposed to WebKit’s sandbox - it also has the privileges to open the device as R/W. This is very odd, because on most systems this driver is root-only (and for good reason). If you want to read more into this, refer to my previous write-up with 4.55FW.
What Are Packet Filters?
\nBelow is an excerpt from the 4.55 bpfwrite writeup.
\nSince the bug is directly in the filter system, it is important to know the basics of what packet filters are. Filters are essentially sets of pseudo-instructions that are parsed by bpf_filter() (which are ran when packets are received). While the pseudo-instruction set is fairly minimal, it allows you to do things like perform basic arithmetic operations and copy values around inside it’s buffer. Breaking down the BPF VM in it’s entirety is far beyond the scope of this write-up, just know that the code produced by it is ran in kernel mode - this is why read/write access to /dev/bpf should be privileged.
Race Conditions
\nRace conditions occur when two processes/threads try to access a shared resource at the same time without mutual exclusion. The problem was ultimately solved by introducing concepts such as the “mutex” or “lock”. The idea is when one thread/process tries to access a resource, it will first acquire a lock, access it, then unlock it once it’s finished. If another thread/process tries to access it while the other has the lock, it will wait until the other thread is finished. This works fairly well - when it’s used properly.
\nLocking is hard to get right, especially when you try to implement fine-grained locking for performance. One single instruction or line of code outside the locking window could introduce a race condition. Not all race conditions are exploitable, but some are (such as this one) - and they can give an attacker very powerful bugs to work with.
Heap Spraying
\nThe process of heap spraying is fairly simple - allocate a bunch of memory and fill it with controlled data in a loop and pray your allocation doesn’t get stolen from underneath you. It’s a very useful technique when exploiting something such as a use-after-free(), as you can use it to get controlled data into your target object’s backing memory.
\nBy extension, it’s useful to do this for a double free() as well, because once we have a stale reference, we can use a heap spray to control the data. Since the object will be marked “free” - the allocator will eventually provide us with control over this memory, even though something else is still using it. That is, unless, something else has already stolen the pointer from you and corrupts it - then you’ll likely get a system crash, and that’s no fun. This is one factor that adds to the variance of exploits, and typically, the smaller the object, the more likely this is to happen.
Follow the link to read more of the article
\nDigitalOcean
\nhttp://do.co/bsdnow
In a change which is bound to be welcomed widely, -current has gained “auto-join” for Wi-Fi networks. Peter Hessler (phessler@) has been working on this for quite some time and he wrote about it in his p2k18 hackathon report. He has committed the work from the g2k18 hackathon in Ljubljana:
\n\nCVSROOT: /cvs
\nModule name: src
\nChanges by: phessler@cvs.openbsd.org 2018/07/11 14:18:09
Modified files:
\n sbin/ifconfig : ifconfig.8 ifconfig.c
\n sys/net80211 : ieee80211_ioctl.c ieee80211_ioctl.h
\n ieee80211_node.c ieee80211_node.h
\n ieee80211_var.h
Log message:
\nIntroduce 'auto-join' to the wifi 802.11 stack.
This allows a system to remember which ESSIDs it wants to connect to, any
\nrelevant security configuration, and switch to it when the network we are
\ncurrently connected to is no longer available.
\nWorks when connecting and switching between WPA2/WPA1/WEP/clear encryptions.
example hostname.if:
\njoin home wpakey password
\njoin work wpakey mekmitasdigoat
\njoin open-lounge
\njoin cafe wpakey cafe2018
\njoin "wepnetwork" nwkey "12345"
\ndhcp
\ninet6 autoconf
\nup
OK stsp@ reyk@
\nand enthusiasm from every hackroom I've been in for the last 3 years
\nThe usage should be clear from the commit message, but basically you ‘join’ all the networks you want to auto-join as you would previously use ‘nwid’ to connect to one specific network. Then the kernel will join the network that’s actually in range and do the rest automagically for you. When you move out of range of that network you lose connectivity until you come in range of the original (where things will continue to work as you’ve been used to) or one of the other networks (where you will associate and then get a new lease).
Thanks to Peter for working on this feature - something many a Wi-Fi using OpenBSD user will be able to benefit from.
\n\nThere are many great options for managing FreeBSD Jails. iocage, warden and ez-jail aim to streamline the process and make it quick an easy to get going. But sometimes the tools built right into the OS are overlooked.
\n\nThis post goes over what is involved in creating and managing jails using only the tools built into FreeBSD.
\n\nFor this guide, I’m going to be putting my jails in /usr/local/jails.
\n\nI’ll start with a very simple, isolated jail. Then I’ll go over how to use ZFS snapshots, and lastly nullfs mounts to share the FreeBSD base files with multiple jails.
\n\nI’ll also show some examples of how to use the templating power of jail.conf to apply similar settings to all your jails.
\n\nFull Jail
\nMake a directory for the jail, or a zfs dataset if you prefer.
\nDownload the FreeBSD base files, and any other parts of FreeBSD you want. In this example I’ll include the 32 bit libraries as well.
\nUpdate your FreeBSD base install.
\nVerify your download. We’re downloading these archives over FTP after all, we should confirm that this download is valid and not tampered with. The freebsd-update IDS command verifies the installation using a PGP key which is in your base system, which was presumably installed with an ISO that you verified using the FreeBSD signed checksums. Admittedly this step is a bit of paranoia, but I think it’s prudent.
\nMake sure you jail has the right timezone and dns servers and a hostname in rc.conf.
\nEdit jail.conf with the details about your jail.
\nStart and login to your jail.
\n11 commands and a config file, but this is the most tedious way to make a jail. With a little bit of templating it can be even easier. So I’ll start by making a template. Making a template is basically the same as steps 1, 2 and 3 above, but with a different destination folder, I’ll condense them here.
Creating a template
\nCreate a template or a ZFS dataset. If you’d like to use the zfs clone method of deploying templates, you’ll need to create a zfs dataset instead of a folder.
\nUpdate your template with freebsd-update.
\nVerify your install
\nAnd that’s it, now you have a fully up to date jail template. If you’ve made this template with zfs, you can easily deploy it using zfs snapshots.
Deploying a template with ZFS snapshots
\nCreate a snapshot. My last freebsd-update to my template brought it to patch level 17, so I’ll call my snapshot p10.
\nClone the snapshot to a new jail.
\nConfigure the jail hostname.
\nAdd the jail definition to jail.conf, make sure you have the global jail settings from jail.conf listed in the fulljail example.
\nStart the jail.
\nThe downside with the zfs approach is that each jail is now a fully independent, and if you need to update your jails, you have to update them all individually. By sharing a template using nullfs mounts you can have only one copy of the base system that only needs to be updated once.
Follow the link to see the rest of the article about
\nThin jails using NullFS mounts
\nSimplifying jail.conf
\nHopefully this has helped you understand the process of how to create and manage FreeBSD jails without tools that abstract away all the details. Those tools are often quite useful, but there is always benefit in learning to do things the hard way. And in this case, the hard way doesn’t seem to be that hard after all.
Meetup in Zurich #4, July edition (July 19) – Which you likely missed, but now you know to look for the August edition!
\nThe next two BSD-PL User group meetings in Warsaw have been scheduled for July 30th and Aug 9th @ 1830 CEST – Submit your topic proposals now
\nLinux Geek Books - Humble Bundle
\nExtend loader(8) geli support to all architectures and all disk-like devices
\nUpgrading from a bootpool to a single encrypted pool – skip the gptzfsboot part, and manually update your EFI partition with loader.efi
\nThe pkgsrc 2018Q2 for Illumos is available with 18500+ binary packages
\nNetBSD ARM64 Images Available with SMP for RPi3 / NanoPi / Pine64 Boards
\nRecently released CDE 2.3.0 running on Tribblix (Illumos)
\nAn Interview With Tech & Science Fiction Author Michael W Lucas
\nA reminder : MeetBSD CFP
\nEuroBSDCon talk acceptances have gone out, and once the tutorials are confirmed, registration will open. That will likely have happened by time you see this episode, so go register! See you in Romania
\nTarsnap
Wilyarti - Adblocked on FreeBSD Continued…
\nAndrew - A Question and a Story
\nMatthew - Thanks
\nBrian - PCI-E Controller
\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
What ZFS blockpointers are, zero-day rewards offered, KDE on FreeBSD status, new FreeBSD core team, NetBSD WiFi refresh, poor man’s CI, and the power of Ctrl+T.
\n\n##Headlines
\n###What ZFS block pointers are and what’s in them
\n\n\nI’ve mentioned ZFS block pointers in the past; for example, when I wrote about some details of ZFS DVAs, I said that DVAs are embedded in block pointers. But I’ve never really looked carefully at what is in block pointers and what that means and implies for ZFS.
\n
\n\n\nThe very simple way to describe a ZFS block pointer is that it’s what ZFS uses in places where other filesystems would simply put a block number. Just like block numbers but unlike things like ZFS dnodes, a block pointer isn’t a separate on-disk entity; instead it’s an on disk data format and an in memory structure that shows up in other things. To quote from the (draft and old) ZFS on-disk specification (PDF):
\n
\n\n\nA block pointer (blkptr_t) is a 128 byte ZFS structure used to physically locate, verify, and describe blocks of data on disk.
\n
\n\n\nBlock pointers are embedded in any ZFS on disk structure that points directly to other disk blocks, both for data and metadata. For instance, the dnode for a file contains block pointers that refer to either its data blocks (if it’s small enough) or indirect blocks, as I saw in this entry. However, as I discovered when I paid attention, most things in ZFS only point to dnodes indirectly, by giving their object number (either in a ZFS filesystem or in pool-wide metadata).
\n
\n\n\nSo what’s in a block pointer itself? You can find the technical details for modern ZFS in spa.h, so I’m going to give a sort of summary. A regular block pointer contains:
\n
\n\n\nJust like basically everything else in ZFS, block pointers don’t have an explicit checksum of their contents. Instead they’re implicitly covered by the checksum of whatever they’re embedded in; the block pointers in a dnode are covered by the overall checksum of the dnode, for example. Block pointers must include a checksum for the data they point to because such data is ‘out of line’ for the containing object.
\n
\n\n\n(The block pointers in a dnode don’t necessarily point straight to data. If there’s more than a bit of data in whatever the dnode covers, the dnode’s block pointers will instead point to some level of indirect block, which itself has some number of block pointers.)
\n
\n\n\nThere is a special type of block pointer called an embedded block pointer. Embedded block pointers directly contain up to 112 bytes of data; apart from the data, they contain only the metadata fields and a logical birth txg. As with conventional block pointers, this data is implicitly covered by the checksum of the containing object.
\n
\n\n\nSince block pointers directly contain the address of things on disk (in the form of DVAs), they have to change any time that address changes, which means any time ZFS does its copy on write thing. This forces a change in whatever contains the block pointer, which in turn ripples up to another block pointer (whatever points to said containing thing), and so on until we eventually reach the Meta Object Set and the uberblock. How this works is a bit complicated, but ZFS is designed to generally make this a relatively shallow change with not many levels of things involved (as I discovered recently).
\n
\n\n\nAs far as I understand things, the logical birth txg of a block pointer is the transaction group in which the block pointer was allocated. Because of ZFS’s copy on write principle, this means that nothing underneath the block pointer has been updated or changed since that txg; if something changed, it would have been written to a new place on disk, which would have forced a change in at least one DVA and thus a ripple of updates that would update the logical birth txg.
\n
\n\n\nHowever, this doesn’t quite mean what I used to think it meant because of ZFS’s level of indirection. If you change a file by writing data to it, you will change some of the file’s block pointers, updating their logical birth txg, and you will change the file’s dnode. However, you won’t change any block pointers and thus any logical birth txgs for the filesystem directory the file is in (or anything else up the directory tree), because the directory refers to the file through its object number, not by directly pointing to its dnode. You can still use logical birth txgs to efficiently find changes from one txg to another, but you won’t necessarily get a filesystem level view of these changes; instead, as far as I can see, you will basically get a view of what object(s) in a filesystem changed (effectively, what inode numbers changed).
\n
\n\n\n(ZFS has an interesting hack to make things like ‘zfs diff’ work far more efficiently than you would expect in light of this, but that’s going to take yet another entry to cover.)
\n
###Rewards of Up to $500,000 Offered for FreeBSD, OpenBSD, NetBSD, Linux Zero-Days
\n\n\n\n\nExploit broker Zerodium is offering rewards of up to $500,000 for zero-days in UNIX-based operating systems like OpenBSD, FreeBSD, NetBSD, but also for Linux distros such as Ubuntu, CentOS, Debian, and Tails.
\n
\nThe offer, first advertised via Twitter earlier this week, is available as part of the company’s latest zero-day acquisition drive. Zerodium is known for buying zero-days and selling them to government agencies and law enforcement.
\nThe company runs a regular zero-day acquisition program through its website, but it often holds special drives with more substantial rewards when it needs zero-days of a specific category.
\n\n\nThe US-based company held a previous drive with increased rewards for Linux zero-days in February, with rewards going as high as $45,000.
\n
\nIn another zero-day acquisition drive announced on Twitter this week, the company said it was looking again for Linux zero-days, but also for exploits targeting BSD systems. This time around, rewards can go up to $500,000, for the right exploit.
\nZerodium told Bleeping Computer they’ll be aligning the temporary rewards for BSD systems with their usual payouts for Linux distros.
\nThe company’s usual payouts for Linux privilege escalation exploits can range from $10,000 to $30,000. Local privilege escalation (LPE) rewards can even reach $100,000 for “an exploit with an exceptional quality and coverage,” such as, for example, a Linux kernel exploit affecting all major distributions.
\nPayouts for Linux remote code execution (RCE) exploits can bring in from $50,000 to $500,000 depending on the targeted software/service and its market share. The highest rewards are usually awarded for LPEs and RCEs affecting CentOS and Ubuntu distros.
\n\n\nThe acquisition price of a submitted zero-day is directly tied to its requirements in terms of user interaction (no click, one click, two clicks, etc.), Zerodium said.
\n
\nOther factors include the exploit reliability, its success rate, the number of vulnerabilities chained together for the final exploit to work (more chained bugs means more chances for the exploit to break unexpectedly), and the OS configuration needed for the exploit to work (exploits are valued more if they work against default OS configs).
\n\n\n“Price difference between systems is mostly driven by market shares,” Zerodium founder Chaouki Bekrar told Bleeping Computer via email.
\n
\nAsked about the logic behind these acquisition drives that pay increased rewards, Bekrar told Bleeping Computer the following:
\n"Our aim is to always have, at any time, two or more fully functional exploits for every major software, hardware, or operating systems, meaning that from time to time we would promote a specific software/system on our social media to acquire new codes and strengthen our existing capabilities or extend them.”
\n“We may also react to customers’ requests and their operational needs,” Bekrar said.
\n\n\n\n\nSince Zerodium drew everyone’s attention to the exploit brokerage market in 2015, the market has gotten more and more crowded, but also more sleazy, with some companies being accused of selling zero-days to government agencies in countries with oppressive or dictatorial regimes, where they are often used against political oponents, journalists, and dissidents, instead of going after real criminals.
\n
\nThe latest company who broke into the zero-day brokerage market is Crowdfense, who recently launched an acquisition program with prizes of $10 million, of which it already paid $4.5 million to researchers.
Digital Ocean
\nhttp://do.co/bsdnow
\n\n\nThe KDE-FreeBSD team (a half-dozen hardy individuals, with varying backgrounds and varying degrees of involvement depending on how employment is doing) has a status message in the #kde-freebsd channel on freenode. Right now it looks like this:
\n
http://FreeBSD.kde.org | Bleeding edge \nhttp://FreeBSD.kde.org/area51.php | Released: Qt 5.10.1, KDE SC 4.14.3, KF5 5.46.0, Applications 18.04.1, Plasma-5.12.5, Kdevelop-5.2.1, Digikam-5.9.0\n
\n\n\n\n\nIt’s been a while since I wrote about KDE on FreeBSD, what with Calamares and third-party software happening as well. We’re better at keeping the IRC topic up-to-date than a lot of other sources of information (e.g. the FreeBSD quarterly reports, or the f.k.o website, which I’ll just dash off and update after writing this).
\n
\n\n\nSo we’re mostly-up-to-date, and mostly all packaged up and ready to go. Much of my day is spent in VMs packaged by other people, but it’s good to have a full KDE developer environment outside of them as well. (PS. Gotta hand it to Tomasz for the amazing application for downloading and displaying a flamingo … niche usecases FTW)
\n
##News Roundup
\n###New FreeBSD Core Team Elected
\n\n\nActive committers to the project have elected your tenth FreeBSD Core
\n
\nTeam.
\n\n\nLet’s extend our gratitude to the outgoing Core Team members:
\n
\n\n\nMatthew, after having served as the Core Team Secretary for the past
\n
\nfour years, will be stepping down from that role.
\n\n\nThe Core Team would also like to thank Dag-Erling Smørgrav for running a
\n
\nflawless election.
\n\n\nThe NetBSD Foundation is pleased to announce a summer 2018 contract with Philip Nelson (phil%NetBSD.org@localhost) to update the IEEE 802.11 stack basing the update on the FreeBSD current code. The goals of the project are:
\n
\n\n\nStatus reports will be posted to tech-net%NetBSD.org@localhost every other week
\n
\nwhile the contract is active.
iXsystems
\n\n###Poor Man’s CI - Hosted CI for BSD with shell scripting and duct tape
\n\n\n\n\nPoor Man’s CI (PMCI - Poor Man’s Continuous Integration) is a collection of scripts that taken together work as a simple CI solution that runs on Google Cloud. While there are many advanced hosted CI systems today, and many of them are free for open source projects, none of them seem to offer a solution for the BSD operating systems (FreeBSD, NetBSD, OpenBSD, etc.)
\n
\n\n\nThe architecture of Poor Man’s CI is system agnostic. However in the implementation provided in this repository the only supported systems are FreeBSD and NetBSD. Support for additional systems is possible.
\n
\n\n\nPoor Man’s CI runs on the Google Cloud. It is possible to set it up so that the service fits within the Google Cloud “Always Free” limits. In doing so the provided CI is not only hosted, but is also free! (Disclaimer: I am not affiliated with Google and do not otherwise endorse their products.)
\n
\n\n\nA CI solution listens for “commit” (or more usually “push”) events, builds the associated repository at the appropriate place in its history and reports the results. Poor Man’s CI implements this very basic CI scenario using a simple architecture, which we present in this section.
\n
Poor Man’s CI consists of the following components and their interactions:
\nController: Controls the overall process of accepting GitHub push events and starting builds. The Controller runs in the Cloud Functions environment and is implemented by the files in the controller source directory. It consists of the following components:
\nPubSub Topics:
\n\nbuilder: A builder is a Compute Engine instance that performs a build of a repository and shuts down when the build is complete. A builder is instantiated from a VM image and a startx (startup-exit) script.
\n\nBuild Logs: A Storage bucket that contains the logs of builds performed by builder instances.
\n\nLogging Sink: A Logging Sink captures builder instance terminate and delete events and posts them into the doneq.
\n\nBUGS
\n\n\n\n\nThe Builder Pool is currently implemented as a PubSub; messages in the PubSub contain the names of available builder instances. Unfortunately a PubSub retains its messages for a maximum of 7 days. It is therefore possible that messages will be discarded and that your PMCI deployment will suddenly find itself out of builder instances. If this happens you can reseed the Builder Pool by running the commands below. However this is a serious BUG that should be fixed. For a related discussion see https://tinyurl.com/ybkycuub.
\n
$ ./pmci queue_post poolq builder0
\n# ./pmci queue_post poolq builder1
\n# ... repeat for as many builders as you want
\n\n\nThe Dispatcher is implemented as a Retry Background Cloud Function. It accepts work messages from the workq and attempts to pull a free name from the poolq. If that fails it returns an error, which instructs the infrastructure to retry. Because the infrastructure does not provide any retry controls, this currently happens immediately and the Dispatcher spins unproductively. This is currently mitigated by a “sleep” (setTimeout), but the Cloud Functions system still counts the Function as running and charges it accordingly. While this fits within the “Always Free” limits, it is something that should eventually be fixed (perhaps by the PubSub team). For a related discussion see https://tinyurl.com/yb2vbwfd.
\n
\n\n\nDid you know that you can check what a process is doing by pressing CTRL+T?
\n
\nHas it happened to you before that you were waiting for something to be finished that can take a lot of time, but there is no easy way to check the status. Like a dd, cp, mv and many others. All you have to do is press CTRL+T where the process is running. This will output what’s happening and will not interrupt or mess with it in any way. This causes the operating system to output the SIGINFO signal.
\nOn FreeBSD it looks like this:
ping pingtest.com\nPING pingtest.com (5.22.149.135): 56 data bytes\n64 bytes from 5.22.149.135: icmp_seq=0 ttl=51 time=86.232 ms\n64 bytes from 5.22.149.135: icmp_seq=1 ttl=51 time=85.477 ms\n64 bytes from 5.22.149.135: icmp_seq=2 ttl=51 time=85.493 ms\n64 bytes from 5.22.149.135: icmp_seq=3 ttl=51 time=85.211 ms\n64 bytes from 5.22.149.135: icmp_seq=4 ttl=51 time=86.002 ms\nload: 1.12 cmd: ping 94371 [select] 4.70r 0.00u 0.00s 0% 2500k\n5/5 packets received (100.0%) 85.211 min / 85.683 avg / 86.232 max\n64 bytes from 5.22.149.135: icmp_seq=5 ttl=51 time=85.725 ms\n64 bytes from 5.22.149.135: icmp_seq=6 ttl=51 time=85.510 ms\n
\n\n\n\n\nAs you can see it not only outputs the name of the running command but the following parameters as well:
\n
94371 – PID\n4.70r – since when is the process running\n0.00u – user time\n0.00s – system time\n0% – CPU usage\n2500k – resident set size of the process or RSS\n``\n\n> An even better example is with the following cp command:\n\n
\n\ncp FreeBSD-11.1-RELEASE-amd64-dvd1.iso /dev/null
\nload: 0.99 cmd: cp 94412 [runnable] 1.61r 0.00u 0.39s 3% 3100k
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 15%
\nload: 0.91 cmd: cp 94412 [runnable] 2.91r 0.00u 0.80s 6% 3104k
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 32%
\nload: 0.91 cmd: cp 94412 [runnable] 4.20r 0.00u 1.23s 9% 3104k
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 49%
\nload: 0.91 cmd: cp 94412 [runnable] 5.43r 0.00u 1.64s 11% 3104k
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 64%
\nload: 1.07 cmd: cp 94412 [runnable] 6.65r 0.00u 2.05s 13% 3104k
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 79%
\nload: 1.07 cmd: cp 94412 [runnable] 7.87r 0.00u 2.43s 15% 3104k
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso -> /dev/null 95%
\n> I prcessed CTRL+T six times. Without that, all the output would have been is the first line.\n\n> Another example how the process is changing states:\n\n
\n\nwget https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-dvd1.iso
\n–2018-06-17 18:47:48– https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-dvd1.iso
\nResolving download.freebsd.org (download.freebsd.org)… 96.47.72.72, 2610:1c1:1:606c::15:0
\nConnecting to download.freebsd.org (download.freebsd.org)|96.47.72.72|:443… connected.
\nHTTP request sent, awaiting response… 200 OK
\nLength: 3348465664 (3.1G) [application/octet-stream]
\nSaving to: ‘FreeBSD-11.1-RELEASE-amd64-dvd1.iso’
FreeBSD-11.1-RELEASE-amd64-dvd1.iso 1%[> ] 41.04M 527KB/s eta 26m 49sload: 4.95 cmd: wget 10152 waiting 0.48u 0.72s
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso 1%[> ] 49.41M 659KB/s eta 25m 29sload: 12.64 cmd: wget 10152 waiting 0.55u 0.85s
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso 2%[=> ] 75.58M 6.31MB/s eta 20m 6s load: 11.71 cmd: wget 10152 running 0.73u 1.19s
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso 2%[=> ] 85.63M 6.83MB/s eta 18m 58sload: 11.71 cmd: wget 10152 waiting 0.80u 1.32s
\nFreeBSD-11.1-RELEASE-amd64-dvd1.iso 14%[==============> ] 460.23M 7.01MB/s eta 9m 0s 1
\n> The bad news is that CTRl+T doesn’t work with Linux kernel, but you can use it on MacOS/OS-X:\n\n
\n\n—> Fetching distfiles for gmp
\n—> Attempting to fetch gmp-6.1.2.tar.bz2 from https://distfiles.macports.org/gmp
\n—> Verifying checksums for gmp
\n—> Extracting gmp
\n—> Applying patches to gmp
\n—> Configuring gmp
\nload: 2.81 cmd: clang 74287 running 0.31u 0.28s
\n> PS: If I recall correctly Feld showed me CTRL+T, thank you!\n\n***\n\n\n##Beastie Bits\n+ [Half billion tries for a HAMMER2 bug](http://lists.dragonflybsd.org/pipermail/commits/2018-May/672263.html)\n+ OpenBSD with various Desktops\n + [OpenBSD 6.3 running twm window manager](https://youtu.be/v6XeC5wU2s4)\n + [OpenBSD 6.3 jwm and rox desktop](https://youtu.be/jlSK2oi7CBc)\n + [OpenBSD 6.3 cwm youtube video](https://youtu.be/mgqNyrP2CPs)\n+ [pf: Increase default state table size](https://svnweb.freebsd.org/base?view=revision&revision=336221)\n***\n\n**Tarsnap**\n\n##Feedback/Questions\n+ Ben Sims - [Full feed?](http://dpaste.com/3XVH91T#wrap)\n+ Scott - [Questions and Comments](http://dpaste.com/08P34YN#wrap)\n+ Troels - [Features of FreeBSD 11.2 that deserve a mention](http://dpaste.com/3DDPEC2#wrap)\n+ [Fred - Show Ideas](http://dpaste.com/296ZA0P#wrap)\n***\n\n- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [feedback@bsdnow.tv](mailto:feedback@bsdnow.tv)\n***\n\n***\n\niXsystems [It's all NAS](https://www.ixsystems.com/blog/its-all-nas/)\n
","summary":"What ZFS blockpointers are, zero-day rewards offered, KDE on FreeBSD status, new FreeBSD core team, NetBSD WiFi refresh, poor man’s CI, and the power of Ctrl+T.","date_published":"2018-07-18T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/ca9b19c1-e202-45d6-ac45-d0048a734c45.mp3","mime_type":"audio/mp3","size_in_bytes":48457846,"duration_in_seconds":4827}]},{"id":"http://feed.jupiter.zone/bsdnow#entry-2259","title":"Episode 254: Bare the OS | BSD Now 254","url":"https://www.bsdnow.tv/254","content_text":"Control flow integrity with HardenedBSD, fixing bufferbloat with OpenBSD’s pf, Bareos Backup Server on FreeBSD, MeetBSD CfP, crypto simplified interface, twitter gems, interesting BSD commits, and more.\n\n##Headlines\n###Silent Fanless FreeBSD Desktop/Server\n\n\nToday I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multa##Headlines\n###Cross-DSO CFI in HardenedBSD\nControl Flow Integrity, or CFI, raises the bar for attackers aiming to hijack control flow and execute arbitrary code. The llvm compiler toolchain, included and used by default in HardenedBSD 12-CURRENT/amd64, supports forward-edge CFI. Backward-edge CFI support is gained via a tangential feature called SafeStack. Cross-DSO CFI builds upon ASLR and PaX NOEXEC for effectiveness.\nHardenedBSD supports non-Cross-DSO CFI in base for 12-CURRENT/amd64 and has it enabled for a few individual ports. The term “non-Cross-DSO CFI” means that CFI is enabled for code within an application’s codebase, but not for the shared libraries it depends on. Supporting non-Cross-DSO CFI is an important initial milestone for supporting Cross-DSO CFI, or CFI applied to both shared libraries and applications.\nThis article discusses where HardenedBSD stands with regards to Cross-DSO CFI in base. We have made a lot of progress, yet we’re not even half-way there.\nBrace yourself: This article is going to be full of references to “Cross-DSO CFI.” Make a drinking game out of it. Or don’t. It’s your call. ;)\n\n\n\nUsing More llvm Toolchain Components\n\n\n\nCFI requires compiling source files with Link-Time Optimization (LTO). I remembered hearing a few years back that llvm developers were able to compile the entirety of FreeBSD’s source code with LTO. Compiling with LTO produces intermediate object files as LLVM IR bitcode instead of ELF objects.\nIn March of 2017, we started compiling all applications with LTO and non-Cross-DSO CFI. This also enabled ld.lld as the default linker in base since CFI requires lld. Commit f38b51668efcd53b8146789010611a4632cafade made the switch to ld.lld as the default linker while enabling non-Cross-DSO CFI at the same time.\nBuilding libraries in base requires applications like ar, ranlib, nm, and objdump. In FreeBSD 12-CURRENT, ar and ranlib are known as “BSD ar” and “BSD ranlib.” In fact, ar and ranlib are the same applications. One is hardlinked to another and the application changes behavior depending on arvgv[0] ending in “ranlib”. The ar, nm, and objdump used in FreeBSD do not support LLVM IR bitcode object files.\nIn preparation for Cross-DSO CFI support, commit fe4bb0104fc75c7216a6dafe2d7db0e3f5fe8257 in October 2017 saw HardenedBSD switching ar, ranlib, nm, and objdump to their respective llvm components. The llvm versions due support LLVM IR bitcode object files (surprise!) There has been some fallout in the ports tree and we’ve added LLVM_AR_UNSAFE and friends to help transition those ports that dislike llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump.\nWith ld.lld, llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump the default, HardenedBSD has effectively switched to a full llvm compiler toolchain in 12-CURRENT/amd64.\n\n\n\nBuilding Libraries With LTO\n\n\n\nThe primary 12-CURRENT development branch in HardenedBSD (hardened/current/master) only builds applications with LTO as mentioned in the secion above. My first attempt at building all static and shared libraries failed due to issues within llvm itself.\nI reported these issues to FreeBSD. Ed Maste (emaste@), Dimitry Andric (dim@), and llvm’s Rafael Espindola expertly helped address these issues. Various commits within the llvm project by Rafael fully and quickly resolved the issues brought up privately in emails.\nWith llvm fixed, I could now build nearly every library in base with LTO. I noticed, however, that if I kept non-Cross-DSO CFI and SafeStack enabled, all applications would segfault. Even simplistic applications like /bin/ls.\nDisabling both non-Cross-DSO CFI and SafeStack, but keeping LTO produced a fully functioning world! I have spent the last few months figuring out why enabling either non-Cross-DSO CFI or SafeStack caused issues. This brings us to today.\n\n\n\nThe Sanitizers in FreeBSD\n\n\n\nFreeBSD brought in all the files required for SafeStack and CFI. When compiling with SafeStack, llvm statically links a full sanitization framework into the application. FreeBSD includes a full copy of the sanitization framework in SafeStack, including the common C++ sanization namespaces. Thus, libclang_rt.safestack included code meant to be shared among all the sanitizers, not just SafeStack.\nI had naively taken a brute-force approach to setting up the libclang_rt.cfi static library. I copied the Makefile from libclang_rt.safestack and used that as a template for libclang_rt.cfi. This approach was incorrect due to breaking the One Definition Rule (ODR). Essentially, I ended up including a duplicate copy of the C++ classes and sanitizer runtime if both CFI and SafeStack were used.\nIn my Cross-DSO CFI development VM, I now have SafeStack disabled across-the-board and am only compiling in CFI. As of 26 May 2018, an LTO-ified world (libs + apps) works in my limited testing. /bin/ls does not crash anymore! The second major milestone for Cross-DSO CFI has now been reached.\n\n\n\nKnown Issues And Limitations\n\n\n\nThere are a few known issues and regressions. Note that this list of known issues essentially also constitutes a “work-in-progress” and every known issue will be fixed prior to the official launch of Cross-DSO CFI.\nIt seems llvm does not like statically compiling applications with LTO that have a mixture of C and C++ code. /sbin/devd is one of these applications. As such, when Cross-DSO CFI is enabled, devd is compiled as a Position-Independent Executable (PIE). Doing this breaks UFS systems where /usr is on a separate partition. We are currently looking into solving this issue to allow devd to be statically compiled again.\nNO_SHARED is now unset in the tools build stage (aka, bootstrap-tools, cross-tools). This is related to the static compilation issue above. Unsetting NO_SHARED for to tools build stage is only a band-aid until we can resolve static compliation with LTO.\nOne goal of our Cross-DSO CFI integration work is to be able to support the cfi-icall scheme when dlopen(3) and dlsym(3)/dlfunc(3) is used. This means the runtime linker (RTLD), must be enhanced to know and care about the CFI runtime. This enhancement is not currently implemented, but is planned.\nWhen Cross-DSO CFI is enabled, SafeStack is disabled. This is because compiling with Cross-DSO CFI brings in a second copy of the sanitizer runtime, violating the One Definition Rule (ODR). Resolving this issue should be straightforward: Unify the sanitizer runtime into a single common library that both Cross-DSO CFI and SafeStack can link against. When the installed world has Cross-DSO CFI enabled, performing a buildworld with Cross-DSO CFI disabled fails. This is somewhat related to the static compilation issue described above.\n\n\n\nCurrent Status\n\n\n\nI’ve managed to get a Cross-DSO CFI world booting on bare metal (my development laptop) and in a VM. Some applications failed to work. Curiously, Firefox still worked (which also means xorg works).\nI’m now working through the known issues list, researching and learning.\n\n\n\nFuture Work\n\n\n\nFixing pretty much everything in the “Known Issues And Limitations” section. ;P\nI need to create a static library that includes only a single copy of the common sanitizer framework code. Applications compiled with CFI or SafeStack will then only have a single copy of the framework.\nNext I will need to integrate support in the RTLD for Cross-DSO CFI. Applications with the cfi-icall scheme enabled that call functions resolved through dlsym(3) currently crash due to the lack of RTLD support. I need to make a design decision as to whether to only support adding cfi-icall whitelist entries only with dlfunc(3) or to also whitelist cfi-icall entries with the more widely used dlsym(3).\nThere’s likely more items in the “TODO” bucket that I am not currently aware of. I’m treading in uncharted territory. I have no firm ETA for any bit of this work. We may gain Cross-DSO CFI support in 2018, but it’s looking like it will be later in either 2019 or 2020.\n\n\n\nConclusion\n\n\n\nI have been working on Cross-DSO CFI support in HardenedBSD for a little over a year now. A lot of progress is being made, yet there’s still some major hurdles to overcome. This work has already helped improve llvm and I hope more commits upstream to both FreeBSD and llvm will happen.\nWe’re getting closer to being able to send out a preliminary Call For Testing (CFT). At the very least, I would like to solve the static linking issues prior to publishing the CFT. Expect it to be published before the end of 2018.\nI would like to thank Ed Maste, Dimitry Andric, and Rafael Espindola for their help, guidance, and support.\n\n\n\n\niXsystems\nFreeNAS 11.2-BETAs are starting to appear\n\n###Bareos Backup Server on FreeBSD\n\n\nEver heard about Bareos? Probably heard about Bacula. Read what is the difference here – Why Bareos forked from Bacula?\nBareos (Backup Archiving Recovery Open Sourced) is a network based open source backup solution. It is 100% open source fork of the backup project from bacula.org site. The fork is in development since late 2010 and it has a lot of new features. The source is published on github and licensed under AGPLv3 license. Bareos supports ‘Always Incremental backup which is interesting especially for users with big data. The time and network capacity consuming full backups only have to be taken once. Bareos comes with WebUI for administration tasks and restore file browser. Bareos can backup data to disk and to tape drives as well as tape libraries. It supports compression and encryption both hardware-based (like on LTO tape drives) and software-based. You can also get professional services and support from Bareos as well as Bareos subscription service that provides you access to special quality assured installation packages.\n\n\n\nI started my sysadmin job with backup system as one of the new responsibilities, so it will be like going back to the roots. As I look on the ‘backup’ market it is more and more popular – especially in cloud oriented environments – to implement various levels of protection like GOLD, SILVER and BRONZE for example. They of course have different retention times, number of backups kept, different RTO and RPO. Below is a example implementation of BRONZE level backups in Bareos. I used 3 groups of A, B and C with FULL backup starting on DAY 0 (A group), DAY 1 (B group) and DAY 2 (C group).\nThis way you still have FULL backups quite often and with 3 groups you can balance the network load. I for the days that we will not be doing FULL backups we will be doing DIFFERENTIAL backups. People often confuse them with INCREMENTAL backups. The difference is that DIFFERENTIAL backups are always against FULL backup, so its always ‘one level of combining’. INCREMENTAL ones are done against last done backup TYPE, so its possible to have 100+ levels of combining against 99 earlier INCREMENTAL backups and the 1 FULL backup. That is why I prefer DIFFERENTIAL ones here, faster recovery. That is all backups is about generally, recovery, some people/companies tend to forget that.\nThe implementation of BRONZE in these three groups is not perfect, but ‘does the job’. I also made ‘simulation’ how these group will overlap at the end/beginning of the month, here is the result.\nNot bad for my taste.\n\n\n\nToday I will show you how to install and configure Bareos Server based on FreeBSD operating system. It will be the most simplified setup with all services on single machine:\n\n\n\nbareos-dir\nbareos-sd\nbareos-webui\nbareos-fd\n\n\n\nI also assume that in order to provide storage space for the backup data itself You would mount resources from external NFS shares.\n\n\n\nTo get in touch with Bareos terminology and technology check their great Manual in HTML or PDF version depending which format You prefer for reading documentation. Also their FAQ provides a lot of needed answers.\n\n\n\nAlso this diagram may be useful for You to get some grip into the Bareos world.\n\n\n\nSystem\n\n\n\nAs every system needs to have its name we will use latin word closest to backup here – replica – for our FreeBSD system hostname. The install would be generally the same as in the FreeBSD Desktop – Part 2 – Install article. Here is our installed FreeBSD system with login prompt.\n","content_html":"Control flow integrity with HardenedBSD, fixing bufferbloat with OpenBSD’s pf, Bareos Backup Server on FreeBSD, MeetBSD CfP, crypto simplified interface, twitter gems, interesting BSD commits, and more.
\n\n##Headlines
\n###Silent Fanless FreeBSD Desktop/Server
\n\n\nToday I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multa##Headlines
\n
\n###Cross-DSO CFI in HardenedBSD
\nControl Flow Integrity, or CFI, raises the bar for attackers aiming to hijack control flow and execute arbitrary code. The llvm compiler toolchain, included and used by default in HardenedBSD 12-CURRENT/amd64, supports forward-edge CFI. Backward-edge CFI support is gained via a tangential feature called SafeStack. Cross-DSO CFI builds upon ASLR and PaX NOEXEC for effectiveness.
\nHardenedBSD supports non-Cross-DSO CFI in base for 12-CURRENT/amd64 and has it enabled for a few individual ports. The term “non-Cross-DSO CFI” means that CFI is enabled for code within an application’s codebase, but not for the shared libraries it depends on. Supporting non-Cross-DSO CFI is an important initial milestone for supporting Cross-DSO CFI, or CFI applied to both shared libraries and applications.
\nThis article discusses where HardenedBSD stands with regards to Cross-DSO CFI in base. We have made a lot of progress, yet we’re not even half-way there.
\nBrace yourself: This article is going to be full of references to “Cross-DSO CFI.” Make a drinking game out of it. Or don’t. It’s your call. ;)
\n\n\nCFI requires compiling source files with Link-Time Optimization (LTO). I remembered hearing a few years back that llvm developers were able to compile the entirety of FreeBSD’s source code with LTO. Compiling with LTO produces intermediate object files as LLVM IR bitcode instead of ELF objects.
\n
\nIn March of 2017, we started compiling all applications with LTO and non-Cross-DSO CFI. This also enabled ld.lld as the default linker in base since CFI requires lld. Commit f38b51668efcd53b8146789010611a4632cafade made the switch to ld.lld as the default linker while enabling non-Cross-DSO CFI at the same time.
\nBuilding libraries in base requires applications like ar, ranlib, nm, and objdump. In FreeBSD 12-CURRENT, ar and ranlib are known as “BSD ar” and “BSD ranlib.” In fact, ar and ranlib are the same applications. One is hardlinked to another and the application changes behavior depending on arvgv[0] ending in “ranlib”. The ar, nm, and objdump used in FreeBSD do not support LLVM IR bitcode object files.
\nIn preparation for Cross-DSO CFI support, commit fe4bb0104fc75c7216a6dafe2d7db0e3f5fe8257 in October 2017 saw HardenedBSD switching ar, ranlib, nm, and objdump to their respective llvm components. The llvm versions due support LLVM IR bitcode object files (surprise!) There has been some fallout in the ports tree and we’ve added LLVM_AR_UNSAFE and friends to help transition those ports that dislike llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump.
\nWith ld.lld, llvm-ar, llvm-ranlib, llvm-nm, and llvm-objdump the default, HardenedBSD has effectively switched to a full llvm compiler toolchain in 12-CURRENT/amd64.
\n\n\nThe primary 12-CURRENT development branch in HardenedBSD (hardened/current/master) only builds applications with LTO as mentioned in the secion above. My first attempt at building all static and shared libraries failed due to issues within llvm itself.
\n
\nI reported these issues to FreeBSD. Ed Maste (emaste@), Dimitry Andric (dim@), and llvm’s Rafael Espindola expertly helped address these issues. Various commits within the llvm project by Rafael fully and quickly resolved the issues brought up privately in emails.
\nWith llvm fixed, I could now build nearly every library in base with LTO. I noticed, however, that if I kept non-Cross-DSO CFI and SafeStack enabled, all applications would segfault. Even simplistic applications like /bin/ls.
\nDisabling both non-Cross-DSO CFI and SafeStack, but keeping LTO produced a fully functioning world! I have spent the last few months figuring out why enabling either non-Cross-DSO CFI or SafeStack caused issues. This brings us to today.
\n\n\nFreeBSD brought in all the files required for SafeStack and CFI. When compiling with SafeStack, llvm statically links a full sanitization framework into the application. FreeBSD includes a full copy of the sanitization framework in SafeStack, including the common C++ sanization namespaces. Thus, libclang_rt.safestack included code meant to be shared among all the sanitizers, not just SafeStack.
\n
\nI had naively taken a brute-force approach to setting up the libclang_rt.cfi static library. I copied the Makefile from libclang_rt.safestack and used that as a template for libclang_rt.cfi. This approach was incorrect due to breaking the One Definition Rule (ODR). Essentially, I ended up including a duplicate copy of the C++ classes and sanitizer runtime if both CFI and SafeStack were used.
\nIn my Cross-DSO CFI development VM, I now have SafeStack disabled across-the-board and am only compiling in CFI. As of 26 May 2018, an LTO-ified world (libs + apps) works in my limited testing. /bin/ls does not crash anymore! The second major milestone for Cross-DSO CFI has now been reached.
\n\n\nThere are a few known issues and regressions. Note that this list of known issues essentially also constitutes a “work-in-progress” and every known issue will be fixed prior to the official launch of Cross-DSO CFI.
\n
\nIt seems llvm does not like statically compiling applications with LTO that have a mixture of C and C++ code. /sbin/devd is one of these applications. As such, when Cross-DSO CFI is enabled, devd is compiled as a Position-Independent Executable (PIE). Doing this breaks UFS systems where /usr is on a separate partition. We are currently looking into solving this issue to allow devd to be statically compiled again.
\nNO_SHARED is now unset in the tools build stage (aka, bootstrap-tools, cross-tools). This is related to the static compilation issue above. Unsetting NO_SHARED for to tools build stage is only a band-aid until we can resolve static compliation with LTO.
\nOne goal of our Cross-DSO CFI integration work is to be able to support the cfi-icall scheme when dlopen(3) and dlsym(3)/dlfunc(3) is used. This means the runtime linker (RTLD), must be enhanced to know and care about the CFI runtime. This enhancement is not currently implemented, but is planned.
\nWhen Cross-DSO CFI is enabled, SafeStack is disabled. This is because compiling with Cross-DSO CFI brings in a second copy of the sanitizer runtime, violating the One Definition Rule (ODR). Resolving this issue should be straightforward: Unify the sanitizer runtime into a single common library that both Cross-DSO CFI and SafeStack can link against. When the installed world has Cross-DSO CFI enabled, performing a buildworld with Cross-DSO CFI disabled fails. This is somewhat related to the static compilation issue described above.
\n\n\nI’ve managed to get a Cross-DSO CFI world booting on bare metal (my development laptop) and in a VM. Some applications failed to work. Curiously, Firefox still worked (which also means xorg works).
\n
\nI’m now working through the known issues list, researching and learning.
\n\n\nFixing pretty much everything in the “Known Issues And Limitations” section. ;P
\n
\nI need to create a static library that includes only a single copy of the common sanitizer framework code. Applications compiled with CFI or SafeStack will then only have a single copy of the framework.
\nNext I will need to integrate support in the RTLD for Cross-DSO CFI. Applications with the cfi-icall scheme enabled that call functions resolved through dlsym(3) currently crash due to the lack of RTLD support. I need to make a design decision as to whether to only support adding cfi-icall whitelist entries only with dlfunc(3) or to also whitelist cfi-icall entries with the more widely used dlsym(3).
\nThere’s likely more items in the “TODO” bucket that I am not currently aware of. I’m treading in uncharted territory. I have no firm ETA for any bit of this work. We may gain Cross-DSO CFI support in 2018, but it’s looking like it will be later in either 2019 or 2020.
\n\n\nI have been working on Cross-DSO CFI support in HardenedBSD for a little over a year now. A lot of progress is being made, yet there’s still some major hurdles to overcome. This work has already helped improve llvm and I hope more commits upstream to both FreeBSD and llvm will happen.
\n
\nWe’re getting closer to being able to send out a preliminary Call For Testing (CFT). At the very least, I would like to solve the static linking issues prior to publishing the CFT. Expect it to be published before the end of 2018.
\nI would like to thank Ed Maste, Dimitry Andric, and Rafael Espindola for their help, guidance, and support.
iXsystems
\nFreeNAS 11.2-BETAs are starting to appear
###Bareos Backup Server on FreeBSD
\n\n\n\n\nEver heard about Bareos? Probably heard about Bacula. Read what is the difference here – Why Bareos forked from Bacula?
\n
\nBareos (Backup Archiving Recovery Open Sourced) is a network based open source backup solution. It is 100% open source fork of the backup project from bacula.org site. The fork is in development since late 2010 and it has a lot of new features. The source is published on github and licensed under AGPLv3 license. Bareos supports ‘Always Incremental backup which is interesting especially for users with big data. The time and network capacity consuming full backups only have to be taken once. Bareos comes with WebUI for administration tasks and restore file browser. Bareos can backup data to disk and to tape drives as well as tape libraries. It supports compression and encryption both hardware-based (like on LTO tape drives) and software-based. You can also get professional services and support from Bareos as well as Bareos subscription service that provides you access to special quality assured installation packages.
\n\n\nI started my sysadmin job with backup system as one of the new responsibilities, so it will be like going back to the roots. As I look on the ‘backup’ market it is more and more popular – especially in cloud oriented environments – to implement various levels of protection like GOLD, SILVER and BRONZE for example. They of course have different retention times, number of backups kept, different RTO and RPO. Below is a example implementation of BRONZE level backups in Bareos. I used 3 groups of A, B and C with FULL backup starting on DAY 0 (A group), DAY 1 (B group) and DAY 2 (C group).
\n
\nThis way you still have FULL backups quite often and with 3 groups you can balance the network load. I for the days that we will not be doing FULL backups we will be doing DIFFERENTIAL backups. People often confuse them with INCREMENTAL backups. The difference is that DIFFERENTIAL backups are always against FULL backup, so its always ‘one level of combining’. INCREMENTAL ones are done against last done backup TYPE, so its possible to have 100+ levels of combining against 99 earlier INCREMENTAL backups and the 1 FULL backup. That is why I prefer DIFFERENTIAL ones here, faster recovery. That is all backups is about generally, recovery, some people/companies tend to forget that.
\nThe implementation of BRONZE in these three groups is not perfect, but ‘does the job’. I also made ‘simulation’ how these group will overlap at the end/beginning of the month, here is the result.
\nNot bad for my taste.
\n\n\nToday I will show you how to install and configure Bareos Server based on FreeBSD operating system. It will be the most simplified setup with all services on single machine:
\n
\n\n\nI also assume that in order to provide storage space for the backup data itself You would mount resources from external NFS shares.
\n
\n\n\nTo get in touch with Bareos terminology and technology check their great Manual in HTML or PDF version depending which format You prefer for reading documentation. Also their FAQ provides a lot of needed answers.
\n
\n\n\nAlso this diagram may be useful for You to get some grip into the Bareos world.
\n
\n","summary":"Control flow integrity with HardenedBSD, fixing bufferbloat with OpenBSD’s pf, Bareos Backup Server on FreeBSD, MeetBSD CfP, crypto simplified interface, twitter gems, interesting BSD commits, and more.","date_published":"2018-07-12T11:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/d28fb670-e841-4f88-b58f-768d8876f126.mp3","mime_type":"audio/mp3","size_in_bytes":54900530,"duration_in_seconds":5483}]},{"id":"http://feed.jupiter.zone/bsdnow#entry-2208","title":"Episode 253: Silence of the Fans | BSD Now 253","url":"https://www.bsdnow.tv/253","content_text":"Fanless server setup with FreeBSD, NetBSD on pinebooks, another BSDCan trip report, transparent network audio, MirBSD's Korn Shell on Plan9, static site generators on OpenBSD, and more.\n\n##Headlines\n###Silent Fanless FreeBSD Desktop/Server\n\n\nToday I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multiple purposes. It also very low power solution, which also means that it will not overheat. Silent means no fans at all, even for the PSU. The format of the system should also be brought to minimum, so Mini-ITX seems best solution here.\n\n\n\nI have chosen Intel based solutions as they are very low power (6-10W), if you prefer AMD (as I often do) the closest solution in comparable price and power is Biostar A68N-2100 motherboard with AMD E1-2100 CPU and 9W power. Of course AMD has even more low power SoC solutions but finding the Mini-ITX motherboard with decent price is not an easy task. For comparison Intel has lots of such solutions below 6W whose can be nicely filtered on the ark.intel.com page. Pity that AMD does not provide such filtration for their products. I also chosen AES instructions as storage encryption (GELI on FreeBSD) today seems as obvious as HTTPS for the web pages.\n\n\n\nHere is how the system look powered up and working\n\n\n\nThis motherboard uses Intel J3355 SoC which uses 10W and has AES instructions. It has two cores at your disposal but it also supports VT-x and EPT extensions so you can even run Bhyve on it.\n\n\n\nComponents\n\n\n\nNow, an example system would look like that one below, here are the components with their prices.\n\n\n\n$49 CPU/Motherboard ASRock J3355B-ITX Mini-ITX\n$14 RAM Crucial 4 GB DDR3L 1.35V (low power)\n$17 PSU 12V 160W Pico (internal)\n$11 PSU 12V 96W FSP (external)\n$5 USB 2.0 Drive 16 GB ADATA\n$4 USB Wireless 802.11n\n$100 TOTAL\n\n\n\nThe PSU 12V 160W Pico (internal) and PSU 12V 96W FSP can be purchased on aliexpress.com or ebay.com for example, at least I got them there. Here is the 12V 160W Pico (internal) PSU and its optional additional cables to power the optional HDDs. If course its one SATA power and one MOLEX power so additional MOLEX-SATA power adapter for about 1$ would be needed. Here is the 12V 96W FSP (external) PSU without the power cord.\n\n\n\nThis gives as total silent fanless system price of about $120. Its about ONE TENTH OF THE COST of the cheapest FreeNAS hardware solution available – the FreeNAS Mini (Diskless) costs $1156 also without disks.\n\n\n\nYou can put plain FreeBSD on top of it or Solaris/Illumos distribution OmniOSce which is server oriented. You can use prebuilt NAS solution based on FreeBSD like FreeNAS, NAS4Free, ZFSguru or even Solaris/Illumos based storage with napp-it appliance.\n\n\n\n\n###An annotated look at a NetBSD Pinebook’s startup\n\n\nPinebook is an affordable 64-bit ARM notebook. Today we’re going to take a look at the kernel output at startup and talk about what hardware support is available on NetBSD.\nPhoto\nPinebook comes with 2GB RAM standard. A small amount of this is reserved by the kernel and framebuffer.\nNetBSD uses flattened device-tree (FDT) to enumerate devices on all Allwinner based SoCs. On a running system, you can inspect the device tree using the ofctl(8) utility:\nPinebook’s Allwinner A64 processor is based on the ARM Cortex-A53. It is designed to run at frequencies up to 1.2GHz.\nThe A64 is a quad core design. NetBSD’s aarch64 pmap does not yet support SMP, so three cores are disabled for now.\nThe interrupt controller is a standard ARM GIC-400 design.\nClock drivers for managing PLLs, module clock dividers, clock gating, software resets, etc. Information about the clock tree is exported in the hw.clk sysctl namespace (root access required to read these values).\n\n\n# sysctl hw.clk.sun50ia64ccu0.mmc2\nhw.clk.sun50ia64ccu0.mmc2.rate = 200000000\nhw.clk.sun50ia64ccu0.mmc2.parent = pll_periph0_2x\nhw.clk.sun50ia64ccu0.mmc2.parent_domain = sun50ia64ccu0\n\n\n\n\nDigital Ocean\nhttp://do.co/bsdnow\n\n###BSDCan 2018 Trip Report: Mark Johnston\n\n\nBSDCan is a highlight of my summers: the ability to have face-to-face conversations with fellow developers and contributors is invaluable and always helps refresh my enthusiasm for FreeBSD. While in a perfect world we would all be able to communicate effectively over the Internet, it’s often noted that locking a group of developers together in a room can be a very efficient way to make progress on projects that otherwise get strung out over time, and to me this is one of the principal functions of BSD conferences. In my case I was able to fix some kgdb bugs that had been hindering me for months; get some opinions on the design of a feature I’ve been working on for FreeBSD 12.0; hear about some ongoing usage of code that I’ve worked on; and do some pair-debugging of an issue that has been affecting another developer.\nAs is tradition, on Tuesday night I dropped off my things at the university residence where I was staying, and headed straight to the Royal Oak. This year it didn’t seem quite as packed with BSD developers, but I did meet several long-time colleagues and get a chance to catch up. In particular, I chatted with Justin Hibbits and got to hear about the bring-up of FreeBSD on POWER9, a new CPU family released by IBM. Justin was able to acquire a workstation based upon this CPU, which is a great motivator for getting FreeBSD into shape on that platform. POWER9 also has some promise in the server market, so it’s important for FreeBSD to be a viable OS choice there.\nWednesday morning saw the beginning of the two-day FreeBSD developer summit, which precedes the conference proper. Gordon Tetlow led the summit and did an excellent job organizing things and keeping to the schedule. The first presentation was by Deb Goodkin of the FreeBSD Foundation, who gave an overview of the Foundation’s role and activities. After Deb’s presentation, present members of the FreeBSD core team discussed the work they had done over the past two years, as well as open tasks that would be handed over to the new core team upon completion of the ongoing election. Finally, Marius Strobl rounded off the day’s presentations by discussing the state and responsibilities of FreeBSD’s release engineering team.\nOne side discussion of interest to me was around the notion of tightening integration with our Bugzilla instance; at moment we do not have any good means to mark a given bug as blocking a release, making it easy for bugs to slip into releases and thus lowering our overall quality. With FreeBSD 12.0 upon us, I plan to help with the triage and fixes for known regressions before the release process begins.\nAfter a break, the rest of the morning was devoted to plans for features in upcoming FreeBSD releases. This is one of my favorite discussion topics and typically takes the form of have/need/want, where developers collectively list features that they’ve developed and intend to upstream (have), features that they are missing (need), and nice-to-have features (want). This year, instead of the usual format, we listed features that are intended to ship in FreeBSD 12.0. The compiled list ended up being quite ambitious given how close we are to the beginning of the release cycle, but many individual developers (including myself) have signed up to deliver work. I’m hopeful that most, if not all of it, will make it into the release.\nAfter lunch, I attended a discussion led by Matt Ahrens and Alexander Motin on OpenZFS. Of particular interest to me were some observations made regarding the relative quantity and quality of contributions made by different “camps” of OpenZFS users (illumos, FreeBSD and ZoL), and their respective track records of upstreaming enhancements to the OpenZFS project. In part due to the high pace of changes in ZoL, the definition of “upstream” for ZFS has become murky, and of late ZFS changes have been ported directly from ZoL. Alexander discussed some known problems with ZFS on FreeBSD that have been discovered through performance testing. While I’m not familiar with ZFS internals, Alexander noted that ZFS’ write path has poor SMP scalability on FreeBSD owing to some limitations in a certain kernel API called taskqueue(9). I would like to explore this problem further and perhaps integrate a relatively new alternative interface which should perform better.\nFriday and Saturday were, of course, taken up by BSDCan talks. Friday’s keynote was by Benno Rice, who provided some history of UNIX boot systems as a precursor to some discussion of systemd and the difficulties presented by a user and developer community that actively resist change. The rest of the morning was consumed by talks and passed by quickly. First was Colin Percival’s detailed examination of where the FreeBSD kernel spends time during boot, together with an overview of some infrastructure he added to track boot times. He also provided a list of improvements that have been made since he started taking measurements, and some areas we can further improve. Colin’s existing work in this area has already brought about substantial reductions in boot time; amusingly, one of the remaining large delays comes from the keyboard driver, which contains a workaround for old PS/2 keyboards. While there seems to be general agreement that the workaround is probably no longer needed on most systems, the lingering uncertainty around this prevents us from removing the workaround. This is, sadly, a fairly typical example of an OS maintenance burden, and underscores the need to carefully document hardware bug workarounds. After this talk, I got to see some rather novel demonstrations of system tracing using dwatch, a new utility by Devin Teske, which aims to provide a user-friendly interface to DTrace. After lunch, I attended talks on netdump, a protocol for transmitting kernel dumps over a network after the system has panicked, and on a VPC implementation for FreeBSD. After the talks ended, I headed yet again to the hacker lounge and had some fruitful discussions on early microcode loading (one of my features for FreeBSD 12.0). These led me to reconsider some aspects of my approach and saved me a lot of time. Finally, I continued my debugging session from Wednesday with help from a couple of other developers.\nSaturday’s talks included a very thorough account by Li-Wen Hsu of his work in organizing a BSD conference in Taipei last year. As one of the attendees, I had felt that the conference had gone quite smoothly and was taken aback by the number of details and pitfalls that Li-Wen enumerated during his talk. This was followed by an excellent talk by Baptiste Daroussin on the difficulties one encounters when deploying FreeBSD in new environments. Baptiste offered criticisms of a number of aspects of FreeBSD, some of which hit close to home as they involved portions of the system that I’ve worked on.\nAt the conclusion of the talks, we all gathered in the main lecture hall, where Dan led a traditional and quite lively auction for charity. I managed to snag a Pine64 board and will be getting FreeBSD installed on it the first chance I get. At the end of the auction, we all headed to ByWard for dinner, concluding yet another BSDCan.\n\n\n\nThanks to Mark for sharing his experiences at this years BSDCan\n\n\n\n\n##News Roundup\n###Transparent network audio with mpd & sndiod\n\n\nLandry Breuil (landry@ when wearing his developer hat) wrote in…\n\n\nI've been a huge fan of MPD over the years to centralize my audio collection, and i've been using it with the http output to stream the music as a radio on the computer i'm currently using…\n\naudio_output {\n type \"sndio\"\n name \"Local speakers\"\n mixer_type \"software\"\n}\naudio_output {\n type \"httpd\"\n name \"HTTP stream\"\n mixer_type \"software\"\n encoder \"vorbis\"\n port \"8000\"\n format \"44100:16:2\"\n}\nthis setup worked for years, allows me to stream my home radio to $work by tunnelling the port 8000 over ssh via LocalForward, but that still has some issues:\n\na distinct timing gap between the 'local output' (ie the speakers connected to the machine where MPD is running) and the 'http output' caused by the time it takes to reencode the stream, which is ugly when you walk through the house and have a 15s delay\nsometimes mplayer as a client doesn't detect the pauses in the stream and needs to be restarted\ni need to configure/start a client on each computer and point it at the sound server url (can do via gmpc shoutcast client plugin…)\nit's not that elegant to reencode the stream, and it wastes cpu cycles\nSo the current scheme is:\n\nmpd -> http output -> network -> mplayer -> sndiod on remote machine\n|\n-> sndio output -> sndiod on soundserver\nFiddling a little bit with mpd outputs and reading the sndio output driver, i remembered sndiod has native network support… and the mpd sndio output allows you to specify a device (it uses SIO_DEVANY by default).\n\nSo in the end, it's super easy to:\n\nenable network support in sndio on the remote machine i want the audio to play by adding -L<local ip> to sndiod_flags (i have two audio devices, with an input coming from the webcam):\nsndiod_flags=\"-L10.246.200.10 -f rsnd/0 -f rsnd/1\"\nopen pf on port 11025 from the sound server ip:\npass in proto tcp from 10.246.200.1 to any port 11025\nconfigure a new output in mpd:\naudio_output {\n type \"sndio\"\n name \"sndio on renton\"\n device \"snd@10.246.200.10/0\"\n mixer_type \"software\"\n}\nand enable the new output in mpd:\n$mpc enable 2\nOutput 1 (Local speakers) is disabled\nOutput 2 (sndio on renton) is enabled\nOutput 3 (HTTP stream) is disabled\nResults in a big win: no gap anymore with the local speakers, no reencoding, no need to configure a client to play the stream, and i can still probably reproduce the same scheme over ssh from $work using a RemoteForward.\n\nmpd -> sndio output 2 -> network -> sndiod on remote machine\n|\n-> sndio output 1 -> sndiod on soundserver\nThanks ratchov@ for sndiod :)\n\n\n\n\n###MirBSD’s Korn Shell on Plan9 Jehanne\n\n\nLet start by saying that I’m not really a C programmer.\nMy last public contribution to a POSIX C program was a little improvement to the Snort’s react module back in 2008.\nSo while I know the C language well enough, I do not know anything about the subtleness of the standard library and I have little experience with POSIX semantics.\nThis is not a big issue with Plan 9, since the C library and compiler are not standard anyway, but with Jehanne (a Plan 9 derivative of my own) I want to build a simple, loosely coupled, system that can actually run useful free software ported from UNIX.\nSo I ported RedHat’s newlib to Jehanne on top of a new system library I wrote, LibPOSIX, that provides the necessary emulations. I wrote several test, checking they run the same on Linux and Jehanne, and then I begun looking for a real-world, battle tested, application to port first.\nI approached MirBSD’s Korn Shell for several reason:\n\n\n\nit is simple, powerful and well written\nit has been ported to several different operating systems\nit has few dependencies\nit’s the default shell in Android, so it’s really battle tested\n\n\n\nI was very confident. I had read the POSIX standard after all! And I had a test suite!\nI remember, I thought “Given newlib, how hard can it be?”\nThe porting begun on September 1, 2017. It was completed by tg on January 5, 2018. 125 nights later.\nTurn out, my POSIX emulation was badly broken. Not just because of the usual bugs that any piece of C can have: I didn’t understood most POSIX semantics at all!\n\n\n\n\niXsystems\n\n###Static site generator with rsync and lowdown on OpenBSD\n\n\n\nssg is a tiny POSIX-compliant shell script with few dependencies:\n\n\nlowdown(1) to parse markdown,\n\n\nrsync(1) to copy temporary files, and\n\n\nentr(1) to watch file changes.\n\n\nIt generates Markdown articles to a static website.\n\n\nIt copies the current directory to a temporary on in /tmp skipping .* and _*, renders all Markdown articles to HTML, generates RSS feed based on links from index.html, extracts the first <h1> tag from every article to generate a sitemap and use it as a page title, then wraps articles with a single HTML template, copies everything from the temporary directory to $DOCS/\n\n\n\n\nWhy not Jekyll or “$X”?\n\n\n\nssg is one hundred times smaller than Jekyll.\n\n\n\nssg and its dependencies are about 800KB combined. Compare that to 78MB of ruby with Jekyll and all the gems. So ssg can be installed in just few seconds on almost any Unix-like operating system.\nObviously, ssg is tailored for my needs, it has all features I need and only those I use.\nKeeping ssg helps you to master your Unix-shell skills: awk, grep, sed, sh, cut, tr. As a web developer you work with lots of text: code and data. So you better master these wonderful tools.\n\n\n\nPerformance\n\n\n\n100 pps. On modern computers ssg generates a hundred pages per second. Half of a time for markdown rendering and another half for wrapping articles into the template. I heard good static site generators work—twice as fast—at 200 pps, so there’s lots of performance that can be gained. ;)\n\n\n\n\n###Why does FreeBSD have virtually no (0%) desktop market share?\n\n\nBecause someone made a horrible design decision back in 1984.\n\n\n\nIn absolute fairness to those involved, it was an understandable decision, both from a research perspective, and from an economic perspective, although likely not, from a technology perspective.\n\n\n\nWhy and what.\n\n\n\nThe decision was taken because the X Window System was intended to run on cheap hardware, and, at the time, that meant reduced functionality in the end-point device with the physical display attached to it.\nAt the same time, another force was acting to also limit X displays to display services only, rather than rolling in both window management and specific widget instances for common operational paradigms.\nMostly, common operational paradigms didn’t really exist for windowing systems because they also simply didn’t exist at the time, and no one really knew how people were going to use the things, and so researchers didn’t want to commit future research to a set of hard constraints.\nSo a decision was made: separate the display services from the application at the lowest level of graphics primitives currently in use at the time.\n\n\n\nThe ramifications of this were pretty staggering.\n\n\n\nFirst, it guaranteed that all higher level graphics would live on the host side of the X protocol, instead of on the display device side of the protocol.\nDespite a good understanding of Moore’s law, and the fact that, since no X Terminals existed at the time as hardware, but were instead running as emulations on workstations that had sufficient capability, this put the higher level GUI object libraries — referred to as “widgets” — in host libraries linked into the applications.\nSecond, it guaranteed that display organization and management paradigms would also live on the host side of the protocol — assumed, in contradiction to the previous decision, to be running on the workstation.\nBut, presumably, at some point, as lightweight X Terminals became available, to migrate to a particular host computer managing compute resource login/access services.\n\n\n\nBetween these early decisions reigned chaos.\n\n\n\nSpecifically, the consequences of these decisions have been with us ever since:\nLook-and-feel are a consequence of the toolkit chosen by the application programmer, rather than a user decision which applies universally to all applications.\nYou could call this “lack of a theme”, and — although I personally despise the idea of customizing or “theming” desktops — this meant that one paradigm chosen by the user would not apply universally across all applications, no matter who had written them.\nWindow management style is a preference.\nYou could call this a more radical version of “theming” — which you will remember, I despise — but a consequence to this is that training is not universal across personnel using such systems, nor is it transferrable.\nIn other words, I can’t send someone to a class, and have them come back and use the computers in the office as a tool, with the computer itself — and the elements not specific to the application itself — disappearing into the background.\nBoth of these ultimately render an X-based system unsuitable for desktops.\nI can’t pay once for training. Training that I do pay for does not easily and naturally translate between applications. Each new version may radically alter the desktop management paradigm into unrecognizability.\n\n\n\nIs there hope for the future?\n\n\n\nWell, the Linux community has been working on something called Wayland, and it is very promising…\n…In the same way X was “very promising” in 1984, because, unfortunately, they are making exactly the same mistakes X made in 1984, rather than correcting them, now that we have 20/20 hindsight, and know what a mature widget library should look like.\nSo Wayland is screwing up again.\nBut hey, it only took us, what, 25 years to get from X in 1987 to Wayland in in 2012.\nMaybe if we try again in 2037, we can get to where Windows was in 1995.\n\n\n\n\n##Beastie Bits\n\n\nNew washing machine comes with 7 pages of open source licenses!\nBSD Jobs Site\nFreeBSD Foundation Update, May 2018\nFreeBSD Journal looking for book reviewers\nzedenv ZFS Boot Environment Manager\n\n\n\n\nTarsnap\n\n##Feedback/Questions\n\n\nWouter - Feedback\nEfraim - OS Suggestion\nkevr - Raspberry Pi2/FreeBSD/Router on a Stick\nVanja - Interview Suggestion\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n","content_html":"As every system needs to have its name we will use latin word closest to backup here – replica – for our FreeBSD system hostname. The install would be generally the same as in the FreeBSD Desktop – Part 2 – Install article. Here is our installed FreeBSD system with login prompt.
\n
Fanless server setup with FreeBSD, NetBSD on pinebooks, another BSDCan trip report, transparent network audio, MirBSD's Korn Shell on Plan9, static site generators on OpenBSD, and more.
\n\n##Headlines
\n###Silent Fanless FreeBSD Desktop/Server
\n\n\nToday I will write about silent fanless FreeBSD desktop or server computer … or NAS … or you name it, it can have multiple purposes. It also very low power solution, which also means that it will not overheat. Silent means no fans at all, even for the PSU. The format of the system should also be brought to minimum, so Mini-ITX seems best solution here.
\n
\n\n\n\n\nI have chosen Intel based solutions as they are very low power (6-10W), if you prefer AMD (as I often do) the closest solution in comparable price and power is Biostar A68N-2100 motherboard with AMD E1-2100 CPU and 9W power. Of course AMD has even more low power SoC solutions but finding the Mini-ITX motherboard with decent price is not an easy task. For comparison Intel has lots of such solutions below 6W whose can be nicely filtered on the ark.intel.com page. Pity that AMD does not provide such filtration for their products. I also chosen AES instructions as storage encryption (GELI on FreeBSD) today seems as obvious as HTTPS for the web pages.
\n
\n\n\nThis motherboard uses Intel J3355 SoC which uses 10W and has AES instructions. It has two cores at your disposal but it also supports VT-x and EPT extensions so you can even run Bhyve on it.
\n
\n\n\nNow, an example system would look like that one below, here are the components with their prices.
\n
\n\n\nThe PSU 12V 160W Pico (internal) and PSU 12V 96W FSP can be purchased on aliexpress.com or ebay.com for example, at least I got them there. Here is the 12V 160W Pico (internal) PSU and its optional additional cables to power the optional HDDs. If course its one SATA power and one MOLEX power so additional MOLEX-SATA power adapter for about 1$ would be needed. Here is the 12V 96W FSP (external) PSU without the power cord.
\n
\n\n\nThis gives as total silent fanless system price of about $120. Its about ONE TENTH OF THE COST of the cheapest FreeNAS hardware solution available – the FreeNAS Mini (Diskless) costs $1156 also without disks.
\n
\n\n\nYou can put plain FreeBSD on top of it or Solaris/Illumos distribution OmniOSce which is server oriented. You can use prebuilt NAS solution based on FreeBSD like FreeNAS, NAS4Free, ZFSguru or even Solaris/Illumos based storage with napp-it appliance.
\n
###An annotated look at a NetBSD Pinebook’s startup
\n\n# sysctl hw.clk.sun50ia64ccu0.mmc2\nhw.clk.sun50ia64ccu0.mmc2.rate = 200000000\nhw.clk.sun50ia64ccu0.mmc2.parent = pll_periph0_2x\nhw.clk.sun50ia64ccu0.mmc2.parent_domain = sun50ia64ccu0\n
\n\nDigital Ocean
\nhttp://do.co/bsdnow
###BSDCan 2018 Trip Report: Mark Johnston
\n\n\n\n\nBSDCan is a highlight of my summers: the ability to have face-to-face conversations with fellow developers and contributors is invaluable and always helps refresh my enthusiasm for FreeBSD. While in a perfect world we would all be able to communicate effectively over the Internet, it’s often noted that locking a group of developers together in a room can be a very efficient way to make progress on projects that otherwise get strung out over time, and to me this is one of the principal functions of BSD conferences. In my case I was able to fix some kgdb bugs that had been hindering me for months; get some opinions on the design of a feature I’ve been working on for FreeBSD 12.0; hear about some ongoing usage of code that I’ve worked on; and do some pair-debugging of an issue that has been affecting another developer.
\n
\nAs is tradition, on Tuesday night I dropped off my things at the university residence where I was staying, and headed straight to the Royal Oak. This year it didn’t seem quite as packed with BSD developers, but I did meet several long-time colleagues and get a chance to catch up. In particular, I chatted with Justin Hibbits and got to hear about the bring-up of FreeBSD on POWER9, a new CPU family released by IBM. Justin was able to acquire a workstation based upon this CPU, which is a great motivator for getting FreeBSD into shape on that platform. POWER9 also has some promise in the server market, so it’s important for FreeBSD to be a viable OS choice there.
\nWednesday morning saw the beginning of the two-day FreeBSD developer summit, which precedes the conference proper. Gordon Tetlow led the summit and did an excellent job organizing things and keeping to the schedule. The first presentation was by Deb Goodkin of the FreeBSD Foundation, who gave an overview of the Foundation’s role and activities. After Deb’s presentation, present members of the FreeBSD core team discussed the work they had done over the past two years, as well as open tasks that would be handed over to the new core team upon completion of the ongoing election. Finally, Marius Strobl rounded off the day’s presentations by discussing the state and responsibilities of FreeBSD’s release engineering team.
\nOne side discussion of interest to me was around the notion of tightening integration with our Bugzilla instance; at moment we do not have any good means to mark a given bug as blocking a release, making it easy for bugs to slip into releases and thus lowering our overall quality. With FreeBSD 12.0 upon us, I plan to help with the triage and fixes for known regressions before the release process begins.
\nAfter a break, the rest of the morning was devoted to plans for features in upcoming FreeBSD releases. This is one of my favorite discussion topics and typically takes the form of have/need/want, where developers collectively list features that they’ve developed and intend to upstream (have), features that they are missing (need), and nice-to-have features (want). This year, instead of the usual format, we listed features that are intended to ship in FreeBSD 12.0. The compiled list ended up being quite ambitious given how close we are to the beginning of the release cycle, but many individual developers (including myself) have signed up to deliver work. I’m hopeful that most, if not all of it, will make it into the release.
\nAfter lunch, I attended a discussion led by Matt Ahrens and Alexander Motin on OpenZFS. Of particular interest to me were some observations made regarding the relative quantity and quality of contributions made by different “camps” of OpenZFS users (illumos, FreeBSD and ZoL), and their respective track records of upstreaming enhancements to the OpenZFS project. In part due to the high pace of changes in ZoL, the definition of “upstream” for ZFS has become murky, and of late ZFS changes have been ported directly from ZoL. Alexander discussed some known problems with ZFS on FreeBSD that have been discovered through performance testing. While I’m not familiar with ZFS internals, Alexander noted that ZFS’ write path has poor SMP scalability on FreeBSD owing to some limitations in a certain kernel API called taskqueue(9). I would like to explore this problem further and perhaps integrate a relatively new alternative interface which should perform better.
\nFriday and Saturday were, of course, taken up by BSDCan talks. Friday’s keynote was by Benno Rice, who provided some history of UNIX boot systems as a precursor to some discussion of systemd and the difficulties presented by a user and developer community that actively resist change. The rest of the morning was consumed by talks and passed by quickly. First was Colin Percival’s detailed examination of where the FreeBSD kernel spends time during boot, together with an overview of some infrastructure he added to track boot times. He also provided a list of improvements that have been made since he started taking measurements, and some areas we can further improve. Colin’s existing work in this area has already brought about substantial reductions in boot time; amusingly, one of the remaining large delays comes from the keyboard driver, which contains a workaround for old PS/2 keyboards. While there seems to be general agreement that the workaround is probably no longer needed on most systems, the lingering uncertainty around this prevents us from removing the workaround. This is, sadly, a fairly typical example of an OS maintenance burden, and underscores the need to carefully document hardware bug workarounds. After this talk, I got to see some rather novel demonstrations of system tracing using dwatch, a new utility by Devin Teske, which aims to provide a user-friendly interface to DTrace. After lunch, I attended talks on netdump, a protocol for transmitting kernel dumps over a network after the system has panicked, and on a VPC implementation for FreeBSD. After the talks ended, I headed yet again to the hacker lounge and had some fruitful discussions on early microcode loading (one of my features for FreeBSD 12.0). These led me to reconsider some aspects of my approach and saved me a lot of time. Finally, I continued my debugging session from Wednesday with help from a couple of other developers.
\nSaturday’s talks included a very thorough account by Li-Wen Hsu of his work in organizing a BSD conference in Taipei last year. As one of the attendees, I had felt that the conference had gone quite smoothly and was taken aback by the number of details and pitfalls that Li-Wen enumerated during his talk. This was followed by an excellent talk by Baptiste Daroussin on the difficulties one encounters when deploying FreeBSD in new environments. Baptiste offered criticisms of a number of aspects of FreeBSD, some of which hit close to home as they involved portions of the system that I’ve worked on.
\nAt the conclusion of the talks, we all gathered in the main lecture hall, where Dan led a traditional and quite lively auction for charity. I managed to snag a Pine64 board and will be getting FreeBSD installed on it the first chance I get. At the end of the auction, we all headed to ByWard for dinner, concluding yet another BSDCan.
##News Roundup
\n###Transparent network audio with mpd & sndiod
\n\n\nLandry Breuil (landry@ when wearing his developer hat) wrote in…
\n
I've been a huge fan of MPD over the years to centralize my audio collection, and i've been using it with the http output to stream the music as a radio on the computer i'm currently using…\n\naudio_output {\n type "sndio"\n name "Local speakers"\n mixer_type "software"\n}\naudio_output {\n type "httpd"\n name "HTTP stream"\n mixer_type "software"\n encoder "vorbis"\n port "8000"\n format "44100:16:2"\n}\nthis setup worked for years, allows me to stream my home radio to $work by tunnelling the port 8000 over ssh via LocalForward, but that still has some issues:\n\na distinct timing gap between the 'local output' (ie the speakers connected to the machine where MPD is running) and the 'http output' caused by the time it takes to reencode the stream, which is ugly when you walk through the house and have a 15s delay\nsometimes mplayer as a client doesn't detect the pauses in the stream and needs to be restarted\ni need to configure/start a client on each computer and point it at the sound server url (can do via gmpc shoutcast client plugin…)\nit's not that elegant to reencode the stream, and it wastes cpu cycles\nSo the current scheme is:\n\nmpd -> http output -> network -> mplayer -> sndiod on remote machine\n|\n-> sndio output -> sndiod on soundserver\nFiddling a little bit with mpd outputs and reading the sndio output driver, i remembered sndiod has native network support… and the mpd sndio output allows you to specify a device (it uses SIO_DEVANY by default).\n\nSo in the end, it's super easy to:\n\nenable network support in sndio on the remote machine i want the audio to play by adding -L<local ip> to sndiod_flags (i have two audio devices, with an input coming from the webcam):\nsndiod_flags="-L10.246.200.10 -f rsnd/0 -f rsnd/1"\nopen pf on port 11025 from the sound server ip:\npass in proto tcp from 10.246.200.1 to any port 11025\nconfigure a new output in mpd:\naudio_output {\n type "sndio"\n name "sndio on renton"\n device "snd@10.246.200.10/0"\n mixer_type "software"\n}\nand enable the new output in mpd:\n$mpc enable 2\nOutput 1 (Local speakers) is disabled\nOutput 2 (sndio on renton) is enabled\nOutput 3 (HTTP stream) is disabled\nResults in a big win: no gap anymore with the local speakers, no reencoding, no need to configure a client to play the stream, and i can still probably reproduce the same scheme over ssh from $work using a RemoteForward.\n\nmpd -> sndio output 2 -> network -> sndiod on remote machine\n|\n-> sndio output 1 -> sndiod on soundserver\nThanks ratchov@ for sndiod :)\n
\n\n###MirBSD’s Korn Shell on Plan9 Jehanne
\n\n\n\n\nLet start by saying that I’m not really a C programmer.
\n
\nMy last public contribution to a POSIX C program was a little improvement to the Snort’s react module back in 2008.
\nSo while I know the C language well enough, I do not know anything about the subtleness of the standard library and I have little experience with POSIX semantics.
\nThis is not a big issue with Plan 9, since the C library and compiler are not standard anyway, but with Jehanne (a Plan 9 derivative of my own) I want to build a simple, loosely coupled, system that can actually run useful free software ported from UNIX.
\nSo I ported RedHat’s newlib to Jehanne on top of a new system library I wrote, LibPOSIX, that provides the necessary emulations. I wrote several test, checking they run the same on Linux and Jehanne, and then I begun looking for a real-world, battle tested, application to port first.
\nI approached MirBSD’s Korn Shell for several reason:
\n\n\nI was very confident. I had read the POSIX standard after all! And I had a test suite!
\n
\nI remember, I thought “Given newlib, how hard can it be?”
\nThe porting begun on September 1, 2017. It was completed by tg on January 5, 2018. 125 nights later.
\nTurn out, my POSIX emulation was badly broken. Not just because of the usual bugs that any piece of C can have: I didn’t understood most POSIX semantics at all!
iXsystems
\n\n###Static site generator with rsync and lowdown on OpenBSD
\n\nssg is a tiny POSIX-compliant shell script with few dependencies:
\nlowdown(1) to parse markdown,
\nrsync(1) to copy temporary files, and
\nentr(1) to watch file changes.
\nIt generates Markdown articles to a static website.
\nIt copies the current directory to a temporary on in /tmp skipping .* and _*, renders all Markdown articles to HTML, generates RSS feed based on links from index.html, extracts the first <h1> tag from every article to generate a sitemap and use it as a page title, then wraps articles with a single HTML template, copies everything from the temporary directory to $DOCS/
\n\n\n\nWhy not Jekyll or “$X”?
\n
\n\n\nssg and its dependencies are about 800KB combined. Compare that to 78MB of ruby with Jekyll and all the gems. So ssg can be installed in just few seconds on almost any Unix-like operating system.
\n
\nObviously, ssg is tailored for my needs, it has all features I need and only those I use.
\nKeeping ssg helps you to master your Unix-shell skills: awk, grep, sed, sh, cut, tr. As a web developer you work with lots of text: code and data. So you better master these wonderful tools.
\n\n\n100 pps. On modern computers ssg generates a hundred pages per second. Half of a time for markdown rendering and another half for wrapping articles into the template. I heard good static site generators work—twice as fast—at 200 pps, so there’s lots of performance that can be gained. ;)
\n
###Why does FreeBSD have virtually no (0%) desktop market share?
\n\n\n\n\nIn absolute fairness to those involved, it was an understandable decision, both from a research perspective, and from an economic perspective, although likely not, from a technology perspective.
\n
\n\n\nThe decision was taken because the X Window System was intended to run on cheap hardware, and, at the time, that meant reduced functionality in the end-point device with the physical display attached to it.
\n
\nAt the same time, another force was acting to also limit X displays to display services only, rather than rolling in both window management and specific widget instances for common operational paradigms.
\nMostly, common operational paradigms didn’t really exist for windowing systems because they also simply didn’t exist at the time, and no one really knew how people were going to use the things, and so researchers didn’t want to commit future research to a set of hard constraints.
\nSo a decision was made: separate the display services from the application at the lowest level of graphics primitives currently in use at the time.
\n\n\nFirst, it guaranteed that all higher level graphics would live on the host side of the X protocol, instead of on the display device side of the protocol.
\n
\nDespite a good understanding of Moore’s law, and the fact that, since no X Terminals existed at the time as hardware, but were instead running as emulations on workstations that had sufficient capability, this put the higher level GUI object libraries — referred to as “widgets” — in host libraries linked into the applications.
\nSecond, it guaranteed that display organization and management paradigms would also live on the host side of the protocol — assumed, in contradiction to the previous decision, to be running on the workstation.
\nBut, presumably, at some point, as lightweight X Terminals became available, to migrate to a particular host computer managing compute resource login/access services.
\n\n\nSpecifically, the consequences of these decisions have been with us ever since:
\n
\nLook-and-feel are a consequence of the toolkit chosen by the application programmer, rather than a user decision which applies universally to all applications.
\nYou could call this “lack of a theme”, and — although I personally despise the idea of customizing or “theming” desktops — this meant that one paradigm chosen by the user would not apply universally across all applications, no matter who had written them.
\nWindow management style is a preference.
\nYou could call this a more radical version of “theming” — which you will remember, I despise — but a consequence to this is that training is not universal across personnel using such systems, nor is it transferrable.
\nIn other words, I can’t send someone to a class, and have them come back and use the computers in the office as a tool, with the computer itself — and the elements not specific to the application itself — disappearing into the background.
\nBoth of these ultimately render an X-based system unsuitable for desktops.
\nI can’t pay once for training. Training that I do pay for does not easily and naturally translate between applications. Each new version may radically alter the desktop management paradigm into unrecognizability.
\n\n\nWell, the Linux community has been working on something called Wayland, and it is very promising…
\n
\n…In the same way X was “very promising” in 1984, because, unfortunately, they are making exactly the same mistakes X made in 1984, rather than correcting them, now that we have 20/20 hindsight, and know what a mature widget library should look like.
\nSo Wayland is screwing up again.
\nBut hey, it only took us, what, 25 years to get from X in 1987 to Wayland in in 2012.
\nMaybe if we try again in 2037, we can get to where Windows was in 1995.
##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nFreeBSD 11.2 has been released, setting up an MTA behind Tor, running pfsense on DigitalOcean, one year of C, using OpenBGPD to announce VM networks, the power to serve, and a BSDCan trip report.
\n\n##Headlines
\n###FreeBSD 11.2-RELEASE Available
\n\n\nOpenSSH has been updated to version 7.5p1.
\n
\nOpenSSL has been updated to version 1.0.2o.
\nThe clang, llvm, lldb and compiler-rt utilities have been updated to version 6.0.0.
\nThe libarchive(3) library has been updated to version 3.3.2.
\nThe libxo(3) library has been updated to version 0.9.0.
\nMajor Device driver updates to:
\n\n\nNew drivers:
\n
\n+ drm-next-kmod driver supporting integrated Intel graphics with the i915 driver.
\n\n\nThe newsyslog(8) utility has been updated to support RFC5424-compliant messages when rotating system logs
\n
\nThe diskinfo(8) utility has been updated to include two new flags, -s which displays the disk identity (usually the serial number), and -p which displays the physical path to the disk in a storage controller.
\nThe top(1) utility has been updated to allow filtering on multiple user names when the -U flag is used
\nThe umount(8) utility has been updated to include a new flag, -N, which is used to forcefully unmount an NFS mounted filesystem.
\nThe ps(1) utility has been updated to display if a process is running with capsicum(4) capability mode, indicated by the flag ‘C’
\nThe service(8) utility has been updated to include a new flag, -j, which is used to interact with services running within a jail(8). The argument to -j can be either the name or numeric jail ID
\nThe mlx5tool(8) utility has been added, which is used to manage Connect-X 4 and Connect-X 5 devices supported by mlx5io(4).
\nThe ifconfig(8) utility has been updated to include a random option, which when used with the ether option, generates a random MAC address for an interface.
\nThe dwatch(1) utility has been introduced
\nThe efibootmgr(8) utility has been added, which is used to manipulate the EFI boot manager.
\nThe etdump(1) utility has been added, which is used to view El Torito boot catalog information.
\nThe linux(4) ABI compatibility layer has been updated to include support for musl consumers.
\nThe fdescfs(5) filesystem has been updated to support Linux®-specific fd(4) /dev/fd and /proc/self/fd behavior
\nSupport for virtio_console(4) has been added to bhyve(4).
\nThe length of GELI passphrases entered when booting a system with encrypted disks is now hidden by default. See the configuration options in geli(8) to restore the previous behavior.
###Setting up an MTA Behind Tor
\n\n\n\n\nThis article will document how to set up OpenSMTPD behind a fully Tor-ified network. Given that Tor’s DNS resolver code does not support MX record lookups, care must be taken for setting up an MTA behind a fully Tor-ified network. OpenSMTPD was chosen because it was easy to modify to force it to fall back to A/AAAA lookups when MX lookups failed with a DNS result code of NOTIMP (4).
\n
\n\n\nNote that as of 08 May 2018, the OpenSMTPD project is planning a configuration file language change. The proposed change has not landed. Once it does, this article will be updated to reflect both the old language and new.
\n
\n\n\nThe reason to use an MTA behing a fully Tor-ified network is to be able to support email behind the .onion TLD. This setup will only allow us to send and receive email to and from the .onion TLD.
\n
Requirements:
\nA fully Tor-ified network
\nHardenedBSD as the operating system
\nA server (or VM) running HardenedBSD behind the fully Tor-ified network.
\n/usr/ports is empty
\nOr is already pre-populated with the HardenedBSD Ports tree
\nWhy use HardenedBSD? We get all the features of FreeBSD (ZFS, DTrace, bhyve, and jails) with enhanced security through exploit mitigations and system hardening. Tor has a very unique threat landscape and using a hardened ecosystem is crucial to mitigating risks and threats.
\n\n\n\nAlso note that this article reflects how I’ve set up my MTA. I’ve included configuration files verbatim. You will need to replace the text that refers to my .onion domain with yours.
\n
\n\n\nOn 08 May 2018, HardenedBSD’s version of OpenSMTPD just gained support for running an MTA behind Tor. The package repositories do not yet contain the patch, so we will compile OpenSMTPD from ports.
\n
iXsystems
\nhttps://www.forbes.com/sites/forbestechcouncil/2018/06/21/strings-attached-knowing-when-and-when-not-to-accept-vc-funding/#30f9f18f46ec
\nhttps://www.ixsystems.com/blog/self-2018-recap/
###Running pfSense on a Digital Ocean Droplet
\n\n\n\n\nI love pfSense (and opnSense, no discrimination here). I use it for just about anything, from homelab to large scale deployments and I’ll give out on any fancy <enter brand name fw appliance here> for a pfSense setup on a decent hardware.
\n
\n\n\nI also love DigitalOcean, if you ever used them, you know why, if you never did, head over and try, you’ll understand why.
\n
\n<shameless plug: head over to JupiterBroadcasting.com, the best technology content out there, they have coupon codes to get you started with DO>.
\n\n\nUnfortunately, while DO offers tremendous amount of useful distros and applications, pfSense isn’t one of them. But, where there’s a will, there’s a way, and here’s how to get pfSense up and running on DO so you can have it as the gatekeeper to your kingdom.
\n
\n\n\nStart by creating a FreeBSD droplet, choose your droplet size (for modest setups, I find the 5$ to be quite awesome):
\n
\n\n\nThere are many useful things you can do with pfSense on your droplet, from OpenVPN, squid, firewalling, fancy routing, url filtering, dns black listing and much much more.
\n
\n\n\nYou have two ways to initiate the initial setup wizard of the web-configurator:
\n
\nSpin up another droplet, log into it and browse your way to the INTERNAL ip address of the internal NIC you’ve set up. This is the long and tedious way, but it’s also somewhat safer as it eliminates the small window of risk the second method poses.
\nor
\nOnce your WAN address is all setup, your pfSense is ready to accept https connection to start the initial web-configurator setup.
\nThing is, there’s a default, well known set of credential to this initial wizard (admin:pfsense), so, there is a slight window of opportunity that someone can swoop in (assuming they know you’ve installed pfsense + your wan IP address + the exact time window between setting up the WAN interface and completing the wizard) and do <enter scary thing here>.
\n\n\nI leave it up to you which of the path you’d like to go, either way, once you’re done with the web-configurator wizard, you’ll have a shiny new pfSense installation at your disposal running on your favorite VPS.
\n
\n\n\nHopefully this was helpful for someone, I hope to get a similar post soon detailing how to get FreeNAS up and running on DO.
\n
\nMany thanks to Tubsta and his blogpost as well as to Allan Jude, Kris Moore and Benedict Reuschling for their AWESOME and inspiring podcast, BSD Now.
##News Roundup
\n###One year of C
\n\n\nIt’s now nearly a year that I started writing non-trivial amounts of C code again (the first sokol_gfx.h commit was on the 14-Jul-2017), so I guess it’s time for a little retrospective.
\n
\n\n\nIn the beginning it was more of an experiment: I wanted to see how much I would miss some of the more useful C++ features (for instance namespaces, function overloading, ‘simple’ template code for containers, …), and whether it is possible to write non-trivial codebases in C without going mad.
\n
\n\n\nHere are all the github projects I wrote in C:
\n
\n\n\nAll in all these are around 32k lines of code (not including 3rd party code like flextGL and HandmadeMath). I think I wrote more C code in the recent 10 months than any other language.
\n
\n\n\nSo one thing seems to be clear: yes, it’s possible to write a non-trivial amount of C code that does something useful without going mad (and it’s even quite enjoyable I might add).
\n
Here’s a few things I learned:
\nPick the right language for a problem
\nC is a perfect match for WebAssembly
\nC99 is a huge improvement over C89
\nThe dangers of pointers and explicit memory management are overrated
\nLess Boilerplate Code
\nLess Language Feature ‘Anxiety’
\nConclusion
\n\n\n\nAll in all my “C experiment” is a success. For a lot of problems, picking C over C++ may be the better choice since C is a much simpler language (btw, did you notice how there are hardly any books, conferences or discussions about C despite being a fairly popular language? Apart from the neverending bickering about undefined behaviour from the compiler people of course ;) There simply isn’t much to discuss about a language that can be learned in an afternoon.
\n
\n\n\nI don’t like some of the old POSIX or Linux APIs as much as the next guy (e.g. ioctl(), the socket API or some of the CRT library functions), but that’s an API design problem, not a language problem. It’s possible to build friendly C APIs with a bit of care and thinking, especially when C99’s designated initialization can be used (C++ should really make sure that the full C99 language can be used from inside C++ instead of continuing to wander off into an entirely different direction).
\n
###Configuring OpenBGPD to announce VM’s virtual networks
\n\n\n\n\nWe use BGP quite heavily at work, and even though I’m not interacting with that directly, it feels like it’s something very useful to learn at least on some basic level. The most effective and fun way of learning technology is finding some practical application, so I decided to see if it could help to improve networking management for my Virtual Machines.
\n
\n\n\nMy setup is fairly simple: I have a host that runs bhyve VMs and I have a desktop system from where I ssh to VMs, both hosts run FreeBSD. All VMs are connected to each other through a bridge and have a common network 10.0.1/24. The point of this exercise is to be able to ssh to these VMs from desktop without adding static routes and without adding vmhost’s external interfaces to the VMs bridge.
\n
\n\n\nI’ve installed openbgpd on both hosts and configured it like this:
\n
vmhost: /usr/local/etc/bgpd.conf\nAS 65002\nrouter-id 192.168.87.48\nfib-update no\n\nnetwork 10.0.1.1/24\n\nneighbor 192.168.87.41 {\n descr "desktop"\n remote-as 65001\n}\n
\n\n\n\n\nHere, router-id is set vmhost’s IP address in my home network (192.168.87/24), fib-update no is set to forbid routing table update, which I initially set for testing, but keeping it as vmhost is not supposed to learn new routes from desktop anyway. network announces my VMs network and neighbor describes my desktop box. Now the desktop box:
\n
desktop: /usr/local/etc/bgpd.conf\nAS 65001\nrouter-id 192.168.87.41\nfib-update yes\n\nneighbor 192.168.87.48 { \n descr "vmhost" \n remote-as 65002 \n}\n
\n\n\n\n\nIt’s pretty similar to vmhost’s bgpd.conf, but no networks are announced here, and fib-update is set to yes because the whole point is to get VM routes added. Both hosts have to have the openbgpd service enabled:
\n
/etc/rc.conf.local\nopenbgpd_enable="YES"\n
\n\n\n\n\nAs mentioned already, similar result could be achieved without using BGP by using either static routes or bridging interfaces differently, but the purpose of this exercise is to get some basic hands-on experience with BGP. Right now I’m looking into extending my setup in order to try more complex BGP schema. I’m thinking about adding some software switches in front of my VMs or maybe adding a second VM host (if budget allows). You’re welcome to comment if you have some ideas how to extend this setup for educational purposes in the context of BGP and networking.
\n
\n\n\nAs a side note, I really like openbgpd so far. Its configuration file format is clean and simple, documentation is good, error and information messages are clear, and CLI has intuitive syntax.
\n
Digital Ocean
\n\n\n\n\n\n\nAll people within the IT Industry should known where the slogan “The Power To Serve” is exposed every day to millions of people. But maybe too much wishful thinking from me. But without “The Power To Serve” the IT industry today will look totally different. Companies like Apple, Juniper, Cisco and even WatsApp would not exist in their current form.
\n
\n\n\nI provide IT architecture services to make your complex IT landscape manageable and I love to solve complex security and privacy challenges. Complex challenges where people, processes and systems are heavily interrelated. For this knowledge intensive work I often run some IT experiments. When you run experiments nowadays you have a choice:
\n
\n\n\nRunning your own developments experiments on your own infrastructure can be time consuming. However smart automation saves time and money. And by creating your own CICD pipeline (Continuous Integration, Continuous Deployment) you stay on top of core infrastructure developments. Even hands-on. Knowing how things work from a technical ‘hands-on’ perspective gives great advantages when it comes to solving complex business IT problems. Making a clear distinguish between a business problem or IT problem is useless. Business and IT problems are related. Sometimes causal related, but more often indirect by one or more non linear feedback loops. Almost every business depends of IT systems. Bad IT means often that your customers will leave your business.
\n
\n\n\nOne of the things of FeeBSD for me is still FreeBSD Jails. In 2015 I had luck to attend to a presentation of the legendary hacker Poul-Henning Kamp . Check his BSD bio to see what he has done for the FreeBSD community! FreeBSD jails are a light way to visualize your system without enormous overhead. Now that the development on Linux for LXD/LXD is more mature (lxd is the next generation system container manager on linux) there is finally again an alternative for a nice chroot Linux based system again. At least when you do not need the overhead and management complexity that comes with Kubernetes or Docker.
\n
\n\n\nFreeBSD means control and quality for me. When there is an open source package I need, I want to install it from source. It gives me more control and always some extra knowledge on how things work. So no precompiled binaries for me on my BSD systems! If a build on FreeBSD fails most of the time this is an alert regarding the quality for me.
\n
\n\n\nIf a complex OSS package is not available at all in the FreeBSD ports collection there should be a reason for it. Is it really that nobody on the world wants to do this dirty maintenance work? Or is there another cause that running this software on FreeBSD is not possible…There are currently 32644 ports available on FreeBSD. So all the major programming language, databases and middleware libraries are present. The FreeBSD organization is a mature organization and since this is one of the largest OSS projects worldwide learning how this community manages to keep innovation and creates and maintains software is a good entrance for learning how complex IT systems function.
\n
\n\n\nFreeBSD is of course BSD licensed. It worked well! There is still a strong community with lots of strong commercial sponsors around the community. Of course: sometimes a GPL license makes more sense. So beside FreeBSD I also love GPL software and the rationale and principles behind it. So my hope is that maybe within the next 25 years the hard battle between BSD vs GPL churches will be more rationalized and normalized. Principles are good, but as all good IT architects know: With good principles alone you never make a good system. So use requirements and not only principles to figure out what OSS license fits your project. There is never one size fits all.
\n
\n\n\nJune 19, 1993 was the day the official name for FreeBSD was agreed upon. So this blog is written to celebrate 25th anniversary of FreeBSD.
\n
###Dave’s BSDCan trip report
\n\n\n\n\nHello guys! During the last show, you asked for a trip report regarding BSDCan 2018.
\n
\nThis was my first time attending BSDCan. However, BSDCan was my second BSD conference overall, my first being vBSDCon 2017 in Reston, VA.
\nArriving early Thursday evening and after checking into the hotel, I headed straight to the Red Lion for the registration, picked up my badge and swag and then headed towards the ‘DMS’ building for the newbies talk. The only thing is, I couldn’t find the DMS building! Fortunately I found a BSDCan veteran who was heading there themselves. My only suggestion is to include the full building name and address on the BSDCan web site, or even a link to Google maps to help out with the navigation. The on-campus street maps didn’t have ‘DMS’ written on them anywhere. But I digress.
\nOnce I made it to the newbies talk hosted by Dan Langille and Michael W Lucas, it highlighted places to meet, an overview of what is happening, details about the ‘BSDCan widow/widower tours’ and most importantly, the 6-2-1 rule!
\nThe following morning, we were present with tea/coffee, muffins and other goodies to help prepare us for the day ahead.
\nThe first talk, “The Tragedy of systemd” covered what systemd did wrong and how the BSD community could improve on the ideas behind it.
\nWith the exception of Michael W Lucas, SSH Key Management and Kirk McKusick, The Evolution of FreeBSD Governance talk, I pretty much attended all of the ZFS talks including the lunchtime BoF session, hosted by Allan Jude. Coming from FreeNAS and being involved in the community, this is where my main interest and motivation lies. Since then I have been able to share some of that information with the FreeNAS community forums and chatroom.
\nI also attended the “Speculating about Intel” lunchtime BoF session hosted by Theo de Raddt, which proved to be “interesting”.
\nThe talks ended with the wrap up session with a few words from Dan, covering the record attendance and made very clear there “was no cabal”. Followed by the the handing over of Groff the BSD goat to a new owner, thank you’s from the FreeBSD Foundation to various community committers and maintainers, finally ending with the charity auction, where a things like a Canadian $20 bill sold for $40, a signed FreeBSD Foundation shirt originally worn by George Neville-Neil, a lost laptop charger, Michael’s used gelato spoon, various books, the last cookie and more importantly, the second to last cookie!
\nAfter the auction, we all headed to the Red Lion for food and drinks, sponsored by iXsystems.
\nI would like to thank the BSDCan organizers, speakers and sponsors for a great conference. I will certainly hope to attend next year!
\nRegards,
\nDave (aka m0nkey_)
##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nDragonflyBSD’s hammer1 encrypted master/slave setup, second part of our BSDCan recap, NomadBSD 1.1-RC1 available, OpenBSD adds an LDAP client to base, FreeBSD gets pNFS support, Intel FPU Speculation Vulnerability confirmed, and what some Unix command names mean.
\n\n##Headlines
\n###DragonflyBSD: Towards a HAMMER1 master/slave encrypted setup with LUKS
\n\n\nI just wanted to share my experience with setting up DragonFly master/slave HAMMER1 PFS’s on top of LUKS
\n
\nSo after a long time using an Synology for my NFS needs, I decided it was time to rethink my setup a little since I had several issues with it :
\n\n\nI have been playing with DragonFly in the past and knew about HAMMER, now I just had the perfect excuse to actually use it in production :) After setting up the OS, creating the LUKS partition and HAMMER FS was easy :
\n
kdload dm
\ncryptsetup luksFormat /dev/serno/<id1>
\ncryptsetup luksOpen /dev/serno/<id1> fort_knox
\nnewfs_hammer -L hammer1_secure_master /dev/mapper/fort_knox
\ncryptsetup luksFormat /dev/serno/<id2>
\ncryptsetup luksOpen /dev/serno/<id2> fort_knox_slave
\nnewfs_hammer -L hammer1_secure_slave /dev/mapper/fort_knox_slave
mount /dev/mapper/fort_knox /fort_knox
\nmount /dev/mapper_fort_know_slave /fort_knox_slave
\n\n\nYou can now put your data under /fort_knox
\n
\nNow, off to setting up the replication, first get the shared-uuid of /fort_knox
hammer pfs-status /fort_knox
\n\n\nCreate a PFS slave “linked” to the master
\n
hammer pfs-slave /fort_knox_slave/pfs/slave shared-uuid=f9e7cc0d-eb59-10e3-a5b5-01e6e7cefc12
\n\n\nAnd then stream your data to the slave PFS !
\n
hammer mirror-stream /fort_knox /fort_knox_slave/pfs/slave
\n\n\nAfter that, setting NFS is fairly trivial even though I had problem with the /etc/exports syntax which is different than Linux
\n
\n\n\nThere’s a few things I wish would be better though but nothing too problematic or without workarounds :
\n
\n\n\nOverall, I am happy, HAMMER1 and PFS are looking really good, DragonFly is a neat Unix and the community is super friendly (Matthew Dillon actually provided me with a kernel patch to fix the broken ACPI on the PC holding this setup, many thanks!), the system is still a “work in progress” but it is already serving my files as I write this post.
\n
\n\n\nLet’s see in 6 months how it goes in the longer run !
\n
###BSDCan 2018 Recap
\n\n\n\n\n“Automating Network Infrastructures with Ansible on FreeBSD” in the DevSummit track. A good talk that connected well with his Ansible tutorial and even allowed some discussions among participants.
\n
\n“All along the dwatch tower”: Devin delivered a well prepared talk. I first thought that the number of slides would not fit into the time slot, but she even managed to give a demo of her work, which was well received. The dwatch tool she wrote should make it easy for people to get started with DTrace without learning too much about the syntax at first. The visualizations were certainly nice to see, combining different tools together in a new way.
\nZFS BoF, lead by Allan and Matthew Ahrens
\nSSH Key Management by Michael W. Lucas. Yet another great talk where I learned a lot. I did not get to the SSH CA chapter in the new SSH Mastery book, so this was a good way to wet my appetite for it and motivated me to look into creating one for the cluster that I’m managing.
\nThe rest of the day was spent at the FreeBSD Foundation table, talking to various folks. Then, Allan and I had an interview with Kirk McKusick for National FreeBSD Day, then we had a core meeting, followed by a core dinner.
\n\n“Flexible Disk Use in OpenZFS”: Matthew Ahrens talking about the feature he is implementing to expand a RAID-Z with a single disk, as well as device removal.
\n
\nAllan’s talk about his efforts to implement ZSTD in OpenZFS as another compression algorithm. I liked his overview slides with the numbers comparing the algorithms for their effectiveness and his personal story about the sometimes rocky road to get the feature implemented.
\n“zrepl - ZFS replication” by Christian Schwarz, was well prepared and even had a demo to show what his snapshot replication tool can do. We covered it on the show before and people can find it under sysutils/zrepl. Feedback and help is welcome.
\n“The Evolution of FreeBSD Governance” by Kirk McKusick was yet another great talk by him covering the early days of FreeBSD until today, detailing some of the progress and challenges the project faced over the years in terms of leadership and governance. This is an ongoing process that everyone in the community should participate in to keep the project healthy and infused with fresh blood.
\nClosing session and auction were funny and great as always.
\nAll in all, yet another amazing BSDCan. Thank you Dan Langille and your organizing team for making it happen! Well done.
Digital Ocean
\n\n\n\n\n\n\nThe first – and hopefully final – release candidate of NomadBSD 1.1 is available!
\n
##News Roundup
\n###LDAP client added to -current
CVSROOT: /cvs\nModule name: src\nChanges by: reyk@cvs.openbsd.org 2018/06/13 09:45:58\n\nLog message:\n Import ldap(1), a simple ldap search client.\n We have an ldapd(8) server and ypldap in base, so it makes sense to\n have a simple LDAP client without depending on the OpenLDAP package.\n This tool can be used in an ssh(1) AuthorizedKeysCommand script.\n \n With feedback from many including millert@ schwarze@ gilles@ dlg@ jsing@\n \n OK deraadt@\n \n Status:\n \n Vendor Tag: reyk\n Release Tags: ldap_20180613\n \n N src/usr.bin/ldap/Makefile\n N src/usr.bin/ldap/aldap.c\n N src/usr.bin/ldap/aldap.h\n N src/usr.bin/ldap/ber.c\n N src/usr.bin/ldap/ber.h\n N src/usr.bin/ldap/ldap.1\n N src/usr.bin/ldap/ldapclient.c\n N src/usr.bin/ldap/log.c\n N src/usr.bin/ldap/log.h\n \n No conflicts created by this import\n
\n\n###Intel® FPU Speculation Vulnerability Confirmed
\n\nSummary:\n\nSystem software may utilize the Lazy FP state restore technique to delay the restoring of state until an instruction operating on that state is actually executed by the new process. Systems using Intel® Core-based microprocessors may potentially allow a local process to infer data utilizing Lazy FP state restore from another process through a speculative execution side channel.\n\nDescription:\n\nSystem software may opt to utilize Lazy FP state restore instead of eager save and restore of the state upon a context switch. Lazy restored states are potentially vulnerable to exploits where one process may infer register values of other processes through a speculative execution side channel that infers their value.\n\n · CVSS - 4.3 Medium CVSS:3.0/AV:L/AC:L/PR:N/UI:N/S:C/C:L/I:N/A:N\nAffected Products:\n\nIntel® Core-based microprocessors.\n\nRecommendations:\n\nIf an XSAVE-enabled feature is disabled, then we recommend either its state component bitmap in the extended control register (XCR0) is set to 0 (e.g. XCR0[bit 2]=0 for AVX, XCR0[bits 7:5]=0 for AVX512) or the corresponding register states of the feature should be cleared prior to being disabled. Also for relevant states (e.g. x87, SSE, AVX, etc.), Intel recommends system software developers utilize Eager FP state restore in lieu of Lazy FP state restore.\n\nAcknowledgements:\n\nIntel would like to thank Julian Stecklina from Amazon Germany, Thomas Prescher from Cyberus Technology GmbH (https://www.cyberus-technology.de/), Zdenek Sojka from SYSGO AG (http://sysgo.com), and Colin Percival for reporting this issue and working with us on coordinated disclosure.\n
\n\niXsystems
\niX Ad Spot
\n###iX Systems - BSDCan 2018 Recap
Merge the pNFS server code from projects/pnfs-planb-server into head.\n\nThis code merge adds a pNFS service to the NFSv4.1 server. Although it is\na large commit it should not affect behaviour for a non-pNFS NFS server.\nSome documentation on how this works can be found at:\nMerge the pN http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt\nand will hopefully be turned into a proper document soon.\nThis is a merge of the kernel code. Userland and man page changes will\ncome soon, once the dust settles on this merge.\nIt has passed a "make universe", so I hope it will not cause build problems.\nIt also adds NFSv4.1 server support for the "current stateid".\n\nHere is a brief overview of the pNFS service:\nA pNFS service separates the Read/Write operations from all the other NFSv4.1\nMetadata operations. It is hoped that this separation allows a pNFS service\nto be configured that exceeds the limits of a single NFS server for either\nstorage capacity and/or I/O bandwidth.\nIt is possible to configure mirroring within the data servers (DSs) so that\nthe data storage file for an MDS file will be mirrored on two or more of\nthe DSs.\nWhen this is used, failure of a DS will not stop the pNFS service and a\nfailed DS can be recovered once repaired while the pNFS service continues\nto operate. Although two way mirroring would be the norm, it is possible\nto set a mirroring level of up to four or the number of DSs, whichever is\nless.\nThe Metadata server will always be a single point of failure,\njust as a single NFS server is.\n\nA Plan B pNFS service consists of a single MetaData Server (MDS) and K\nData Servers (DS), all of which are recent FreeBSD systems.\nClients will mount the MDS as they would a single NFS server.\nWhen files are created, the MDS creates a file tree identical to what a\nsingle NFS server creates, except that all the regular (VREG) files will\nbe empty. As such, if you look at the exported tree on the MDS directly\non the MDS server (not via an NFS mount), the files will all be of size 0.\nEach of these files will also have two extended attributes in the system\nattribute name space:\npnfsd.dsfile - This extended attrbute stores the information that\n the MDS needs to find the data storage file(s) on DS(s) for this file.\npnfsd.dsattr - This extended attribute stores the Size, AccessTime, ModifyTime\n and Change attributes for the file, so that the MDS doesn't need to\n acquire the attributes from the DS for every Getattr operation.\nFor each regular (VREG) file, the MDS creates a data storage file on one\n(or more if mirroring is enabled) of the DSs in one of the "dsNN"\nsubdirectories. The name of this file is the file handle\nof the file on the MDS in hexadecimal so that the name is unique.\nThe DSs use subdirectories named "ds0" to "dsN" so that no one directory\ngets too large. The value of "N" is set via the sysctl vfs.nfsd.dsdirsize\non the MDS, with the default being 20.\nFor production servers that will store a lot of files, this value should\nprobably be much larger.\nIt can be increased when the "nfsd" daemon is not running on the MDS,\nonce the "dsK" directories are created.\n\nFor pNFS aware NFSv4.1 clients, the FreeBSD server will return two pieces\nof information to the client that allows it to do I/O directly to the DS.\nDeviceInfo - This is relatively static information that defines what a DS\n is. The critical bits of information returned by the FreeBSD\n server is the IP address of the DS and, for the Flexible\n File layout, that NFSv4.1 is to be used and that it is\n "tightly coupled".\n There is a "deviceid" which identifies the DeviceInfo.\nLayout - This is per file and can be recalled by the server when it\n is no longer valid. For the FreeBSD server, there is support\n for two types of layout, call File and Flexible File layout.\n Both allow the client to do I/O on the DS via NFSv4.1 I/O\n operations. The Flexible File layout is a more recent variant\n that allows specification of mirrors, where the client is\n expected to do writes to all mirrors to maintain them in a\n consistent state. The Flexible File layout also allows the\n client to report I/O errors for a DS back to the MDS.\n The Flexible File layout supports two variants referred to as\n "tightly coupled" vs "loosely coupled". The FreeBSD server always\n uses the "tightly coupled" variant where the client uses the\n same credentials to do I/O on the DS as it would on the MDS.\n For the "loosely coupled" variant, the layout specifies a\n synthetic user/group that the client uses to do I/O on the DS.\n The FreeBSD server does not do striping and always returns\n layouts for the entire file. The critical information in a layout\n is Read vs Read/Writea and DeviceID(s) that identify which\n DS(s) the data is stored on.\n\nAt this time, the MDS generates File Layout layouts to NFSv4.1 clients\nthat know how to do pNFS for the non-mirrored DS case unless the sysctl\nvfs.nfsd.default_flexfile is set non-zero, in which case Flexible File\nlayouts are generated.\nThe mirrored DS configuration always generates Flexible File layouts.\nFor NFS clients that do not support NFSv4.1 pNFS, all I/O operations\nare done against the MDS which acts as a proxy for the appropriate DS(s).\nWhen the MDS receives an I/O RPC, it will do the RPC on the DS as a proxy.\nIf the DS is on the same machine, the MDS/DS will do the RPC on the DS as\na proxy and so on, until the machine runs out of some resource, such as\nsession slots or mbufs.\nAs such, DSs must be separate systems from the MDS.\n\n***\n\n###[What does {some strange unix command name} stand for?](http://www.unixguide.net/unix/faq/1.3.shtml)\n\n+ awk = "Aho Weinberger and Kernighan" \n+ grep = "Global Regular Expression Print" \n+ fgrep = "Fixed GREP". \n+ egrep = "Extended GREP" \n+ cat = "CATenate" \n+ gecos = "General Electric Comprehensive Operating Supervisor" \n+ nroff = "New ROFF" \n+ troff = "Typesetter new ROFF" \n+ tee = T \n+ bss = "Block Started by Symbol\n+ biff = "BIFF" \n+ rc (as in ".cshrc" or "/etc/rc") = "RunCom" \n+ Don Libes' book "Life with Unix" contains lots more of these \ntidbits. \n***\n\n##Beastie Bits\n+ [RetroBSD: Unix for microcontrollers](http://retrobsd.org/wiki/doku.php)\n+ [On the matter of OpenBSD breaking embargos (KRACK)](https://marc.info/?l=openbsd-tech&m=152910536208954&w=2)\n+ [Theo's Basement Computer Paradise (1998)](https://zeus.theos.com/deraadt/hosts.html)\n+ [Airport Extreme runs NetBSD](https://jcs.org/2018/06/12/airport_ssh)\n+ [What UNIX shell could have been](https://rain-1.github.io/shell-2.html)\n\n***\nTarsnap ad\n***\n\n##Feedback/Questions\n+ We need more feedback and questions. Please email feedback@bsdnow.tv \n+ Also, many of you owe us BSDCan trip reports! We have shared what our experience at BSDCan was like, but we want to hear about yours. What can we do better next year? What was it like being there for the first time?\n+ [Jason writes in](https://slexy.org/view/s205jU58X2)\n + https://www.wheelsystems.com/en/products/wheel-fudo-psm/\n+ [June 19th was National FreeBSD Day](https://twitter.com/search?src=typd&q=%23FreeBSDDay)\n***\n\n- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [feedback@bsdnow.tv](mailto:feedback@bsdnow.tv)\n***\n\n
","summary":"DragonflyBSD’s hammer1 encrypted master/slave setup, second part of our BSDCan recap, NomadBSD 1.1-RC1 available, OpenBSD adds an LDAP client to base, FreeBSD gets pNFS support, Intel FPU Speculation Vulnerability confirmed, and what some Unix command names mean.","date_published":"2018-06-21T05:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/034d5002-639f-4744-a773-9c000ce91d1c.mp3","mime_type":"audio/mp3","size_in_bytes":53300210,"duration_in_seconds":5323}]},{"id":"http://feed.jupiter.zone/bsdnow#entry-2107","title":"Episode 250: BSDCan 2018 Recap | BSD Now 250","url":"https://www.bsdnow.tv/250","content_text":"TrueOS becoming a downstream fork with Trident, our BSDCan 2018 recap, HardenedBSD Foundation founding efforts, VPN with OpenIKED on OpenBSD, FreeBSD on a System76 Galago Pro, and hardware accelerated crypto on Octeons.\n\n##Headlines##\n###TrueOS to Focus on Core Operating System\n\n\nThe TrueOS Project has some big plans in the works, and we want to take a minute and share them with you. Many have come to know TrueOS as the “graphical FreeBSD” that makes things easy for newcomers to the BSDs. Today we’re announcing that TrueOS is shifting our focus a bit to become a cutting-edge operating system that keeps all of the stability that you know and love from ZFS (OpenZFS) and FreeBSD, and adds additional features to create a fresh, innovative operating system. Our goal is to create a core-centric operating system that is modular, functional, and perfect for do-it-yourselfers and advanced users alike.\n\n\n\nTrueOS will become a downstream fork that will build on FreeBSD by integrating new software technologies like OpenRC and LibreSSL. Work has already begun which allows TrueOS to be used as a base platform for other projects, including JSON-based manifests, integrated Poudriere / pkg tools and much more. We’re planning on a six month release cycle to keep development moving and fresh, allowing us to bring you hot new features to ZFS, bhyve and related tools in a timely manner. This makes TrueOS the perfect fit to serve as the basis for building other distributions.\n\n\n\nSome of you are probably asking yourselves “But what if I want to have a graphical desktop?” Don’t worry! We’re making sure that everyone who knows and loves the legacy desktop version of TrueOS will be able to continue using a FreeBSD-based, graphical operating system in the future. For instance, if you want to add KDE, just use sudo pkg install kde and voila! You have your new shiny desktop. Easy right? This allows us to get back to our roots of being a desktop agnostic operating system. If you want to add a new desktop environment, you get to pick the one that best suits your use.\n\n\n\nWe know that some of you will still be looking for an out-of-the-box solution similar to legacy PC-BSD and TrueOS. We’re happy to announce that Project Trident will take over graphical FreeBSD development going forward. Not much is going to change in that regard other than a new name! You’ll still have Lumina Desktop as a lightweight and feature-rich desktop environment and tons of utilities from the legacy TrueOS toolchain like sysadm and AppCafe. There will be migration paths available for those that would like to move to other FreeBSD-based distributions like Project Trident or GhostBSD.\n\n\n\nWe look forward to this new chapter for TrueOS and hope you will give the new edition a spin! Tell us what you think about the new changes by leaving us a comment. Don’t forget you can ask us questions on our Twitter and be a part of our community by joining the new TrueOS Forums when they go live in about a week. Thanks for being a loyal fan of TrueOS.\n\n\n###Project Trident FAQ\n\n\nQ: Why did you pick the name “Project Trident”?\n\n\n\nA: We were looking for a name that was unique, yet would still relate to the BSD community. Since Beastie (the FreeBSD mascot) is always pictured with a trident, it felt like that would be a great name.\n\n\n\nQ: Where can users go for technical support?\n\n\n\nA: At the moment, Project Trident will continue sharing the TrueOS community forums and Telegram channels. We are currently evaluating dedicated options for support channels in the future.\n\n\n\nQ: Can I help contribute to the project?\n\n\n\nA: We are always looking for developers who want to join the project. If you’re not a developer you can still help, as a community project we will be more reliant on contributions from the community in the form of how-to guides and other user-centric documentation and support systems.\n\n\n\nQ: How is the project supported financially?\n\n\n\nA: Project Trident is sponsored by the community, from both individuals and corporations. iXsystems has stepped up as the first enterprise-level sponsor of the project, and has been instrumental in getting Project Trident up and running. Please visit the Sponsors page to see all the current sponsors.\n\n\n\nQ: How can I help support the project financially?\n\n\n\nA: Several methods exist, from one time or recurring donations via Paypal to limited time swag t-shirt campaigns during the year. We are also looking into more alternative methods of support, so please visit the Sponsors page to see all the current methods of sponsorship.\n\n\n\nQ: Will there be any transparency of the financial donations and expenditures?\n\n\n\nA: Yes, we will be totally open with how much money comes into the project and what it is spent on. Due to concerns of privacy, we will not identify individuals and their donation amounts unless they specifically request to be identified. We will release a monthly overview in/out ledger, so that community members can see where their money is going.\n\n\n\n\nRelationship with TrueOS\n\n\nProject Trident does have very close ties to the TrueOS project, since most of the original Project Trident developers were once part of the TrueOS project before it became a distribution platform. For users of the TrueOS desktop, we have some additional questions and answers below.\n\n\nQ: Do we need to be at a certain TrueOS install level/release to upgrade?\n\n\n\n\nA: As long as you have a TrueOS system which has been updated to at least the 18.03 release you should be able to just perform a system update to be automatically upgraded to Project Trident.\n\n\n\nQ: Which members moved from TrueOS to Project Trident?\n\n\n\nA: Project Trident is being led by prior members of the TrueOS desktop team. Ken and JT (development), Tim (documentation) and Rod (Community/Support). Since Project Trident is a community-first project, we look forward to working with new members of the team.\n\n\n\n\niXsystems\n\n###BSDCan\n\n\nBSDCan finished Saturday last week\nIt started with the GoatBoF on Tuesday at the Royal Oak Pub, where people had a chance to meet and greet. Benedict could not attend due to an all-day FreeBSD Foundation meeting and and even FreeBSD Journal Editorial Board meeting.\nThe FreeBSD devsummit was held the next two days in parallel to the tutorials. Gordon Tetlow, who organized the devsummit, opened the devsummit. Deb Goodkin from the FreeBSD Foundation gave the first talk with a Foundation update, highlighting current and future efforts. Li-Wen Hsu is now employed by the Foundation to assist in QA work (Jenkins, CI/CD) and Gordon Tetlow has a part-time contract to help secteam as their secretary.\nNext, the FreeBSD core team (among them Allan and Benedict) gave a talk about what has happened this last term. With a core election currently running, some of these items will carry over to the next core team, but there were also some finished ones like the FCP process and FreeBSD members initiative. People in the audience asked questions on various topics of interest.\nAfter the coffee break, the release engineering team gave a talk about their efforts in terms of making releases happen in time and good quality.\nBenedict had to give his Ansible tutorial in the afternoon, which had roughly 15 people attending. Most of them beginners, we could get some good discussions going and I also learned a few new tricks. The overall feedback was positive and one even asked what I’m going to teach next year.\nThe second day of the FreeBSD devsummit began with Gordon Tetlow giving an insight into the FreeBSD Security team (aka secteam). He gave a overview of secteam members and responsibilities, explaining the process based on a long past advisory. Developers were encouraged to help out secteam. NDAs and proper disclosure of vulnerabilities were also discussed, and the audience had some feedback and questions.\nWhen the coffee break was over, the FreeBSD 12.0 planning session happened. A Google doc served as a collaborative way of gathering features and things left to do. People signed up for it or were volunteered. Some features won’t make it into 12.0 as they are not 100% ready for prime time and need a few more rounds of testing and bugfixing. Still, 12.0 will have some compelling features.\nA 360° group picture was taken after lunch, and then people split up into the working groups for the afternoon or started hacking in the UofO Henderson residence.\nBenedict and Allan both attended the OpenZFS working group, lead by Matt Ahrens. He presented the completed and outstanding work in FreeBSD, without spoiling too much of the ZFS presentations of various people that happened later at the conference.\nBenedict joined the boot code session a bit late (hallway track is the reason) when most things seem to have already been discussed.\nBSDCan 2018 — Ottawa (In Pictures)\niXsystems Photos from BSDCan 2018\n\n\n\n\n##News Roundup\n###June HardenedBSD Foundation Update\n\n\nWe at HardenedBSD are working towards starting up a 501©(3) not-for-profit organization in the USA. Setting up this organization will allow future donations to be tax deductible. We’ve made progress and would like to share with you the current state of affairs.\n\n\n\nWe have identified, sent invitations out, and received acceptance letters from six people who will serve on the HardenedBSD Foundation Board of Directors. You can find their bios below. In the latter half of June 2018 or the beginning half of July 2018, we will meet for the first time as a board and formally begin the process of creating the documentation needed to submit to the local, state, and federal tax services.\n\n\n\nHere’s a brief introduction to those who will serve on the board:\n\n\n\n\nW. Dean Freeman (Advisor): Dean has ten years of professional experience with deploying and security Unix and networking systems, including assessing systems security for government certification and assessing the efficacy of security products. He was introduced to Unix via FreeBSD 2.2.8 on an ISP shell account as a teenager. Formerly, he was the Snort port maintainer for FreeBSD while working in the Sourcefire VRT, and has contributed entropy-related patches to the FreeBSD and HardenedBSD projects – a topic on which he presented at vBSDCon 2017.\n\n\nBen La Monica (Advisor): Ben is a Senior Technology Manager of Software Engineering at Morningstar, Inc and has been developing software for over 15 years in a variety of languages. He advocates open source software and enjoys tinkering with electronics and home automation.\n\n\nGeorge Saylor (Advisor): George is a Technical Directory at G2, Inc. Mr. Saylor has over 28 years of information systems and security experience in a broad range of disciplines. His core focus areas are automation and standards in the event correlation space as well as penetration and exploitation of computer systems. Mr Saylor was also a co-founder of the OpenSCAP project.\n\n\nVirginia Suydan (Accountant and general administrator): Accountant and general administrator for the HardenedBSD Foundation. She has worked with Shawn Webb for tax and accounting purposes for over six years.\n\n\nShawn Webb (Director): Co-founder of HardenedBSD and all-around infosec wonk. He has worked and played in the infosec industry, doing both offensive and defensive research, for around fifteen years. He loves open source technologies and likes to frustrate the bad guys.\n\n\nBen Welch (Advisor): Ben is currently a Security Engineer at G2, Inc. He graduated from Pennsylvania College of Technology with a Bachelors in Information Assurance and Security. Ben likes long walks, beaches, candlelight dinners, and attending various conferences like BSides and ShmooCon.\n\n\n\n\n\n###Your own VPN with OpenIKED & OpenBSD\n\n\nRemote connectivity to your home network is something I think a lot of people find desirable. Over the years, I’ve just established an SSH tunnel and use it as a SOCKS proxy, sending my traffic through that. It’s a nice solution for a “poor man’s VPN”, but it can be a bit clunky, and it’s not great having to expose SSH to the world, even if you make sure to lock everything down \n\n\n\nI set out the other day to finally do it properly. I’d come across this great post by Gordon Turner: OpenBSD 6.2 VPN Endpoint for iOS and macOS\n\n\n\nWhilst it was exactly what I was looking for, it outlined how to set up an L2TP VPN. Really, I wanted IKEv2 for performance and security reasons (I won’t elaborate on this here, if you’re curious about the differences, there’s a lot of content out on the web explaining this).\n\n\n\nThe client systems I’d be using have native support for IKEv2 (iOS, macOS, other BSD systems). But, I couldn’t find any tutorials in the same vein.\n\n\n\nSo, let’s get stuck in!\n\n\n\nA quick note ✍️\n\n\n\nThis guide will walk through the set up of an IKEv2 VPN using OpenIKED on OpenBSD. It will detail a “road warrior” configuration, and use a PSK (pre-shared-key) for authentication. I’m sure it can be easily adapted to work on any other platforms that OpenIKED is available on, but keep in mind my steps are specifically for OpenBSD.\n\n\n\nServer Configuration\n\n\n\nAs with all my home infrastructure, I crafted this set-up declaratively. So, I had the deployment of the VM setup in Terraform (deployed on my private Triton cluster), and wrote the configuration in Ansible, then tied them together using radekg/terraform-provisioner-ansible.\n\n\n\nOne of the reasons I love Ansible is that its syntax is very simplistic, yet expressive. As such, I feel it fits very well into explaining these steps with snippets of the playbook I wrote. I’ll link the full playbook a bit further down for those interested.\n\n\n\nSee the full article for the information on:\nsysctl parameters\nThe naughty list (optional)\nConfigure the VPN network interface\nConfigure the firewall\nConfigure the iked service\nGateway configuration\nClient configuration\nTroubleshooting\n\n\n\n\nDigitalOcean\n\n###FreeBSD on a System76 Galago Pro\n\n\nHey all, It’s been a while since I last posted but I thought I would hammer something out here. My most recent purchase was a System76 Galago Pro. I thought, afer playing with POP! OS a bit, is there any reason I couldn’t get BSD on this thing. Turns out the answer is no, no there isnt and it works pretty decently.\n\n\n\nTo get some accounting stuff out of the way I tested this all on FreeBSD Head and 11.1, and all of it is valid as of May 10, 2018. Head is a fast moving target so some of this is only bound to improve.\n\n\n\n\nThe hardware\n\n\nIntel Core i5 Gen 8\n\n\nUHD Graphics 620\n\n\n16 GB DDR4 Ram\n\n\nRTL8411B PCI Express Card Reader\n\n\nRTL8111 Gigabit ethernet controller\n\n\nIntel HD Audio\n\n\nSamsung SSD 960 PRO 512GB NVMe\n\n\nThe caveats\n\n\n\n\nThere are a few things that I cant seem to make work straight out of the box, and that is the SD Card reader, the backlight, and the audio is a bit finicky. Also the trackpad doesn’t respond to two finger scrolling. The wiki is mostly up to date, there are a few edits that need to be made still but there is a bug where I cant register an account yet so I haven’t made all the changes.\n\n\n\nProcessor\n\n\n\nIt works like any other Intel processor. Pstates and throttling work.\n\n\n\nGraphics\n\n\n\nThe boot menu sets itself to what looks like 1024x768, but works as you expect in a tiny window. The text console does the full 3200x1800 resolution, but the text is ultra tiny. There isnt a font for the console that covers hidpi screens yet. As for X Windows it requres the drm-kmod-next package. Once installed follow the directions from the package and it works with almost no fuss. I have it running on X with full intel acceleration, but it is running at it’s full 3200x1800 resolution, to scale that down just do xrandr --output eDP-1 --scale 0.5x0.5 it will blow it up to roughly 200%. Due to limitations with X windows and hidpi it is harder to get more granular.\n\n\n\nIntel Wireless 8265\n\n\n\nThe wireless uses the iwm module, as of right now it does not seem to automagically load right now. Adding iwm_load=“YES” will cause the module to load on boot and kldload iwm\n\n\n\nBattery\n\n\n\nI seem to be getting about 5 hours out of the battery, but everything reports out of the box as expected. I could get more by throttling the CPU down speed wise.\n\n\n\nOverall impression\n\n\n\nIt is a pretty decent experience. While not as polished as a Thinkpad there is a lot of potential with a bit of work and polishing. The laptop itself is not bad, the keyboard is responsive. The build quality is pretty solid. My only real complaint is the trackpad is stiff to click and sort of tiny. They seem to be a bit indifferent to non linux OSes running on the gear but that isnt anything new. I wont have any problems using it and is enough that when I work through this laptop, but I’m not sure at this stage if my next machine will be a System76 laptop, but they have impressed me enough to put them in the running when I go to look for my next portable machine but it hasn’t yet replaced the hole left in my heart by lenovo messing with the thinkpad.\n\n\n\n\n###Hardware accelerated AES/HMAC-SHA on octeons\n\nIn this commit, visa@ submitted code (disabled for now) to use built-in acceleration on octeon CPUs, much like AESNI for x86s.\n\nI decided to test tcpbench(1) and IPsec, before and after updating and enabling the octcrypto(4) driver.\n\nI didn't capture detailed perf stats from before the update, I had heard someone say that Edgerouter Lite boxes would only do some 6MBit/s over ipsec, so I set up a really simple ipsec.conf with ike esp from A to B leading to a policy of\n\nesp tunnel from A to B spi 0xdeadbeef auth hmac-sha2-256 enc aes\ngoing from one ERL to another (I collect octeons, so I have a bunch to test with) and let tcpbench run for a while on it. My numbers hovered around 7Mbit/s, which coincided with what I've heard, and also that most of the CPU gets used while doing it.\nThen I edited /sys/arch/octeon/conf/GENERIC, removed the # from octcrypto0 at mainbus0 and recompiled. Booted into the new kernel and got a octcrypto0 line in dmesg, and it was time to rock the ipsec tunnel again. The crypto algorithm and HMAC used by default on ipsec coincides nicely with the list of accelerated functions provided by the driver.\n\nBefore we get to tunnel traffic numbers, just one quick look at what systat pigs says while the ipsec is running at full steam:\n\n PID USER NAME CPU 20\\ 40\\ 60\\ 80\\ 100\\\n 58917 root crypto 52.25 #################\n 42636 root softnet 42.48 ##############\n (idle) 29.74 #########\n 1059 root tcpbench 24.22 #######\n 67777 root crynlk 19.58 ######\nSo this indicates that the load from doing ipsec and generating the traffic is somewhat nicely evened out over the two cores in the Edgerouter, and there's even some CPU left unused, which means I can actually ssh into it and have it usable. I have had it running for almost 2 days now, moving some 2.1TB over the tunnel.\nNow for the new and improved performance numbers:\n\n 204452123 4740752 37.402 100.00% \nConn: 1 Mbps: 37.402 Peak Mbps: 58.870 Avg Mbps: 37.402\n 204453149 4692968 36.628 100.00% \nConn: 1 Mbps: 36.628 Peak Mbps: 58.870 Avg Mbps: 36.628\n 204454167 5405552 42.480 100.00% \nConn: 1 Mbps: 42.480 Peak Mbps: 58.870 Avg Mbps: 42.480\n 204455188 5202496 40.804 100.00% \nConn: 1 Mbps: 40.804 Peak Mbps: 58.870 Avg Mbps: 40.804\n 204456194 5062208 40.256 100.00% \nConn: 1 Mbps: 40.256 Peak Mbps: 58.870 Avg Mbps: 40.256\n\nThe tcpbench numbers fluctuate up and down a bit, but the output is nice enough to actually keep tabs on the peak values. Peaking to 58.8MBit/s! Of course, as you can see, the average is lower but nice anyhow.\n\nA manyfold increase in performance, which is good enough in itself, but also moves the throughput from a speed that would make a poor but cheap gateway to something actually useful and decent for many home network speeds. Biggest problem after this gets enabled will be that my options to buy cheap used ERLs diminish.\n\n\n\n\n##Beastie Bits\n\n\nUsing FreeBSD Text Dumps\nllvm’s lld now the default linker for amd64 on FreeBSD\nAuthor Discoverability\nPledge and Unveil in OpenBSD {pdf}\nEuroBSDCon 2018 CFP Closes June 17, hurry up and get your submissions in\nJust want to attend, but need help getting to the conference? Applications for the Paul Schenkeveld travel grant accepted until June 15th\n\n\n\n\nTarsnap\n\n##Feedback/Questions\n\n\nCasey - ZFS on Digital Ocean\nJürgen - A Question\nKevin - Failover best practice\nDennis - SQL\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n","content_html":"TrueOS becoming a downstream fork with Trident, our BSDCan 2018 recap, HardenedBSD Foundation founding efforts, VPN with OpenIKED on OpenBSD, FreeBSD on a System76 Galago Pro, and hardware accelerated crypto on Octeons.
\n\n##Headlines##
\n###TrueOS to Focus on Core Operating System
\n\n\nThe TrueOS Project has some big plans in the works, and we want to take a minute and share them with you. Many have come to know TrueOS as the “graphical FreeBSD” that makes things easy for newcomers to the BSDs. Today we’re announcing that TrueOS is shifting our focus a bit to become a cutting-edge operating system that keeps all of the stability that you know and love from ZFS (OpenZFS) and FreeBSD, and adds additional features to create a fresh, innovative operating system. Our goal is to create a core-centric operating system that is modular, functional, and perfect for do-it-yourselfers and advanced users alike.
\n
\n\n\nTrueOS will become a downstream fork that will build on FreeBSD by integrating new software technologies like OpenRC and LibreSSL. Work has already begun which allows TrueOS to be used as a base platform for other projects, including JSON-based manifests, integrated Poudriere / pkg tools and much more. We’re planning on a six month release cycle to keep development moving and fresh, allowing us to bring you hot new features to ZFS, bhyve and related tools in a timely manner. This makes TrueOS the perfect fit to serve as the basis for building other distributions.
\n
\n\n\nSome of you are probably asking yourselves “But what if I want to have a graphical desktop?” Don’t worry! We’re making sure that everyone who knows and loves the legacy desktop version of TrueOS will be able to continue using a FreeBSD-based, graphical operating system in the future. For instance, if you want to add KDE, just use sudo pkg install kde and voila! You have your new shiny desktop. Easy right? This allows us to get back to our roots of being a desktop agnostic operating system. If you want to add a new desktop environment, you get to pick the one that best suits your use.
\n
\n\n\nWe know that some of you will still be looking for an out-of-the-box solution similar to legacy PC-BSD and TrueOS. We’re happy to announce that Project Trident will take over graphical FreeBSD development going forward. Not much is going to change in that regard other than a new name! You’ll still have Lumina Desktop as a lightweight and feature-rich desktop environment and tons of utilities from the legacy TrueOS toolchain like sysadm and AppCafe. There will be migration paths available for those that would like to move to other FreeBSD-based distributions like Project Trident or GhostBSD.
\n
\n\n\n\n\nWe look forward to this new chapter for TrueOS and hope you will give the new edition a spin! Tell us what you think about the new changes by leaving us a comment. Don’t forget you can ask us questions on our Twitter and be a part of our community by joining the new TrueOS Forums when they go live in about a week. Thanks for being a loyal fan of TrueOS.
\n
\n\n\nA: We were looking for a name that was unique, yet would still relate to the BSD community. Since Beastie (the FreeBSD mascot) is always pictured with a trident, it felt like that would be a great name.
\n
\n\n\nA: At the moment, Project Trident will continue sharing the TrueOS community forums and Telegram channels. We are currently evaluating dedicated options for support channels in the future.
\n
\n\n\nA: We are always looking for developers who want to join the project. If you’re not a developer you can still help, as a community project we will be more reliant on contributions from the community in the form of how-to guides and other user-centric documentation and support systems.
\n
\n\n\nA: Project Trident is sponsored by the community, from both individuals and corporations. iXsystems has stepped up as the first enterprise-level sponsor of the project, and has been instrumental in getting Project Trident up and running. Please visit the Sponsors page to see all the current sponsors.
\n
\n\n\nA: Several methods exist, from one time or recurring donations via Paypal to limited time swag t-shirt campaigns during the year. We are also looking into more alternative methods of support, so please visit the Sponsors page to see all the current methods of sponsorship.
\n
\n\n\nA: Yes, we will be totally open with how much money comes into the project and what it is spent on. Due to concerns of privacy, we will not identify individuals and their donation amounts unless they specifically request to be identified. We will release a monthly overview in/out ledger, so that community members can see where their money is going.
\n
Relationship with TrueOS
\nProject Trident does have very close ties to the TrueOS project, since most of the original Project Trident developers were once part of the TrueOS project before it became a distribution platform. For users of the TrueOS desktop, we have some additional questions and answers below.
\nQ: Do we need to be at a certain TrueOS install level/release to upgrade?
\n\n\n\nA: As long as you have a TrueOS system which has been updated to at least the 18.03 release you should be able to just perform a system update to be automatically upgraded to Project Trident.
\n
\n\n\nA: Project Trident is being led by prior members of the TrueOS desktop team. Ken and JT (development), Tim (documentation) and Rod (Community/Support). Since Project Trident is a community-first project, we look forward to working with new members of the team.
\n
iXsystems
\n\n###BSDCan
\n\n##News Roundup
\n###June HardenedBSD Foundation Update
\n\n\nWe at HardenedBSD are working towards starting up a 501©(3) not-for-profit organization in the USA. Setting up this organization will allow future donations to be tax deductible. We’ve made progress and would like to share with you the current state of affairs.
\n
\n\n\nWe have identified, sent invitations out, and received acceptance letters from six people who will serve on the HardenedBSD Foundation Board of Directors. You can find their bios below. In the latter half of June 2018 or the beginning half of July 2018, we will meet for the first time as a board and formally begin the process of creating the documentation needed to submit to the local, state, and federal tax services.
\n
\n\n\nHere’s a brief introduction to those who will serve on the board:
\n
W. Dean Freeman (Advisor): Dean has ten years of professional experience with deploying and security Unix and networking systems, including assessing systems security for government certification and assessing the efficacy of security products. He was introduced to Unix via FreeBSD 2.2.8 on an ISP shell account as a teenager. Formerly, he was the Snort port maintainer for FreeBSD while working in the Sourcefire VRT, and has contributed entropy-related patches to the FreeBSD and HardenedBSD projects – a topic on which he presented at vBSDCon 2017.
\nBen La Monica (Advisor): Ben is a Senior Technology Manager of Software Engineering at Morningstar, Inc and has been developing software for over 15 years in a variety of languages. He advocates open source software and enjoys tinkering with electronics and home automation.
\nGeorge Saylor (Advisor): George is a Technical Directory at G2, Inc. Mr. Saylor has over 28 years of information systems and security experience in a broad range of disciplines. His core focus areas are automation and standards in the event correlation space as well as penetration and exploitation of computer systems. Mr Saylor was also a co-founder of the OpenSCAP project.
\nVirginia Suydan (Accountant and general administrator): Accountant and general administrator for the HardenedBSD Foundation. She has worked with Shawn Webb for tax and accounting purposes for over six years.
\nShawn Webb (Director): Co-founder of HardenedBSD and all-around infosec wonk. He has worked and played in the infosec industry, doing both offensive and defensive research, for around fifteen years. He loves open source technologies and likes to frustrate the bad guys.
\nBen Welch (Advisor): Ben is currently a Security Engineer at G2, Inc. He graduated from Pennsylvania College of Technology with a Bachelors in Information Assurance and Security. Ben likes long walks, beaches, candlelight dinners, and attending various conferences like BSides and ShmooCon.
\n###Your own VPN with OpenIKED & OpenBSD
\n\n\n\n\nRemote connectivity to your home network is something I think a lot of people find desirable. Over the years, I’ve just established an SSH tunnel and use it as a SOCKS proxy, sending my traffic through that. It’s a nice solution for a “poor man’s VPN”, but it can be a bit clunky, and it’s not great having to expose SSH to the world, even if you make sure to lock everything down
\n
\n\n\nI set out the other day to finally do it properly. I’d come across this great post by Gordon Turner: OpenBSD 6.2 VPN Endpoint for iOS and macOS
\n
\n\n\nWhilst it was exactly what I was looking for, it outlined how to set up an L2TP VPN. Really, I wanted IKEv2 for performance and security reasons (I won’t elaborate on this here, if you’re curious about the differences, there’s a lot of content out on the web explaining this).
\n
\n\n\nThe client systems I’d be using have native support for IKEv2 (iOS, macOS, other BSD systems). But, I couldn’t find any tutorials in the same vein.
\n
\n\n\nSo, let’s get stuck in!
\n
\n\n\nThis guide will walk through the set up of an IKEv2 VPN using OpenIKED on OpenBSD. It will detail a “road warrior” configuration, and use a PSK (pre-shared-key) for authentication. I’m sure it can be easily adapted to work on any other platforms that OpenIKED is available on, but keep in mind my steps are specifically for OpenBSD.
\n
\n\n\nAs with all my home infrastructure, I crafted this set-up declaratively. So, I had the deployment of the VM setup in Terraform (deployed on my private Triton cluster), and wrote the configuration in Ansible, then tied them together using radekg/terraform-provisioner-ansible.
\n
\n\n\nOne of the reasons I love Ansible is that its syntax is very simplistic, yet expressive. As such, I feel it fits very well into explaining these steps with snippets of the playbook I wrote. I’ll link the full playbook a bit further down for those interested.
\n
DigitalOcean
\n\n###FreeBSD on a System76 Galago Pro
\n\n\n\n\nHey all, It’s been a while since I last posted but I thought I would hammer something out here. My most recent purchase was a System76 Galago Pro. I thought, afer playing with POP! OS a bit, is there any reason I couldn’t get BSD on this thing. Turns out the answer is no, no there isnt and it works pretty decently.
\n
\n\n\nTo get some accounting stuff out of the way I tested this all on FreeBSD Head and 11.1, and all of it is valid as of May 10, 2018. Head is a fast moving target so some of this is only bound to improve.
\n
The hardware
\nIntel Core i5 Gen 8
\nUHD Graphics 620
\n16 GB DDR4 Ram
\nRTL8411B PCI Express Card Reader
\nRTL8111 Gigabit ethernet controller
\nIntel HD Audio
\nSamsung SSD 960 PRO 512GB NVMe
\nThe caveats
\n\n\n\nThere are a few things that I cant seem to make work straight out of the box, and that is the SD Card reader, the backlight, and the audio is a bit finicky. Also the trackpad doesn’t respond to two finger scrolling. The wiki is mostly up to date, there are a few edits that need to be made still but there is a bug where I cant register an account yet so I haven’t made all the changes.
\n
\n\n\nIt works like any other Intel processor. Pstates and throttling work.
\n
\n\n\nThe boot menu sets itself to what looks like 1024x768, but works as you expect in a tiny window. The text console does the full 3200x1800 resolution, but the text is ultra tiny. There isnt a font for the console that covers hidpi screens yet. As for X Windows it requres the drm-kmod-next package. Once installed follow the directions from the package and it works with almost no fuss. I have it running on X with full intel acceleration, but it is running at it’s full 3200x1800 resolution, to scale that down just do xrandr --output eDP-1 --scale 0.5x0.5 it will blow it up to roughly 200%. Due to limitations with X windows and hidpi it is harder to get more granular.
\n
\n\n\nThe wireless uses the iwm module, as of right now it does not seem to automagically load right now. Adding iwm_load=“YES” will cause the module to load on boot and kldload iwm
\n
\n\n\nI seem to be getting about 5 hours out of the battery, but everything reports out of the box as expected. I could get more by throttling the CPU down speed wise.
\n
\n\n\nIt is a pretty decent experience. While not as polished as a Thinkpad there is a lot of potential with a bit of work and polishing. The laptop itself is not bad, the keyboard is responsive. The build quality is pretty solid. My only real complaint is the trackpad is stiff to click and sort of tiny. They seem to be a bit indifferent to non linux OSes running on the gear but that isnt anything new. I wont have any problems using it and is enough that when I work through this laptop, but I’m not sure at this stage if my next machine will be a System76 laptop, but they have impressed me enough to put them in the running when I go to look for my next portable machine but it hasn’t yet replaced the hole left in my heart by lenovo messing with the thinkpad.
\n
###Hardware accelerated AES/HMAC-SHA on octeons
\n\nIn this commit, visa@ submitted code (disabled for now) to use built-in acceleration on octeon CPUs, much like AESNI for x86s.\n\nI decided to test tcpbench(1) and IPsec, before and after updating and enabling the octcrypto(4) driver.\n\nI didn't capture detailed perf stats from before the update, I had heard someone say that Edgerouter Lite boxes would only do some 6MBit/s over ipsec, so I set up a really simple ipsec.conf with ike esp from A to B leading to a policy of\n\nesp tunnel from A to B spi 0xdeadbeef auth hmac-sha2-256 enc aes\ngoing from one ERL to another (I collect octeons, so I have a bunch to test with) and let tcpbench run for a while on it. My numbers hovered around 7Mbit/s, which coincided with what I've heard, and also that most of the CPU gets used while doing it.\nThen I edited /sys/arch/octeon/conf/GENERIC, removed the # from octcrypto0 at mainbus0 and recompiled. Booted into the new kernel and got a octcrypto0 line in dmesg, and it was time to rock the ipsec tunnel again. The crypto algorithm and HMAC used by default on ipsec coincides nicely with the list of accelerated functions provided by the driver.\n\nBefore we get to tunnel traffic numbers, just one quick look at what systat pigs says while the ipsec is running at full steam:\n\n PID USER NAME CPU 20\\ 40\\ 60\\ 80\\ 100\\\n 58917 root crypto 52.25 #################\n 42636 root softnet 42.48 ##############\n (idle) 29.74 #########\n 1059 root tcpbench 24.22 #######\n 67777 root crynlk 19.58 ######\nSo this indicates that the load from doing ipsec and generating the traffic is somewhat nicely evened out over the two cores in the Edgerouter, and there's even some CPU left unused, which means I can actually ssh into it and have it usable. I have had it running for almost 2 days now, moving some 2.1TB over the tunnel.\nNow for the new and improved performance numbers:\n\n 204452123 4740752 37.402 100.00% \nConn: 1 Mbps: 37.402 Peak Mbps: 58.870 Avg Mbps: 37.402\n 204453149 4692968 36.628 100.00% \nConn: 1 Mbps: 36.628 Peak Mbps: 58.870 Avg Mbps: 36.628\n 204454167 5405552 42.480 100.00% \nConn: 1 Mbps: 42.480 Peak Mbps: 58.870 Avg Mbps: 42.480\n 204455188 5202496 40.804 100.00% \nConn: 1 Mbps: 40.804 Peak Mbps: 58.870 Avg Mbps: 40.804\n 204456194 5062208 40.256 100.00% \nConn: 1 Mbps: 40.256 Peak Mbps: 58.870 Avg Mbps: 40.256\n\nThe tcpbench numbers fluctuate up and down a bit, but the output is nice enough to actually keep tabs on the peak values. Peaking to 58.8MBit/s! Of course, as you can see, the average is lower but nice anyhow.\n\nA manyfold increase in performance, which is good enough in itself, but also moves the throughput from a speed that would make a poor but cheap gateway to something actually useful and decent for many home network speeds. Biggest problem after this gets enabled will be that my options to buy cheap used ERLs diminish.\n
\n\n##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nOpenZFS and DTrace updates in NetBSD, NetBSD network security stack audit, Performance of MySQL on ZFS, OpenSMTP results from p2k18, legacy Windows backup to FreeNAS, ZFS block size importance, and NetBSD as router on a stick.
\n
##Headlines
\n###ZFS and DTrace update lands in NetBSD
\n\n\nmerge a new version of the CDDL dtrace and ZFS code. This changes the upstream vendor from OpenSolaris to FreeBSD, and this version is based on FreeBSD svn r315983.
\n
\n\n\nin addition to the 10 years of improvements from upstream, this version also has these NetBSD-specific enhancements:
\n\n
\n- dtrace FBT probes can now be placed in kernel modules.
\n- ZFS now supports mmap().
\n
###NetBSD network stack security audit
\n\n\n\n\nOver the last five months, hundreds of patches were committed to the source tree as a result of this work. Dozens of bugs were fixed, among which a good number of actual, remotely-triggerable vulnerabilities.
\n
\n\n\nChanges were made to strengthen the networking subsystems and improve code quality: reinforce the mbuf API, add many KASSERTs to enforce assumptions, simplify packet handling, and verify compliance with RFCs. This was done in several layers of the NetBSD kernel, from device drivers to L4 handlers.
\n
\nIn the course of investigating several bugs discovered in NetBSD, I happened to look at the network stacks of other operating systems, to see whether they had already fixed the issues, and if so how. Needless to say, I found bugs there too.
\n\n\nThe IPv6 Buffer Overflow: The overflow allowed an attacker to write one byte of packet-controlled data into ‘packet_storage+off’, where ‘off’ could be approximately controlled too. This allowed at least a pretty bad remote DoS/Crash
\n
\nThe IPsec Infinite Loop: When receiving an IPv6-AH packet, the IPsec entry point was not correctly computing the length of the IPv6 suboptions, and this, before authentication. As a result, a specially-crafted IPv6 packet could trigger an infinite loop in the kernel (making it unresponsive). In addition this flaw allowed a limited buffer overflow - where the data being written was however not controllable by the attacker.
\nThe IPPROTO Typo: While looking at the IPv6 Multicast code, I stumbled across a pretty simple yet pretty bad mistake: at one point the Pim6 entry point would return IPPROTO_NONE instead of IPPROTO_DONE. Returning IPPROTO_NONE was entirely wrong: it caused the kernel to keep iterating on the IPv6 packet chain, while the packet storage was already freed.
\nThe PF Signedness Bug: A bug was found in NetBSD’s implementation of the PF firewall, that did not affect the other BSDs. In the initial PF code a particular macro was used as an alias to a number. This macro formed a signed integer. NetBSD replaced the macro with a sizeof(), which returns an unsigned result.
\nThe NPF Integer Overflow: An integer overflow could be triggered in NPF, when parsing an IPv6 packet with large options. This could cause NPF to look for the L4 payload at the wrong offset within the packet, and it allowed an attacker to bypass any L4 filtering rule on IPv6.
\nThe IPsec Fragment Attack: I noticed some time ago that when reassembling fragments (in either IPv4 or IPv6), the kernel was not removing the M_PKTHDR flag on the secondary mbufs in mbuf chains. This flag is supposed to indicate that a given mbuf is the head of the chain it forms; having the flag on secondary mbufs was suspicious.
\nWhat Now: Not all protocols and layers of the network stack were verified, because of time constraints, and also because of unexpected events: the recent x86 CPU bugs, which I was the only one able to fix promptly. A todo list will be left when the project end date is reached, for someone else to pick up. Me perhaps, later this year? We’ll see.
\nThis security audit of NetBSD’s network stack is sponsored by The NetBSD Foundation, and serves all users of BSD-derived operating systems. The NetBSD Foundation is a non-profit organization, and welcomes any donations that help continue funding projects of this kind.
DigitalOcean
\n\n\n\n\n\n\nI used sysbench to create a table of 10M rows and then, using export/import tablespace, I copied it 329 times. I ended up with 330 tables for a total size of about 850GB. The dataset generated by sysbench is not very compressible, so I used lz4 compression in ZFS. For the other ZFS settings, I used what can be found in my earlier ZFS posts but with the ARC size limited to 1GB. I then used that plain configuration for the first benchmarks. Here are the results with the sysbench point-select benchmark, a uniform distribution and eight threads. The InnoDB buffer pool was set to 2.5GB.
\n
\nIn both cases, the load is IO bound. The disk is doing exactly the allowed 3000 IOPS. The above graph appears to be a clear demonstration that XFS is much faster than ZFS, right? But is that really the case? The way the dataset has been created is extremely favorable to XFS since there is absolutely no file fragmentation. Once you have all the files opened, a read IOP is just a single fseek call to an offset and ZFS doesn’t need to access any intermediate inode. The above result is about as fair as saying MyISAM is faster than InnoDB based only on table scan performance results of unfragmented tables and default configuration. ZFS is much less affected by the file level fragmentation, especially for point access type.
\n\n\nZFS stores the files in B-trees in a very similar fashion as InnoDB stores data. To access a piece of data in a B-tree, you need to access the top level page (often called root node) and then one block per level down to a leaf-node containing the data. With no cache, to read something from a three levels B-tree thus requires 3 IOPS.
\n
\n\n\nThe extra IOPS performed by ZFS are needed to access those internal blocks in the B-trees of the files. These internal blocks are labeled as metadata. Essentially, in the above benchmark, the ARC is too small to contain all the internal blocks of the table files’ B-trees. If we continue the comparison with InnoDB, it would be like running with a buffer pool too small to contain the non-leaf pages. The test dataset I used has about 600MB of non-leaf pages, about 0.1% of the total size, which was well cached by the 3GB buffer pool. So only one InnoDB page, a leaf page, needed to be read per point-select statement.
\n
\n\n\nTo correctly set the ARC size to cache the metadata, you have two choices. First, you can guess values for the ARC size and experiment. Second, you can try to evaluate it by looking at the ZFS internal data. Let’s review these two approaches.
\n
\n\n\nYou’ll read/hear often the ratio 1GB of ARC for 1TB of data, which is about the same 0.1% ratio as for InnoDB. I wrote about that ratio a few times, having nothing better to propose. Actually, I found it depends a lot on the recordsize used. The 0.1% ratio implies a ZFS recordsize of 128KB. A ZFS filesystem with a recordsize of 128KB will use much less metadata than another one using a recordsize of 16KB because it has 8x fewer leaf pages. Fewer leaf pages require less B-tree internal nodes, hence less metadata. A filesystem with a recordsize of 128KB is excellent for sequential access as it maximizes compression and reduces the IOPS but it is poor for small random access operations like the ones MySQL/InnoDB does.
\n
\n\n\nI was reluctant to grow the ARC to 7GB, which was nearly half the overall system memory. At best, the ZFS performance would only match XFS. A larger InnoDB page size would increase the CPU load for decompression on an instance with only two vCPUs; not great either. The last option, the L2ARC, was the most promising.
\n
\n\n\nZFS is much more complex than XFS and EXT4 but, that also means it has more tunables/options. I used a simplistic setup and an unfair benchmark which initially led to poor ZFS results. With the same benchmark, very favorable to XFS, I added a ZFS L2ARC and that completely reversed the situation, more than tripling the ZFS results, now 66% above XFS.
\n
\n\n\nWe have seen in this post why the general perception is that ZFS under-performs compared to XFS or EXT4. The presence of B-trees for the files has a big impact on the amount of metadata ZFS needs to handle, especially when the recordsize is small. The metadata consists mostly of the non-leaf pages (or internal nodes) of the B-trees. When properly cached, the performance of ZFS is excellent. ZFS allows you to optimize the use of EBS volumes, both in term of IOPS and size when the instance has fast ephemeral storage devices. Using the ephemeral device of an i3.large instance for the ZFS L2ARC, ZFS outperformed XFS by 66%.
\n
TL;DR:\nOpenBSD #p2k18 hackathon took place at Epitech in Nantes.\nI was organizing the hackathon but managed to make progress on OpenSMTPD.\nAs mentioned at EuroBSDCon the one-line per rule config format was a design error.\nA new configuration grammar is almost ready and the underlying structures are simplified.\nRefactor removes ~750 lines of code and solves _many_ issues that were side-effects of the design error.\nNew features are going to be unlocked thanks to this.\n
\n\n\n\n\nOpenSMTPD started ten years ago out of dissatisfaction with other solutions, mainly because I considered them way too complex for me not to get things wrong from time to time.
\n
\nThe initial configuration format was very different, I was inspired by pyr@’s hoststated, which eventually became relayd, and designed my configuration format with blocks enclosed by brackets.
\nWhen I first showed OpenSMTPD to pyr@, he convinced me that PF-like one-line rules would be awesome, and it was awesome indeed.
\nIt helped us maintain our goal of simple configuration files, it helped fight feature creeping, it helped us gain popularity and become a relevant MTA, it helped us get where we are now 10 years later.
\nThat being said, I believe this was a design error. A design error that could not have been predicted until we hit the wall to understand WHY this was an error. One-line rules are semantically wrong, they are SMTP wrong, they are wrong.
\nOne-line rules are making the entire daemon more complex, preventing some features from being implemented, making others more complex than they should be, they no longer serve our goals.
\nTo get to the point: we should move to two-line rules :-)
Anatomy of a design error
\nOpenSMTPD started ten years ago out of dissatisfaction with other solutions, mainly because I considered them way too complex for me not to get things wrong from time to time.
The initial configuration format was very different, I was inspired by pyr@’s hoststated, which eventually became relayd, and designed my configuration format with blocks enclosed by brackets.
\n\nWhen I first showed OpenSMTPD to pyr@, he convinced me that PF-like one-line rules would be awesome, and it was awesome indeed.
\n\nIt helped us maintain our goal of simple configuration files, it helped fight feature creeping, it helped us gain popularity and become a relevant MTA, it helped us get where we are now 10 years later.
\n\nThat being said, I believe this was a design error. A design error that could not have been predicted until we hit the wall to understand WHY this was an error. One-line rules are semantically wrong, they are SMTP wrong, they are wrong.
\n\nOne-line rules are making the entire daemon more complex, preventing some features from being implemented, making others more complex than they should be, they no longer serve our goals.
\n\nTo get to the point: we should move to two-line rules :-)
\n\n\n\n\nOpenSMTPD decides to accept or reject messages based on one-line rules such as:
\n
accept from any for domain poolp.org deliver to mbox
\n\n\nWhich can essentially be split into three units:
\n
\n\n\nTo ensure that we meet the requirements of the transactions, the matching must be performed during the SMTP transaction before we take a decision for the recipient.
\n
\nGiven that the rule is atomic, that it doesn’t have an identifier and that the action is part of it, the two only ways to make sure we can remember the action to take later on at delivery time is to either:
\n\n\nThe first solution, which we’ve been using for a decade, was to save the action within the envelope and kind of carve it in stone. This works fine… however it comes with the downsides that errors fixed in configuration files can’t be caught up by envelopes, that delivery action must be validated way ahead of time during the SMTP transaction which is much trickier, that the parsing of delivery methods takes place as the _smtpd user rather than the recipient user, and that envelope structures that are passed all over OpenSMTPD carry delivery-time informations, and more, and more, and more. The code becomes more complex in general, less safe in some particular places, and some areas are nightmarish to deal with because they have to deal with completely unrelated code that can’t be dealt with later in the code path.
\n
\n\n\nThe second solution can’t be done. An envelope may be the result of nested rules, for example an external client, hitting an alias, hitting a user with a .forward file resolving to a user. An envelope on disk may no longer match any rule or it may match a completely different rule If we could ensure that it matched the same rule, evaluating the ruleset may spawn new envelopes which would violate the transaction. Trying to imagine how we could work around this leads to more and more and more RFC violations, incoherent states, duplicate mails, etc…
\n
\n\n\nThere is simply no way to deal with this with atomic rules, the matching and the action must be two separate units that are evaluated at two different times, failure to do so will necessarily imply that you’re either using our first solution and all its downsides, or that you are currently in a world of pain trying to figure out why everything is burning around you. The minute the action is written to an on-disk envelope, you have failed.
\n
\n\n\nA proper ruleset must define a set of matching patterns resolving to an action identifier that is carved in stone, AND a set of named action set that is resolved dynamically at delivery time.
\n
Break
\n\n##News Roundup
\n###Backing up a legacy Windows machine to a FreeNAS with rsync
\n\n\nI have some old Windows servers (10 years and counting) and I have been using rsync to back them up to my FreeNAS box. It has been working great for me.
\n
\n\n\nFirst of all, I do have my Windows servers backup in virtualized format. However, those are only one-time snapshops that I run once in a while. These are classic ASP IIS web servers that I can easily put up on a new VM. However, many of these legacy servers generate gigabytes of data a day in their repositories. Running VM conversion daily is not ideal.
\n
\n\n\nMy solution was to use some sort of rsync solution just for the data repos. I’ve tried some applications that didn’t work too well with Samba shares and these old servers have slow I/O. Copying files to external sata or usb drive was not ideal. We’ve moved on from Windows to Linux and do not have any Windows file servers of capacity to provide network backups. Hence, I decided to use Delta Copy with FreeNAS. So here is a little write up on how to set it up. I have 4 Windows 2000 servers backing up daily with this method.
\n
\n\n\nFirst, download Delta Copy and install it. It is open-source and pretty much free. It is basically a wrapper for cygwin’s rsync. When you install it, it will ask you to install the Server services which allows you to run it as a Rsync server on Windows. You don’t need to do this. Instead, you will be just using the Delta Copy Client application. But before we do that, we will need to configure our Rsync service for our Windows Clients on FreeNAS.
\n
\n\n\nThere you have it. Windows rsync to FreeNAS using DeltaCopy.
\n
\nThe nice thing about FreeNAS is you don’t have to modify /etc/rsyncd.conf files. Everything can be done in the web admin.
iXsystems
\n\n###How to write ATF tests for NetBSD
\n\n\n\n\nI have recently started contributing to the amazing NetBSD foundation. I was thinking of trying out a new OS for a long time. Switching to the NetBSD OS has been a fun change.
\n
\n\n\nMy first contribution to the NetBSD foundation was adding regression tests for the Address Sanitizer (ASan) in the Automated Testing Framework(ATF) which NetBSD has. I managed to complete it with the help of my really amazing mentor Kamil. This post is gonna be about the ATF framework that NetBSD has and how to you can add multiple tests with ease.
\n
\n\n\nIn ATF tests we will basically be talking about test programs which are a suite of test cases for a specific application or program.
\n
\n\n\nThere are a variety of commands that the atf suite offers. These include :
\n
atf-check: The versatile command that is a vital part of the checking process. man page
\natf-run: Command used to run a test program. man page
\natf-fail: Report failure of a test case.
\natf-report: used to pretty print the atf-run. man page
\natf-set: To set atf test conditions.
\nWe will be taking a better look at the syntax and usage later.
\nLet’s start with the Basics
\n\n\n\nThe ATF testing framework comes preinstalled with a default NetBSD installation. It is used to write tests for various applications and commands in NetBSD. One can write the Test programs in either the C language or in shell script. In this post I will be dealing with the Bash part.
\n
###The Importance of ZFS Block Size
\n\n\n\n\nOne of the important tunables in ZFS is the recordsize (for normal datasets) and volblocksize (for zvols). These default to 128KB and 8KB respectively.
\n
\nAs I understand it, this is the unit of work in ZFS. If you modify one byte in a large file with the default 128KB record size, it causes the whole 128KB to be read in, one byte to be changed, and a new 128KB block to be written out.
\nAs a result, the official recommendation is to use a block size which aligns with the underlying workload: so for example if you are using a database which reads and writes 16KB chunks then you should use a 16KB block size, and if you are running VMs containing an ext4 filesystem, which uses a 4KB block size, you should set a 4KB block size
\nYou can see it has a 16GB total file size, of which 8.5G has been touched and consumes space - that is, it’s a “sparse” file. The used space is also visible by looking at the zfs filesystem which this file resides in
\nThen I tried to copy the image file whilst maintaining its “sparseness”, that is, only touching the blocks of the zvol which needed to be touched. The original used only 8.42G, but the copy uses 14.6GB - almost the entire 16GB has been touched! What’s gone wrong?
\nI finally realised that the difference between the zfs filesystem and the zvol is the block size. I recreated the zvol with a 128K block size
\nThat’s better. The disk usage of the zvol is now exactly the same as for the sparse file in the filesystem dataset
###Using a Raspberry Pi 2 as a Router on a Stick Starring NetBSD
\n\n\n\n\nA few weeks ago I set about upgrading my feeble networking skills by playing around with a Cisco 2970 switch. I set up a couple of VLANs and found the urge to set up a router to route between them. The 2970 isn’t a modern layer 3 switch so what am I to do?
\n
\n\n\nWhy not make use of the Raspberry Pi 2 that I’ve never used and put it to some good use as a ‘router on a stick’.
\n
\n\n\nI could install a Linux based OS as I am quite familiar with it but where’s the fun in that? In my home lab I use SmartOS which by the way is a shit hot hypervisor but as far as I know there aren’t any Illumos distributions for the Raspberry Pi. On the desktop I use Solus OS which is by far the slickest Linux based OS that I’ve had the pleasure to use but Solus’ focus is purely desktop. It’s looking like BSD then!
\n
\n\n\nI believe FreeBSD is renowned for it’s top notch networking stack and so I wrote to the BSDNow show on Jupiter Broadcasting for some help but it seems that the FreeBSD chaps from the show are off on a jolly to some BSD conference or another(love the show by the way).
\n
\n\n\nIt looks like me and the luvverly NetBSD are on a date this Saturday. I’ve always had a secret love for NetBSD. She’s a beautiful, charming and promiscuous lover(looking at the supported architectures) and I just can’t stop going back to her despite her misgivings(ahem, zfs). Just my type of grrrl!
\n
\n\n\nLet’s crack on…
\n
##Beastie Bits
\n\nTarsnap
\n\n##Feedback/Questions
\n\nDragonflyBSD release 5.2.1 is here, BPF kernel exploit writeup, Remote Debugging the running OpenBSD kernel, interview with Patrick Mooney, FreeBSD buildbot setup in a jail, dumping your USB, and 5 years of gaming on FreeBSD.
\n\n\n Meltdown and Spectre mitigation support\n Meltdown isolation and spectre mitigation support added. Meltdown mitigation is automatically enabled for all Intel cpus. Spectre mitigation must be enabled manually via sysctl if desired, using sysctls machdep.spectremitigation and machdep.meltdownmitigation.\n HAMMER2\n H2 has received a very large number of bug fixes and performance improvements. We can now recommend H2 as the default root filesystem in non-clustered mode.\n Clustered support is not yet available.\n ipfw Updates\n Implement state based \"redirect\", i.e. without using libalias.\n ipfw now supports all possible ICMP types.\n Fix ICMPMAXTYPE assumptions (now 40 as of this release).\n Improved graphics support\n The drm/i915 kernel driver has been updated to support Intel Coffeelake GPUs\n Add 24-bit pixel format support to the EFI frame buffer code.\n Significantly improve fbio support for the \"scfb\" XOrg driver. This allows EFI frame buffers to be used by X in situations where we do not otherwise support the GPU.\n Partly implement the FBIOBLANK ioctl for display powersaving.\n Syscons waits for drm modesetting at appropriate places, avoiding races.
\n\n\nNote: While this bug is primarily interesting for exploitation on the PS4, this bug can also potentially be exploited on other unpatched platforms using FreeBSD if the attacker has read/write permissions on /dev/bpf, or if they want to escalate from root user to kernel code execution. As such, I've published it under the \"FreeBSD\" folder and not the \"PS4\" folder.
\n
\n\n\nWelcome to the kernel portion of the PS4 4.55FW full exploit chain write-up. This bug was found by qwerty, and is fairly unique in the way it's exploited, so I wanted to do a detailed write-up on how it worked. The full source of the exploit can be found here. I've previously covered the webkit exploit implementation for userland access here.
\n
\n\n\nInterestingly, this bug is actually a FreeBSD bug and was not (at least directly) introduced by Sony code. While this is a FreeBSD bug however, it's not very useful for most systems because the /dev/bpf device driver is root-owned, and the permissions for it are set to 0600 (meaning owner has read/write privileges, and nobody else does) - though it can be used for escalating from root to kernel mode code execution. However, let’s take a look at the make_dev() call inside the PS4 kernel for /dev/bpf (taken from a 4.05 kernel dump).
\n
\nseg000:FFFFFFFFA181F15B lea rdi, unk_FFFFFFFFA2D77640\nseg000:FFFFFFFFA181F162 lea r9, aBpf ; \"bpf\"\nseg000:FFFFFFFFA181F169 mov esi, 0\nseg000:FFFFFFFFA181F16E mov edx, 0\nseg000:FFFFFFFFA181F173 xor ecx, ecx\nseg000:FFFFFFFFA181F175 mov r8d, 1B6h\nseg000:FFFFFFFFA181F17B xor eax, eax\nseg000:FFFFFFFFA181F17D mov cs:qword_FFFFFFFFA34EC770, 0\nseg000:FFFFFFFFA181F188 call make_dev\n
\n\n\nWe see UID 0 (the UID for the root user) getting moved into the register for the 3rd argument, which is the owner argument. However, the permissions bits are being set to 0x1B6, which in octal is 0666. This means anyone can open /dev/bpf with read/write privileges. I’m not sure why this is the case, qwerty speculates that perhaps bpf is used for LAN gaming. In any case, this was a poor design decision because bpf is usually considered privileged, and should not be accessible to a process that is completely untrusted, such as WebKit. On most platforms, permissions for /dev/bpf will be set to 0x180, or 0600.
\n
\n\n\nThe class of the bug abused in this exploit is known as a \"race condition\". Before we get into bug specifics, it's important for the reader to understand what race conditions are and how they can be an issue (especially in something like a kernel). Often in complex software (such as a kernel), resources will be shared (or \"global\"). This means other threads could potentially execute code that will access some resource that could be accessed by another thread at the same point in time. What happens if one thread accesses this resource while another thread does without exclusive access? Race conditions are introduced.
\n \nRace conditions are defined as possible scenarios where events happen in a sequence different than the developer intended which leads to undefined behavior. In simple, single-threaded programs, this is not an issue because execution is linear. In more complex programs where code can be running in parallel however, this becomes a real issue. To prevent these problems, atomic instructions and locking mechanisms were introduced. When one thread wants to access a critical resource, it will attempt to acquire a \"lock\". If another thread is already using this resource, generally the thread attempting to acquire the lock will wait until the other thread is finished with it. Each thread must release the lock to the resource after they're done with it, failure to do so could result in a deadlock.
\n \nWhile locking mechanisms such as mutexes have been introduced, developers sometimes struggle to use them properly. For example, what if a piece of shared data gets validated and processed, but while the processing of the data is locked, the validation is not? There is a window between validation and locking where that data can change, and while the developer thinks the data has been validated, it could be substituted with something malicious after it is validated, but before it is used. Parallel programming can be difficult, especially when, as a developer, you also want to factor in the fact that you don't want to put too much code in between locking and unlocking as it can impact performance.
\n
iXsystems
\n\n\n $ qemu-img create -f raw disk.raw 5G\n $ qemu-system-x8664 -m 256M \\\n -drive format=raw,file=install63.fs \\\n -drive format=raw,file=disk.raw\n +> Custom Kernel\n +> To debug the kernel, we need a version of the kernel with debugging symbols and for that we have to recompile it first. The process is documented at Building the System from Source:\n ...\n +> Then we can copy the bsd kernel to the guest machine and keep the bsd.gdb on the host to start the remote debugging via gdb.\n +> Remote debugging kernel\n +> Now it's to time to boot the guest with the new custom kernel. Remember that the -s argument enables the gdb server on qemu on localhost port 1234 by default:\n $ qemu-system-x8664 -m 256M -s \\\n -net nic -net user \\\n -drive format=raw,file=install63.fs \\\n +> Now to finally attach to the running kernel:
\n\n\nIn this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism \"jails\". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here.
\n
Table of contents
\n\nChoosing host operating system and version for buildbot
\n\n\nWe choose the released version of FreeBSD (11.1-RELEASE at the moment). There is no particular reason for it, and as a matter of fact buildbot as a Python-based server is very cross-platform; therefore the underlying OS platform and version should not make a large difference.
\n \nIt will make a difference for what you do with buildbot, however. For instance, poudriere is the de-facto standard for building packages from source on FreeBSD. Builds run in jails which may be any FreeBSD base system version older or equal to the host’s version (reason will be explained below). In other words, if the host is FreeBSD 11.1, build jails created by poudriere could e.g. use 9.1, 10.3, 11.0, 11.1, but potentially not version 12 or newer because of incompatibilities with the host’s kernel (jails do not run their own kernel as full virtual machines do). To not prolong this article over the intended scope, the details of which nice things could be done or automated with buildbot are not covered.
\n \nPackage names on the FreeBSD platform are independent of the OS version, since external software (as in: not part of base system) is maintained in FreeBSD ports. So, if your chosen FreeBSD version (here: 11) is still officially supported, the packages mentioned in this post should work. In the unlikely event of package name changes before you read this article, you should be able to find the actual package names like pkg search buildbot.
\n \nOther operating systems like the various Linux distributions will use different package names but might also offer buildbot pre-packaged. If not, the buildbot installation manual offers steps to install it manually. In such case, the downside is that you will have to maintain and update the buildbot modules outside the stability and (semi-)automatic updates of your OS packages.
\n
DigitalOcean
\n\n\n\n\nOne of the many new features of OpenBSD 6.3 is the possibility to dump USB traffic to userland via bpf(4). This can be done with tcpdump(8) by specifying a USB bus as interface:
\n
```
\n\ntcpdump: listening on usb0, link-type USBPCAP\n12:28:03.317945 bus 0 < addr 1: ep1 intr 2\n 0000: 0400 ..
\n\n12:28:03.318018 bus 0 > addr 1: ep0 ctrl 8\n 0000: 00a3 0000 0002 0004 00 .........
\n[...]\n```
\n\n\nAs you might have noted I decided to implement the existing USBPcap capture format. A capture format is required because USB packets do not include all the necessary information to properly interpret them. I first thought I would implement libpcap's DLTUSB but then I quickly realize that this was not a standard. It is instead a FreeBSD specific format which has been since then renamed DLTUSBFREEBSD.\n But I didn't want to embrace xkcd #927, so I look at the existing formats: DLTUSBFREEBSD, DLTUSBLINUX, DLTUSBLINUXMMAPPED, DLTUSBDARWIN and DLT_USBPCAP. I was first a bit sad to see that nobody could agree on a common format then I moved on and picked the simplest one: USBPcap.\n Implementing an already existing format gives us out-of-box support for all the tools supporting it. That's why having common formats let us share our energy. In the case of USBPcap it is already supported by Wireshark, so you can already inspect your packet graphically. For that you need to first capture raw packets:
\n
```
\n\ntcpdump: listening on usb0, link-type USBPCAP\n^C\n208 packets received by filter\n0 packets dropped by kernel\n```
\n\n\n\n\nUSB packets can be quite big, that's why I'm not using tcpdump(8)'s default packet size. In this case, I want to make sure I can dump the complete uaudio(4) frames.\n It is important to say that what is dumped to userland is what the USB stack sees. Packets sent on the wire might differ, especially when it comes to retries and timing. So this feature is not here to replace any USB analyser, however I hope that it will help people understand how things work and what the USB stack is doing. Even I found some interesting timing issues while implementing isochronous support.
\n
\n\n\nAs soon as you're there you can enable an httpd(8) daemon, it's already installed on OpenBSD, you just need to configure it:
\n
www# vi /etc/httpd.conf
```\nserver \"www.example.com\" {\n listen on * port 80\n root \"/htdocs/www.example.com\"\n}
\n\nserver \"example.com\" {\n listen on * port 80\n block return 301 \"http://www.example.com$REQUEST_URI\"\n}\n```
\n\nwww# mkdir -p /var/www/htdocs/www.example.com
\nwww# httpd -n\nconfiguration ok\n
\nwww# rcctl enable httpd\nwww# rcctl start httpd\n
Publish your website
Copy your website content into /var/www/htdocs/www.example.com and then test it your web browser.
http://XXX.XXX.XXX.XXX/
\n\n\nYour web server should be up and running.
\n
\n\n\nIf there is another HTTPS server using this domain, configure that server to redirect all HTTPS requests to HTTP.
\n \nNow as your new server is ready you can update DNS records accordingly.
\n
\n example.com. 300 IN A XXX.XXX.XXX.XXX\nwww.example.com. 300 IN A XXX.XXX.XXX.XXX\n
$ dig example.com www.example.com
Check IP addresses it answer sections. If they are correct, you should be able to access your new web server by its domain name.
\n\n\nFor, quite literally a year or more, KMail and Akonadi on FreeBSD have been only marginally useful, at best. KDE4 era KMail was pretty darn good, but everything after that has had a number of FreeBSD users tearing out their hair. Sure, you can go to Trojitá, which has its own special problems and is generally “meh”, or bail out entirely to webmail, but .. KMail is a really great mail client when it works. Which, on Linux desktops, is nearly always, and on FreeBSD, is was nearly never.
\n \nI looked at it with Dan and Volker last summer, briefly, and we got not much further than “hmm”. There’s a message about “The world is going to end!” which hardly makes sense, it means that a message has been truncated or corrupted while traversing a UNIX domain socket.
\n \nNow Alexandre Martins — praise be! — has wandered in with a likely solution. KDE Bug 381850 contains a suggestion, which deserves to be publicised (and tested):
\n
sysctl net.local.stream.recvspace=65536
\nsysctl net.local.stream.sendspace=65536
\n\n\nThe default FreeBSD UNIX local socket buffer space is 8kiB. Bumping the size up to 64kiB — which matches the size that Linux has by default — suddenly makes KMail and Akonadi shine again. No other changes, no recompiling, just .. bump the sysctls (perhaps also in /etc/sysctl.conf) and KMail from Area51 hums along all day without ending the world.
\n \nSince changing this value may have other effects, and Akonadi shouldn’t be dependent on a specific buffer size anyway, I’m looking into the Akonadi code (encouraged by Dan) to either automatically size the socket buffers, or to figure out where in the underlying code the assumption about buffer size lives. So for now, sysctl can make KMail users on FreeBSD happy, and later we hope to have things fully automatic (and if that doesn’t pan out, well, pkg-message exists).
\n \nPS. Modern KDE PIM applications — Akonadi, KMail — which live in the deskutils/ category of the official FreeBSD ports were added to the official tree April 10th, so you can get your fix now from the official tree.
\n
Tarsnap ad
\n\nFreeBSD internship learnings, exciting developments coming to FreeBSD, running FreeNAS on DigitalOcean, Network Manager control for OpenBSD, OpenZFS User Conference Videos are here and batch editing files with ed.
\n\n\n\n\nHi, my name is Mitchell Horne. I am a computer engineering student at the University of Waterloo, currently in my third year of studies, and fortunate to have been one of the FreeBSD Foundation’s co-op students this past term (January to April). During this time I worked under Ed Maste, in the Foundation’s small Kitchener office, along with another co-op student Arshan Khanifar. My term has now come to an end, and so I’d like to share a little bit about my experience as a newcomer to FreeBSD and open-source development.
\n \nI’ll begin with some quick background — and a small admission of guilt. I have been an open-source user for a large part of my life. When I was a teenager I started playing around with Linux, which opened my eyes to the wider world of free software. Other than some small contributions to GNOME, my experience has been mostly as an end user; however, the value of these projects and the open-source philosophy was not lost on me, and is most of what motivated my interest in this position. Before beginning this term I had no personal experience with any of the BSDs, although I knew of their existence and was extremely excited to receive the position. I knew it would be a great opportunity for growth, but I must confess that my naivety about FreeBSD caused me to make the silent assumption that this would be a form of compromise — a stepping stone that would eventually allow me to work on open-source projects that are somehow “greater” or more “legitimate”. After four months spent immersed in this project I have learned how it operates, witnessed its community, and learned about its history. I am happy to admit that I was completely mistaken. Saying it now seems obvious, but FreeBSD is a project with its own distinct uses, goals, and identity. For many there may exist no greater opportunity than to work on FreeBSD full time, and with what I know now I would have a hard time coming up with a project that is more “legitimate”.
\n
\n\n\nIn all cases, the work I submitted this term was reviewed by no less than two people before being committed. The feedback and criticism I received was always both constructive and to the point, and it commented on everything from high-level ideas to small style issues. I appreciate having these thorough reviews in place, since I believe it ultimately encourages people to accept only their best work. It is indicative of the high quality that already exists within every aspect of this project, and this commitment to quality is something that should continue to be honored as a core value. As I’ve discovered in some of my previous work terms, it is all too easy cut corners in the name of a deadline or changing priorities, but the fact that FreeBSD doesn’t need to make these types of compromises is a testament to the power of free software.
\n \nIt’s a small thing, but the quality and completeness of the FreeBSD documentation was hugely helpful throughout my term. Everything you might need to know about utilities, library functions, the kernel, and more can be found in a man page; and the handbook is a great resource as both an introduction to the operating system and a reference. I only wish I had taken some time earlier in the term to explore the different documents more thoroughly, as they cover a wide range of interesting and useful topics. The effort people put into writing and maintaining FreeBSD’s documentation is easy to overlook, but its value cannot be overstated.
\n
\n\n\nAlthough there was a lot I enjoyed, there were certainly many struggles I faced throughout the term, and lessons to be learned from them. I expect that some of issues I faced may be specific to FreeBSD, while others may be common to open-source projects in general. I don’t have enough experience to speculate on which is which, so I will leave this to the reader.
\n \nThe first lesson can be summed up simply: you have to advocate for your own work. FreeBSD is made up in large part by volunteer efforts, and in many cases there is more work to go around than people available to do it. A consequence of this is that there will not be anybody there to check up on you. Even in my position where I actually had a direct supervisor, Ed often had his plate full with so many other things that the responsibility to find someone to look at my work fell to me. Admittedly, a couple of smaller changes I worked on got left behind or stuck in review simply because there wasn’t a clear person/place to reach out to.
\n \nI think this is both a barrier of entry to FreeBSD and a mental hurdle that I needed to get over. If there’s a change you want to see included or reviewed, then you may have to be the one to push for it, and there’s nothing wrong with that. Perhaps this process should be easier for newcomers or infrequent contributors (the disconnect between Bugzilla and Phabricator definitely leaves a lot to be desired), but we also have to be aware that this simply isn’t the reality right now. Getting your work looked at may require a little bit more self-motivation, but I’d argue that there are much worse problems a project like FreeBSD could have than this.
\n \nI understand this a lot better now, but it is still something I struggle with. I’m not naturally the type of person who easily connects with others or asks for help, so I see this as an area for future growth rather than simply a struggle I encountered and overcame over the course of this work term. Certainly it is an important skill to understand the value of your own work, and equally important is the ability to communicate that value to others.
\n \nI also learned the importance of starting small. My first week or two on the job mainly involved getting set up and comfortable with the workflow. After this initial stage, I began exploring the project and found myself overwhelmed by its scale. With so many possible areas to investigate, and so much work happening at once, I felt quite lost on where to begin. Many of the potential projects I found were too far beyond my experience level, and most small bugs were picked up and fixed quickly by more experienced contributors before I could even get to them.
\n \nIt’s easy to make the mistake that FreeBSD is made up solely of a few rock-star committers that do everything. This is how it appears at face-value, as reading through commits, bug reports, and mailing lists yields a few of the same names over and over. The reality is that just as important are the hundreds of users and infrequent contributors who take the time to submit bug reports, patches, or feedback. Even though there are some people who would fall under the umbrella of a rock-star committer, they didn’t get there overnight. Rather, they have built their skills and knowledge through many years of involvement in FreeBSD and similar projects.
\n \nAs a student coming into this project and having high expectations of myself, it was easy to set the bar too high by comparing myself against those big committers, and feel that my work was insignificant, inadequate, and simply too infrequent. In reality, there is no reason I should have felt this way. In a way, this comparison is disrespectful to those who have reached this level, as it took them a long time to get there, and it’s a humbling reminder that any skill worth learning requires time, patience, and dedication. It is easy to focus on an end product and simply wish to be there, but in order to be truly successful one must start small, and find satisfaction in the struggle of learning something new. I take pride in the many small successes I’ve had throughout my term here, and appreciate the fact that my journey into FreeBSD and open-source software is only just beginning.
\n
\n\n\nI would like to close with some brief thank-you’s. First, to everyone at the Foundation for being so helpful, and allowing this position to exist in the first place. I am extremely grateful to have been given this unique opportunity to learn about and give back to the open-source world. I’d also like to thank my office mates; Ed: for being an excellent mentor, who offered an endless wealth of knowledge and willingness to share it. My classmate and fellow intern Arshan: for giving me a sense of camaraderie and the comforting reminder that at many moments he was as lost as I was. Finally, a quick thanks to everyone else I crossed paths with who offered reviews and advice. I appreciate your help and look forward to working with you all further.
\n \nI am walking away from this co-op with a much greater appreciation for this project, and have made it a goal to remain involved in some capacity. I feel that I’ve gained a little bit of a wider perspective on my place in the software world, something I never really got from my previous co-ops. Whether it ends up being just a stepping stone, or the beginning of much larger involvement, I thoroughly enjoyed my time here.
\n
DigitalOcean\nDigital Ocean Promo Link for BSD Now Listeners
\n\nI just remind the scope of this small tool:
\n\nEnhancements in this version
\n\n\nThis is my second development version: 0.2.\n I've added performed several changes in the code:
\n
\n\n\nThe source code is still on the git of Sourceforge.net. \n You can see the files here
\n \nAnd you can download the last version here
\n
\n\n\nI'm using this script on my OpenBSD laptop since about 5 months. In my case, I'm mainly using the openbox menus and the --restart option.
\n
\n\n\nThe openbox menus are working fine. As explain in my previous blog, I just have to create 2 entries in my openbox's menu.xml file, and all the rest comes automatically from nmctl itself thanks to the --list and --scan options.\n I've not changed this part of nmctl since it works as expected (for me :-) ).
\n
\n\n\nBecause I'm very lazy, and because OpenBSD is very simple to use, I've added the command \"nmctl --restart\" in the /etc/apm/resume script. Thanks to apmd, this script will be used each time I'm opening the lid of my laptop. \n In other words, each time I'll opening my laptop, nmctl will search the optimum network connection for me.\n But I had several issues in this scenario.\n Most of the problems were linked to the arp table issues. Indeed, in some circumstances, my proxy IP address was associated to the cable interface instead of the wifi interface or vice-versa. As consequence I'm not able to connect to the proxy, thus not able to connect to internet. So the ping to google (final test nmctl perform) is failing.\n Knowing that anyhow, I'm doing a full arp cleanup, it's not clear for me from where this problem come from. To solve this situation I've implemented a \"retry\" concept. In other words, before testing an another possible network connection (as listed in my /etc/nmctl.conf file), the script try 3x the current connection's parameters.\n If you want to reduce or increase this figures, you can do it via the --retry parameter.
\n
\n\n\nWhere ever I'm located, my laptop is now connecting automatically to the wifi / cable connection previously identified for this location.\n Currently I have 3 places where I have Wifi credentials and 2 offices places where I just have to plug the network cable.\n Since the /etc/apm/resume scripts is triggered when I open the lid of the laptop, I just have to make sure that I plug the RJ45 before opening the laptop. For the rest, I do not have to type any commands, OpenBSD do all what is needed ;-).\n I hotels or restaurants, I can just connect to the Open Wifi thanks to the openbox menu created by \"nmctl --scan\".
\n
Next steps
Documentation
\n\n\nThe tool is missing lot of documentation. I appreciate OpenBSD for his great documentation, so I have to do the same.\n I plan to write a README and a man page at first instances.\n But since my laziness, I will do it as soon as I see some interest for this tool from other persons.
\n
\n\n\nI now have to travel and see how to see the script react on the different situations.\n Interested persons are welcome to share with me the outcome of their tests.\n I'm curious how it work.
\n
\n\n\nOpenBSD 6.3 oceton upgrade instructions may not factor that your ERL is running from the USB key they want wiped with the miniroot63.fs image loaded on.\n Place the bsd.rd for OpenBSD 6.3 on the sd0i slice used by U-Boot for the kernel, and then edit the boot command to run it.
\n
\n\n\nThe OpenBSD documentation is comprehensive, but there might be rough corners around what are probably edge cases in their user base. People running EdgeRouter Lite hardware for example, who are looking to upgrade from 6.2 to 6.3.\n The documentation, which gave us everything we needed last time, left me with some questions about how to upgrade. In INSTALL.octeon, the Upgrading section does mention:\n The best solution, whenever possible, is to backup your data and reinstall from scratch\n I had to check if that directive existed in the documentation for other architectures. I wondered if oceton users were getting singled out. We were not. Just simplicity and pragmatism.
\n
\n\n\nTo upgrade OpenBSD 6.3 from a previous version, start with the general instructions in the section \"Installing OpenBSD\".\n But that section requires us to boot off of TFTP or NFS. Which I don’t want to do right now. Could also use a USB stick with the miniroot63.fs installed on it.\n But as the ERL only has a single USB port, we would have to remove the USB stick with the current install on it. Once we get to the Install or Upgrade prompt, there would be nothing to upgrade.\n Well, I guess I could use a USB hub. But the ERL’s USB port is inside the case. With all the screws in. And the tools are neatly put away. And I’d have to pull the USB hub from behind a workstation. And it’s two am. And I cleaned up the cabling in the lab this past weekend. Looks nice for once.\n So I don’t want to futz around with all that.\n There must be an almost imperceptibly easier way of doing this than setting up a TFTP server or NFS share in five minutes… Right?
\n
iXsystems\nBoise Technology Show 2018 Recap
\n\n\n\n\ned is this sort of terrifying text editor. A typical interaction with ed for me in the past has gone something like this:
\n
\n$ ed\nhelp\n?\nh\n?\nasdfasdfasdfsadf\n?\n<close terminal in frustration>\n
\n\n\nBasically if you do something wrong, ed will just print out a single, unhelpful, ?. So I’d basically dismissed ed as an old arcane Unix tool that had no practical use today.\n vi is a successor to ed, except with a visual interface instead of this ?
\n
\n\n\nSo if Ed is a terrifying thing that only prints ? at you, why am I writing a blog post about it? WELL!!!!\n On April 1 this year, Michael W Lucas published a new short book called Ed Mastery. I like his writing, and even though it was sort of an april fool’s joke, it was ALSO a legitimate actual real book, and so I bought it and read it to see if his claims that Ed is actually interesting were true.\n And it was so cool!!!! I found out:
\n
\n\n\nAll of that was a cool Unix history lesson, but did not make me want to actually use Ed in real life. But!!!
\n \nThe other neat thing about Ed (that did make me want to use it!) is that any Ed session corresponds to a script that you can replay! So if I know Ed, then I can use Ed basically as a way to easily apply vim-macro-like programs to my files.
\n
Tarsnap
\n\nHow Intel docs were misinterpreted by almost any OS, a look at the mininet SDN emulator, do’s and don’ts for FreeBSD, OpenBSD community going gold, ed mastery is a must read, and the distributed object store minio on FreeBSD.
\n\n\n\n\nA statement in the System Programming Guide of the Intel 64 and IA-32 Architectures Software Developer's Manual (SDM) was mishandled in the development of some or all operating-system kernels, resulting in unexpected behavior for #DB exceptions that are deferred by MOV SS or POP SS, as demonstrated by (for example) privilege escalation in Windows, macOS, some Xen configurations, or FreeBSD, or a Linux kernel crash.\n OS kernels may not expect this order of events and may therefore experience unexpected behavior when it occurs. \n + A detailed white paper describes this behavior here\n + FreeBSD Commit\n Thank you to the MSRC Incident Response Team, and in particular Greg Lenti and Nate Warfield, for coordinating the response to this issue across multiple vendors.\n Thanks to Computer Recycling at The Working Center of Kitchener for making hardware available to allow us to test the patch on additional CPU families.\n + FreeBSD Security Advisory\n + DragonFlyBSD Post\n + NetBSD does not support debug register and so is not affected.\n + OpenBSD also appears to not be affected, “We are not aware of further vendor information regarding this vulnerability.”\n + IllumOS Not Impacted
\n
\n At this year’s AsiaBSDCon, I presented a talk about a SDN network emulator called Mininet, and my ongoing work to make it more portable. That presentation was focused on the OpenBSD version of the port, and I breezed past the detail that I also had a version or Mininet working on FreeBSD. Because I was given the opportunity, I’d like to share a bit about the FreeBSD version of Mininet. It will not only be about what Mininet is and why it might be interesting, but also a recounting of my experience as a user making a first-time attempt at porting an application to FreeBSD.\n Mininet started off as a tool used by academic researchers to emulate OpenFlow networks when they didn’t have convenient access to actual networks. Because of its history, Mininet became associated strongly with networks that use OpenFlow for their control channels. But, it has also become fairly popular among developers working in, and among several universities for research and teaching about, SDN (Software Defined Networking)\n I began using Mininet as an intern at my university’s network research lab. I was using FreeBSD by that time, and wasn’t too happy to learn that Mininet wouldn’t work on anything but Linux. I gradually got tired of having to run a Linux VM just to use Mininet, and one day it clicked in my mind that I can actually try porting it to FreeBSD.\n Mininet creates a topology using the resource virtualization features that Linux has. Specifically, nodes are bash processes running in network namespaces, and the nodes are interconnected using veth virtual Ethernet links. Switches and controllers are just nodes whose shells have run the right commands to configure a software switch or start a controller application. Mininet can therefore be viewed as a series of Python libraries that run the system commands necessary to create network namespaces and veth interfaces, assemble a specified topology, and coordinate how user commands aimed at nodes (since they are just shells) are run.\n Coming back to the port, I chose to use vnet jails to replace the network namespaces, and epair(4) links to replace the veth links. For the SDN functionality, I needed at least one switch and controller that can be run on FreeBSD. I chose OpenvSwitch(OVS) for the switch, since it was available in ports and is well-known by the SDN world, and Ryu for the controller since it’s being actively developed and used and supports more recent versions of OpenFlow.\n I have discussed the possibility of upstreaming my work. Although they were excited about it, I was asked about a script for creating VMs with Mininet preinstalled, and continuous integration support for my fork of the repository. I started taking a look at the release scripts for creating a VM, and after seeing that it would be much easier to use the scripts if I can get Mininet and Ryu added to the ports tree, I also tried a hand at submitting some ports. For CI support, Mininet uses Travis, which unfortunately doesn’t support FreeBSD. For this, I plan to look at a minimalistic CI tool called contbuild, which looks simple enough to get running and is written portably.\n This is very much a work-in-progress, and one going at a glacial pace. Even though the company that I work for does use Mininet, but doesn’t use FreeBSD, so this is something that I’ve been working on in my free time. Earlier on, it was the learning curve that made progress slow. When I started, I hadn’t done anything more than run FreeBSD on a laptop, and uneventfully build a few applications from the ports tree. Right off the bat, using vnet jails meant learning how to build and run a custom kernel. This was the easy part, as the handbook was clear about how to do this. When I moved from using FreeBSD 10.3 to 11, I found that I can panic my machine by quickly creating and destroying OVS switches and jails. I submitted a bug report, but decided to go one step further and actually try to debug the panic for myself. With the help of a few people well-versed in systems programming and the developer’s handbook, I was able to come up with a fix, and get it accepted. This pretty much brings my porting experiment to the present day, where I’m slowly working out the pieces that I mentioned earlier.\n In the beginning, I thought that this Mininet port would be a weekend project where I come out knowing thing or two about using vnet jails and with one less VM to run. Instead, it became a crash course in building and debugging kernels and submitting bug reports, patches, and ports. It’d like to mention that I wouldn’t have gotten far at all if it weren’t for the helpful folks, the documentation, and how debuggable FreeBSD is. I enjoy good challenges and learning experiences, and this has definitely been both.
```\nMonthly paypal donations from the OpenBSD community have made the community the OpenBSD Foundation's first Gold level contributor for 2018!
\n\n2018 is the third consecutive year that the community has reached Gold status or better.
\n\nThese monthly paypal commitments by the community are our most reliable source of funds and thus the most useful for financial planning purposes. We are extremely thankful for the continuing support and hope the community matches their 2017 achievement of Platinum status. Or even their 2016 achievement of Iridium status.
\n\nSign up now for a monthly donation!
\n\nNote that Bitcoin contributions have been re-enabled now that our Bitcoin intermediary has re-certified our Canadian paperwork.
\n\nhttps://www.openbsdfoundation.org/donations.html\n```
\n\n\n\n\nIn some circles on the Internet, your choice of text editor is a serious matter.
\n \nWe've all seen the threads on mailing lits, USENET news groups and web forums about the relative merits of Emacs vs vi, including endless iterations of flame wars, and sometimes even involving lesser known or non-portable editing environments.
\n \nAnd then of course, from the Linux newbies we have seen an endless stream of tweeted graphical 'memes' about the editor vim (aka 'vi Improved') versus the various apparently friendlier-to-some options such as GNU nano. Apparently even the 'improved' version of the classical and ubiquitous vi(1) editor is a challenge even to exit for a significant subset of the younger generation.
\n \nYes, your choice of text editor or editing environment is a serious matter. Mainly because text processing is so fundamental to our interactions with computers.
\n \nBut for those of us who keep our systems on a real Unix (such as OpenBSD or FreeBSD), there is no real contest. The OpenBSD base system contains several text editors including vi(1) and the almost-emacs mg(1), but ed(1) remains the standard editor.
\n \nNow Michael Lucas has written a book to guide the as yet uninitiated to the fundamentals of the original Unix text editor. It is worth keeping in mind that much of Unix and its original standard text editor written back when the standard output and default user interface was more likely than not a printing terminal.
\n \nTo some of us, reading and following the narrative of Ed Mastery is a trip down memory lane. To others, following along the text will illustrate the horror of the world of pre-graphic computer interfaces. For others again, the fact that ed(1) doesn't use your terminal settings much at all offers hope of fixing things when something or somebody screwed up your system so you don't have a working terminal for that visual editor.
\n
DigitalOcean\nDigital Ocean Promo Link for BSD Now Listeners
\n\n\n\n\nFree and open source distributed object storage server compatible with Amazon S3 v2/v4 API. Offers data protection against hardware failures using erasure code and bitrot detection. Supports highly available distributed setup. Provides confidentiality, integrity and authenticity assurances for encrypted data with negligible performance overhead. Both server side and client side encryption are supported. Below is the image of example Minio setup.
\n
The Minio identifies itself as the ZFS of Cloud Object Storage. This guide will show You how to setup highly available distributed Minio storage on the FreeBSD operating system with ZFS as backend for Minio data. For convenience we will use FreeBSD Jails operating system level virtualization.
\n\n\n\n\nThe setup will assume that You have 3 datacenters and assumption that you have two datacenters in whose the most of the data must reside and that the third datacenter is used as a ‘quorum/witness’ role. Distributed Minio supports up to 16 nodes/drives total, so we may juggle with that number to balance data between desired datacenters. As we have 16 drives to allocate resources on 3 sites we will use 7 + 7 + 2 approach here. The datacenters where most of the data must reside have 7/16 ratio while the ‘quorum/witness’ datacenter have only 2/16 ratio. Thanks to built in Minio redundancy we may loose (turn off for example) any one of those machines and our object storage will still be available and ready to use for any purpose.
\n
\n\n\nFirst we will create 3 jails for our proof of concept Minio setup, storage1 will have the ‘quorum/witness’ role while storage2 and storage3 will have the ‘data’ role. To distinguish commands I type on the host system and storageX Jail I use two different prompts, this way it should be obvious what command to execute and where.
\n
\n\n\nLet's set the record straight for securing kcgi CGI and FastCGI applications with pledge(2). This is focussed on secure OpenBSD deployments.
\n
\n\n\nInternally, kcgi makes considerable use of available security tools. But it's also designed to be invoked in a secure environment. We'll start with pledge(2), which has been around on OpenBSD since version 5.9. If you're reading this tutorial, you're probably on OpenBSD, and you probably have knowledge of pledge(2).
\n \nHow to begin? Read kcgi(3). It includes canonical information on which pledge(2) promises you'll need for each function in the library. This is just a tutorial—the manpage is canonical and overrides what you may read here.
\n \nNext, assess the promises that your application needs. From kcgi(3), it's easy to see which promises we'll need to start. You'll need to augment this list with whichever tools you're also using. The general push is to start with the broadest set of required promises, then restrict as quickly as possible. Sometimes this can be done in a single pledge(2), but other times it takes a few.
\n
Allan’s recap of the ZFS User conference, first impressions of OmniOS by a BSD user, Nextcloud 13 setup on FreeBSD, OpenBSD on a fanless desktop computer, an intro to HardenedBSD, and DragonFlyBSD getting some SMP improvements.
\n
\n\n\nI had been using FreeBSD as my main web server OS since 2012 and I liked it so much that I even contributed money and code to it. However, since the FreeBSD guys (and gals) decided to install anti-tech feminism, I have been considering to move away from it for quite some time now.
\n \nAs my growing needs require stronger hardware, it was finally time to rent a new server. I do not intend to run FreeBSD on it. Although the most obvious choice would be OpenBSD (I run it on another server and it works just fine), I plan to have a couple of databases running on the new machine, and database throughput has never been one of OpenBSD's strong points. This is my chance to give illumos another try. As neither WiFi nor desktop environments are relevant on a no-X11 server, the server-focused OmniOS seemed to fit my needs.
\n \nMy current (to be phased out) setup on FreeBSD is:
\n
\n\n\nI would not consider anything of that too esoteric for a modern operating system. Since I was not really using anything mod_rewrite-related, I was perfectly ready to replace apache24 by nginx, remembering that the prepackaged apache24 on FreeBSD did not support HTTPS out of the box and I had ended up installing it from the ports. That is the only change in my setup which I am actively planning.
\n \nSo here's what I noticed.
\n
\n\n\nHooray, a BSD boot loader! Finally an operating system without grub - I made my experiences with that and I don't want to repeat them too often.
\n \nIt is weird that the installer won't accept \"mydomain.org\" as a hostname but sendmail complains that \"mydomain\" is not a valid hostname right from the start, OmniOS sent me into Maintenance Mode to fix that. A good start, right? So the first completely new thing I had to find out on my new shiny toy was how to change the hostname. There is no /etc/rc.conf in it and hostname mydomain.org was only valid for one login session. I found out that the hostname has to be changed in three different files under /etc on Solaris - the third one did not even exist for me. Changing the other two files seems to have solved this problem for me.
\n
\n\n\n~ I was wondering how many resources my (mostly idle) new web server was using - I always thought Solaris was rather fat, but it still felt fast to me.
\n \nAh, right - we're in Unixland and we need to think outside of the box. This table was really helpful: although a number of things are different between OmniOS and SmartOS, I found out that the *stat tools do what top does. I could probably just install top from one of the package managers, but I failed to find a reason to do so. I had 99% idle CPU and RAM - that's all I wanted to know.
\n \n~ Trying to set up twtxt informed me that Python 3.6 (from pkgin) expects LANG and LC_ALL to be set. Weird - did FreeBSD do that for me? It's been a while ... at least that was easy to fix.
\n \n~ SMF - Solaris's version of init - confuses me. It has \"levels\" similar to Gentoo's OpenRC, but it mostly shuts up during the boot process. Stuff from pkgsrc, e.g. nginx, comes with a description how to set up the particular service, but I should probably read more about it. What if, one day, I install a package which is not made ready for OmniOS? I'll have to find out how to write SMF scripts. But that should not be my highest priority.
\n \n~ The OmniOS documentation talks a lot about \"zones\" which, if I understand that correctly, mostly equal FreeBSD's \"jails\". This could be my chance to try to respect a better separation between my various services - if my lazyness won't take over again. (It probably will.)
\n \n~ OmniOS's default shell - rather un-unixy - seems to be the bash. Update: I was informed about a mistake here: the default shell is ksh93, there are bogus .bashrc files lying around though.
\n \n~ Somewhere in between, my sshd had a hiccup or, at least, logging into it took longer than usual. If that happens again, I should investigate.
\n
\n\n\nBy the time of me writing this, I have a basic web server with an awesome performance and a lot of applications ready to be configured only one click away. The more I play with it, the more I have the feeling that I have missed a lot while wasting my time with FreeBSD. For a system that is said to be \"dying\", OmniOS feels well-thought and, when equipped with a reasonable package management, comes with everything I need to reproduce my FreeBSD setup without losing functionality.
\n \nI'm looking forward to what will happen with it.
\n
DigitalOcean\nhttp://do.co/bsdnow
\n\n(includes 'Open-source RISC-V core quickstart' and 'An introductory workshop to NetBSD on embedded platforms')](http://oshug.org/pipermail/oshug/2018-April/000635.html)
\n\n```\nHi All,
\n\nI'm pleased to announce that we have 10 talks and 7 workshops confirmed\nfor Open Source Hardware Camp 2018, with the possibility of one or two\nmore. Registration is now open!
\n\nFor the first time ever we will be hosting OSHCamp in Lincoln and a huge\nthanks to Sarah Markall for helping to make this happen.
\n\nAs in previous years, there will be a social event on the Saturday\nevening and we have a room booked at the Wig and Mitre. Food will be\navailable.
\n\nThere will likely be a few of us meeting up for pre-conference drinks on\nthe Friday evening also.
\n\nDetails of the programme can be found below and, as ever, we have an\nexcellent mix of topics being covered.
\n\nCheers,
\n\nAndrew\n```
\n\n\n\n\nOn the 30th June 2018, 09:00 Saturday morning - 16:00 on the Sunday\n afternoon at The Blue Room, The Lawn, Union Rd, Lincoln, LN1 3BU.
\n
\n\n\nToday I would like to share a setup of Nextcloud 13 running on a FreeBSD system. To make things more interesting it would be running inside a FreeBSD Jail. I will not describe the Nextcloud setup itself here as its large enough for several blog posts.
\n \nOfficial Nextcloud 13 documentation recommends following setup:
\n
\n\n\nI prefer PostgreSQL database to MySQL/MariaDB and I prefer fast and lean Nginx web server to Apache, so my setup is based on these components:
\n
\n\n\nThe Memcached subsystem is least important, it can be easily changed into something more modern like Redis for example. I prefer not to use any third party tools for FreeBSD Jails management. Not because they are bad or something like that. There are just many choices for good FreeBSD Jails management and I want to provide a GENERIC example for Nextcloud 13 in a Jail, not for a specific management tool.
\n
\n\n\nLets start with preparing the FreeBSD Host with needed settings. We need to allow using raw sockets in Jails. For the future optional upgrades of the Jail we will also allow using chflags(1) in Jails.
\n
\n\n\nYou asked me about my setup. Here you go.
\n \nI’ve been using OpenBSD on servers for years as a web developer, but never had a chance to dive in to system administration before. If you appreciate the simplicity of OpenBSD and you have to give it a try on your desktop.
\n \nBear in mind, this is a relatively cheap ergonomic setup, because all I need is xterm(1) with Vim and Firefox, I don’t care about CPU/GPU performance or mobility too much, but I want a large screen and a good keyboard.
\n
\nItem Price, USD\nZotac CI527 NANO-BE $371\n16GB RAM Crucial DDR4-2133 $127\n250GB SSD Samsung 850 EVO $104\nAsus VZ249HE 23.8\" IPS Full HD $129\nErgoDox EZ V3, Cherry MX Brown, blank DCS $325\nKensington Orbit Trackball $33\nTotal $1,107\n
\n\n\nI tried few times to install OpenBSD on my MacBooks—I heard some models are compatible with it,—but in my case it was a bit of a fiasco (thanks to Nvidia and Broadcom). That’s why I bought a new computer, just to be able to run this wonderful operating system.
\n \nNow I run -stable on my desktop and servers. Servers are supposed to be reliable, that’s obvious, why not run -current on a desktop? Because -stable is shipped every six months and I that’s is often enough for me. I prefer slow fashion.
\n
iXsystems\niX Ad Spot NAB 2018 – Michael Dexter’s Recap
\n\n\n\n\nHardenedBSD is a security enhanced fork of FreeBSD which happened in 2014. HardenedBSD is implementing many exploit mitigation and security technologies on top of FreeBSD which all started with implementation of Address Space Layout Randomization (ASLR). The fork has been created for ease of development.
\n \nTo cite the https://hardenedbsd.org/content/about page – “HardenedBSD aims to implement innovative exploit mitigation and security solutions for the FreeBSD community. (…) HardenedBSD takes a holistic approach to security by hardening the system and implementing exploit mitigation technologies.”
\n \nMost FreeBSD enthusiasts know mfsBSD project by Martin Matuska – http://mfsbsd.vx.sk/ – FreeBSD system loaded completely into memory. The mfsBSD synonym for the HardenedBSD world is SoloBSD – http://www.solobsd.org/ – which is based on HardenedBSD sources.
\n \nOne may ask how HardenedBSD project compared to more well know for its security OpenBSD system and it is very important question. The OpenBSD developers try to write ‘good’ code without dirty hacks for performance or other reasons. Clean and secure code is most important in OpenBSD world. The OpenBSD project even made security audit of all OpenBSD code available, line by line. This was easier to achieve in FreeBSD or HardenedBSD because OpenBSD code base its about ten times smaller. This has also other implications, possibilities. While FreeBSD (and HardenedBSD) offer many new features like mature SMP subsystem even with some NUMA support, ZFS filesystem, GEOM storage framework, Bhyve virtualization, Virtualbox option and many other new modern features the OpenBSD remains classic UNIX system with UFS filesystem and with very ‘theoretical’ SMP support. The vmm project tried to implement new hypervisor in OpenBSD world, but because of lack of support for graphics its for OpenBSD, Illumos and Linux currently, You will not virtualize Windows or Mac OS X there. This is also only virtualization option for OpenBSD as there are no Jails on OpenBSD. Current Bhyve implementation allows one even to boot latest Windows 2019 Technology Preview.
\n \nA HardenedBSD project is FreeBSD system code base with LOTS of security mechanisms and mitigations that are not available on FreeBSD system. For example entire lib32 tree has been disabled by default on HardenedBSD to make it more secure. Also LibreSSL is the default SSL library on HardenedBSD, same as OpenBSD while FreeBSD uses OpenSSL for compatibility reasons.
\n \nComparison between LibreSSL and OpenSSL vulnerabilities.
\n
\n\n\nOne may see HardenedBSD as FreeBSD being successfully pulled up to the OpenBSD level (at least that is the goal), but as FreeBSD has tons more code and features it will be harder and longer process to achieve the goal.
\n \nAs I do not have that much competence on the security field I will just repost the comparison from the HardenedBSD project versus other BSD systems. The comparison is also available here – https://hardenedbsd.org/content/easy-feature-comparison – on the HardenedBSD website.
\n
\n\n\nNote: This article is predominantly based on work by Hiltjo Posthuma who you should read because I would have spent far too much time failing to set things up if it wasn’t for their post. Not only have they written lots of very interesting posts, they write some really brilliant programs
\n \nSince I started university 3 years ago, I started using lots of services from lots of different companies. The “cloud” trend led me to believe that I wanted other people to look after my data for me. I was wrong. Since finding myself loving the ethos of OpenBSD, I found myself wanting to apply this ethos to the services I use as well. Not only is it important to me because of the security benefits, but also because I like the minimalist style OpenBSD portrays. This is the first in a mini-series documenting my move from bloated, hosted, sometimes proprietary services to minimal, well-written, free, self-hosted services.
\n
\n\n\nThese are the programs I am going to be using to get my git server up and running:
\n
\nhttpd(8)\nacme-client(1)\ngit(1)\ncgit(1)\nslowcgi(8)\n
\n\n\nEnsure you have the necessary flags enabled in your /etc/rc.conf.local:
\n
\n\n\nWhen using the OpenBSD httpd(8), it will serve it’s content in a chrooted environment,which defaults to the home directory of the user it runs as, which is www in this case. This means that the chroot is limited to the directory /var/www and it’s contents.
\n \nIn order to configure cgit, there must be a cgitrc file available to cgit. This is found at the location stored in $CGIT_CONFIG, which defaults to /conf/cgitrc. Because of the chroot, this file is actually stored at /var/www/conf/cgitrc.
\n
Tarsnap ad
\n\nArcan and OpenBSD, running OpenBSD 6.3 on RPI 3, why C is not a low-level language, HardenedBSD switching back to OpenSSL, how the Internet was almost broken, EuroBSDcon CfP is out, and the BSDCan 2018 schedule is available.
\n\n\n\n\nLet me preface this by saying that this is a (very) long and medium-rare technical article about the security considerations and minutiae of porting (most of) the Arcan ecosystem to work under OpenBSD. The main point of this article is not so much flirting with the OpenBSD crowd or adding further noise to software engineering topics, but to go through the special considerations that had to be taken, as notes to anyone else that decides to go down this overgrown and lonesome trail, or are curious about some less than obvious differences between how these things “work” on Linux vs. other parts of the world.
\n \nA disclaimer is also that most of this have been discovered by experimentation and combining bits and pieces scattered in everything from Xorg code to man pages, there may be smarter ways to solve some of the problems mentioned – this is just the best I could find within the time allotted. I’d be happy to be corrected, in patch/pull request form that is 😉
\n \nEach section will start with a short rant-like explanation of how it works in Linux, and what the translation to OpenBSD involved or, in the cases that are still partly or fully missing, will require. The topics that will be covered this time are:
\n
\n\n\nInstalling the OpenBSD on raspberry pi 3 is very easy and well documented which almost convinced me of not writing about it, but still I felt like it may help somebody new to the project (But again I really recommend reading the document if you are interested and have the time).
\n \nNote: I'm always running snapshots and recommend anybody to do it as well. But the snapshots links will change to the next version every 6 month, so I changed the links to the 6.3 version to keep the blog post valid over times. If you're familiar to the OpenBSD flavors, feel free to use the snapshots links instead.
\n
\n\n\nDue to the lack of driver, the OpenBSD can not boot directly from the SD Card yet, So we'll need an USB Stick for the installtion target aside the SD Card for the U-Boot and installer. Also, a Serial Console connection is required. I Used a PL2303 USB to Serial (TTL) adapter connected to my Laptop via USB port and connected to the Raspberry via TX, RX and GND pins.
\n
iXsystems\nhttps://www.ixsystems.com/blog/truenas-m-series-veeam-pr-2018/
\n\n\n\n\nEvery month or so, someone will ask me what happened to Larrabee and why it failed so badly. And I then try to explain to them that not only didn't it fail, it was a pretty huge success. And they are understandably very puzzled by this, because in the public consciousness Larrabee was like the Itanic and the SPU rolled into one, wasn't it? Well, not quite. So rather than explain it in person a whole bunch more times, I thought I should write it down.
\n \nThis is not a history, and I'm going to skip a TON of details for brevity. One day I'll write the whole story down, because it's a pretty decent escapade with lots of fun characters. But not today. Today you just get the very start and the very end.
\n \nWhen I say \"Larrabee\" I mean all of Knights, all of MIC, all of Xeon Phi, all of the \"Isle\" cards - they're all exactly the same chip and the same people and the same software effort. Marketing seemed to dream up a new codeword every week, but there was only ever three chips:
\n
\n\n\nThat's it. There were some other codenames I've forgotten over the years, but they're all of one of the above chips. Behind all the marketing smoke and mirrors there were only three chips ever made (so far), and only four planned in total (we had a thing called LRB3 planned between KNC and KNL for a while). All of them are \"Larrabee\", whether they do graphics or not.
\n \nWhen Larrabee was originally conceived back in about 2005, it was called \"SMAC\", and its original goals were, from most to least important:
\n
\n\n\nThat ordering is important - in terms of engineering and focus, Larrabee was never primarily a graphics card. If Intel had wanted a kick-ass graphics card, they already had a very good graphics team begging to be allowed to build a nice big fat hot discrete GPU - and the Gen architecture is such that they'd build a great one, too. But Intel management didn't want one, and still doesn't. But if we were going to build Larrabee anyway, they wanted us to cover that market as well.
\n \n... the design of Larrabee was of a CPU with a very wide SIMD unit, designed above all to be a real grown-up CPU - coherent caches, well-ordered memory rules, good memory protection, true multitasking, real threads, runs Linux/FreeBSD, etc. Larrabee, in the form of KNC, went on to become the fastest supercomputer in the world for a couple of years, and it's still making a ton of money for Intel in the HPC market that it was designed for, fighting very nicely against the GPUs and other custom architectures. Its successor, KNL, is just being released right now (mid 2016) and should do very nicely in that space too. Remember - KNC is literally the same chip as LRB2. It has texture samplers and a video out port sitting on the die. They don't test them or turn them on or expose them to software, but they're still there - it's still a graphics-capable part.
\n \nBut it's still actually running FreeBSD on that card, and under FreeBSD it's just running an x86 program called DirectXGfx (248 threads of it).
\n
\n\n\nIn the wake of the recent Meltdown and Spectre vulnerabilities, it's worth spending some time looking at root causes. Both of these vulnerabilities involved processors speculatively executing instructions past some kind of access check and allowing the attacker to observe the results via a side channel. The features that led to these vulnerabilities, along with several others, were added to let C programmers continue to believe they were programming in a low-level language, when this hasn't been the case for decades.
\n \nProcessor vendors are not alone in this. Those of us working on C/C++ compilers have also participated.
\n
\n\n\nComputer science pioneer Alan Perlis defined low-level languages this way: \"A programming language is low level when its programs require attention to the irrelevant.\"
\n \nWhile, yes, this definition applies to C, it does not capture what people desire in a low-level language. Various attributes cause people to regard a language as low-level. Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise's computer at the other. Low-level languages are \"close to the metal,\" whereas high-level languages are closer to how humans think.
\n \nFor a language to be \"close to the metal,\" it must provide an abstract machine that maps easily to the abstractions exposed by the target platform. It's easy to argue that C was a low-level language for the PDP-11. They both described a model in which programs executed sequentially, in which memory was a flat space, and even the pre- and post-increment operators cleanly lined up with the PDP-11 addressing modes.
\n
Fast PDP-11 Emulators
\n\n\n\n\n\n\nThe root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11. This is essential because it allows C programmers to continue in the belief that their language is close to the underlying hardware.
\n \nC code provides a mostly serial abstract machine (until C11, an entirely serial machine if nonstandard vendor extensions were excluded). Creating a new thread is a library operation known to be expensive, so processors wishing to keep their execution units busy running C code rely on ILP (instruction-level parallelism). They inspect adjacent operations and issue independent ones in parallel. This adds a significant amount of complexity (and power consumption) to allow programmers to write mostly sequential code. In contrast, GPUs achieve very high performance without any of this logic, at the expense of requiring explicitly parallel programs.
\n \nThe quest for high ILP was the direct cause of Spectre and Meltdown. A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine, which expects each operation to complete before the next one begins). A typical heuristic for C code is that there is a branch, on average, every seven instructions. If you wish to keep such a pipeline full from a single thread, then you must guess the targets of the next 25 branches. This, again, adds complexity; it also means that an incorrect guess results in work being done and then discarded, which is not ideal for power consumption. This discarded work has visible side effects, which the Spectre and Meltdown attacks could exploit.
\n \nOn a modern high-end core, the register rename engine is one of the largest consumers of die area and power. To make matters worse, it cannot be turned off or power gated while any instructions are running, which makes it inconvenient in a dark silicon era when transistors are cheap but powered transistors are an expensive resource. This unit is conspicuously absent on GPUs, where parallelism again comes from multiple threads rather than trying to extract instruction-level parallelism from intrinsically scalar code. If instructions do not have dependencies that need to be reordered, then register renaming is not necessary.
\n \nConsider another core part of the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades. A modern processor often has three levels of cache in between registers and main memory, which attempt to hide latency.
\n \nThe cache is, as its name implies, hidden from the programmer and so is not visible to C. Efficient use of the cache is one of the most important ways of making code run quickly on a modern processor, yet this is completely hidden by the abstract machine, and programmers must rely on knowing implementation details of the cache (for example, two values that are 64-byte-aligned may end up in the same cache line) to write efficient code.
\n
\n\n\nOver a year ago, HardenedBSD switched to LibreSSL as the default cryptographic library in base for 12-CURRENT. 11-STABLE followed suit later on. Bernard Spil has done an excellent job at keeping our users up-to-date with the latest security patches from LibreSSL.
\n \nAfter recently updating 12-CURRENT to LibreSSL 2.7.2 from 2.6.4, it has become increasingly clear to us that performing major upgrades requires a team larger than a single person. Upgrading to 2.7.2 caused a lot of fallout in our ports tree. As of 28 Apr 2018, several ports we consider high priority are still broken. As it stands right now, it would take Bernard a significant amount of his spare personal time to fix these issues.
\n \nUntil we have a multi-person team dedicated to maintaining LibreSSL in base along with the patches required in ports, HardenedBSD will use OpenSSL going forward as the default cryptographic library in base. LibreSSL will co-exist with OpenSSL in the source tree, as it does now. However, MK_LIBRESSL will default to \"no\" instead of the current \"yes\". Bernard will continue maintaining LibreSSL in base along with addressing the various problematic ports entries.
\n \nTo provide our users with ample time to plan and perform updates, we will wait a period of two months prior to making the switch. The switch will occur on 01 Jul 2018 and will be performed simultaneously in 12-CURRENT and 11-STABLE. HardenedBSD will archive a copy of the LibreSSL-centric package repositories and binary updates for base for a period of six months after the switch (expiring the package repos on 01 Jan 2019). This essentially gives our users eight full months for an upgrade path.
\n \nAs part of the switch back to OpenSSL, the default NTP daemon in base will switch back from OpenNTPd to ISC NTP. Users who have localopenntpdenable=\"YES\" set in rc.conf will need to switch back to ntpd_enable=\"YES\".
\n \nUsers who build base from source will want to fully clean their object directories. Any and all packages that link with libcrypto or libssl will need to be rebuilt or reinstalled.
\n \nWith the community's help, we look forward to the day when we can make the switch back to LibreSSL. We at HardenedBSD believe that providing our users options to rid themselves of software monocultures can better increase security and manage risk.
\n
DigitalOcean\nhttp://do.co/bsdnow -- $100 credit for 60 days
\n\n\n\n\nIn the summer of 2008, security researcher Dan Kaminsky disclosed how he had found a huge flaw in the Internet that could let attackers redirect web traffic to alternate servers and disrupt normal operations. In this Hacker History video, Kaminsky describes the flaw and notes the issue remains unfixed.
\n \n“We were really concerned about web pages and emails 'cause that’s what you get to compromise when you compromise DNS,” Kaminsky says. “You think you’re sending an email to IBM but it really goes to the bad guy.”
\n \nAs the phone book of the Internet, DNS translates easy-to-remember domain names into IP addresses so that users don’t have to remember strings of numbers to reach web applications and services. Authoritative nameservers publish the IP addresses of domain names. Recursive nameservers talk to authoritative servers to find addresses for those domain names and saves the information into its cache to speed up the response time the next time it is asked about that site. While anyone can set up a nameserver and configure an authoritative zone for any site, if recursive nameservers don’t point to it to ask questions, no one will get those wrong answers.
\n \nWe made the Internet less flammable.
\n \nKaminsky found a fundamental design flaw in DNS that made it possible to inject incorrect information into the nameserver's cache, or DNS cache poisoning. In this case, if an attacker crafted DNS queries looking for sibling names to existing domains, such as 1.example.com, 2.example.com, and 3.example.com, while claiming to be the official \"www\" server for example.com, the nameserver will save that server IP address for “www” in its cache.
\n \n“The server will go, ‘You are the official. Go right ahead. Tell me what it’s supposed to be,’” Kaminsky says in the video.
\n \nSince the issue affected nearly every DNS server on the planet, it required a coordinated response to address it. Kaminsky informed Paul Vixie, creator of several DNS protocol extensions and application, and Vixie called an emergency summit of major IT vendors at Microsoft’s headquarters to figure out what to do.
\n \nThe “fix” involved combining the 16-bit transaction identifier that DNS lookups used with UDP source ports to create 32-bit transaction identifiers. Instead of fixing the flaw so that it can’t be exploited, the resolution focused on making it take more than ten seconds, eliminating the instantaneous attack.
\n \n“[It’s] not like we repaired DNS,” Kaminsky says. “We made the Internet less flammable.”
\n \nDNSSEC (Domain Name System Security Extensions), is intended to secure DNS by adding a cryptographic layer to DNS information. The root zone of the internet was signed for DNSSEC in July 2010 and the .com Top Level Domain (TLD) was finally signed for DNSSEC in April 2011. Unfortunately, adoption has been slow, even ten years after Kaminsky first raised the alarm about DNS, as less than 15 percent of users pass their queries to DNSSEC validating resolvers.
\n \nThe Internet was never designed to be secure. The Internet was designed to move pictures of cats.
\n \nNo one expected the Internet to be used for commerce and critical communications. If people lose faith in DNS, then all the things that depend on it are at risk.
\n \n“What are we going to do? Here is the answer. Some of us gotta go out fix it,” Kaminsky says.
\n
We have released a new OpenIndiana Hipster snapshot 2018.04. The noticeable changes:
\n\nMore information can be found in 2018.04 Release notes and new medias can be downloaded from http://dlc.openindiana.org.
Tarsnap ad
\n\niX Ad spot: iXsystems TrueNAS M-Series Blows Away Veeam Backup Certification Tests
","summary":"Arcan and OpenBSD, running OpenBSD 6.3 on RPI 3, why C is not a low-level language, HardenedBSD switching back to OpenSSL, how the Internet was almost broken, EuroBSDcon CfP is out, and the BSDCan 2018 schedule is available.","date_published":"2018-05-03T03:00:00.000-04:00","attachments":[{"url":"https://aphid.fireside.fm/d/1437767933/c91b88f1-e824-4815-bcb8-5227818d6010/a46e2baa-82ee-4acb-9678-978f26dbd32c.mp3","mime_type":"audio/mp3","size_in_bytes":61656187,"duration_in_seconds":5132}]},{"id":"http://feed.jupiter.zone/bsdnow#entry-1826","title":"Episode 243: Understanding The Scheduler | BSD Now 243","url":"https://www.bsdnow.tv/243","content_text":"OpenBSD 6.3 and DragonflyBSD 5.2 are released, bug fix for disappearing files in OpenZFS on Linux (and only Linux), understanding the FreeBSD CPU scheduler, NetBSD on RPI3, thoughts on being a committer for 20 years, and 5 reasons to use FreeBSD in 2018.\n\nHeadlines\n\nOpenBSD 6.3 released\n\n\nPunctual as ever, OpenBSD 6.3 has been releases with the following features/changes:\n\n\n\n Improved HW support, including:\n SMP support on OpenBSD/arm64 platforms\n vmm/vmd improvements:\n IEEE 802.11 wireless stack improvements\n Generic network stack improvements\n Installer improvements\n Routing daemons and other userland network improvements\n Security improvements\n dhclient(8) improvements\n Assorted improvements\n OpenSMTPD 6.0.4\n OpenSSH 7.7\n LibreSSL 2.7.2\n \n \n\n\nDragonFlyBSD 5.2 released\n\n\n\n\n Big-ticket items\n Meltdown and Spectre mitigation support\n Meltdown isolation and spectre mitigation support added. Meltdown mitigation is automatically enabled for all Intel cpus. Spectre mitigation must be enabled manually via sysctl if desired, using sysctls machdep.spectremitigation and machdep.meltdownmitigation.\n HAMMER2\n H2 has received a very large number of bug fixes and performance improvements. We can now recommend H2 as the default root filesystem in non-clustered mode.\n Clustered support is not yet available.\n ipfw Updates\n Implement state based \"redirect\", i.e. without using libalias.\n ipfw now supports all possible ICMP types.\n Fix ICMPMAXTYPE assumptions (now 40 as of this release).\n Improved graphics support\n The drm/i915 kernel driver has been updated to support Intel Coffeelake GPUs\n Add 24-bit pixel format support to the EFI frame buffer code.\n Significantly improve fbio support for the \"scfb\" XOrg driver. This allows EFI frame buffers to be used by X in situations where we do not otherwise support the GPU.\n Partly implement the FBIOBLANK ioctl for display powersaving.\n Syscons waits for drm modesetting at appropriate places, avoiding races.\n + For more details, check out the “All changes since DragonFly 5.0” section.\n\n\n\n\n\n\n\n\n\nZFS on Linux bug causes files to disappear\n\n\nA bug in ZoL 0.7.7 caused 0.7.8 to be released just 3 days after the release\nThe bug only impacts Linux, the change that caused the problem was not upstreamed yet, so does not impact ZFS on illumos, FreeBSD, OS X, or Windows\nThe bug can cause files being copied into a directory to not be properly linked to the directory, so they will no longer be listed in the contents of the directory\nZoL developers are working on a tool to allow you to recover the data, since no data was actually lost, the files were just not properly registered as part of the directory\nThe bug was introduced in a commit made in February, that attempted to improve performance of datasets created with the case insensitivity option. In an effort to improve performance, they introduced a limit to cap to give up (return ENOSPC) if growing the directory ZAP failed twice.\nThe ZAP is the key-value pair data structure that contains metadata for a directory, including a hash table of the files that are in a directory. When a directory has a large number of files, the ZAP is converted to a FatZAP, and additional space may need to be allocated as additional files are added.\n\n\n\n Commit cc63068 caused ENOSPC error when copy a large amount of files between two directories. The reason is that the patch limits zap leaf expansion to 2 retries, and return ENOSPC when failed.\n Finding the root cause of this issue was somewhat hampered by the fact that many people were not able to reproduce the issue. It turns out this was caused by an entirely unrelated change to GNU coreutils.\n On later versions of GNU Coreutils, the files were returned in a sorted order, resulting in them hitting different buckets in the hash table, and not tripping the retry limit\n Tools like rsync were unaffected, because they always sort the files before copying\n If you did not see any ENOSPC errors, you were likely not impacted\n The intent for limiting retries is to prevent pointlessly growing table to max size when adding a block full of entries with same name in different case in mixed mode. However, it turns out we cannot use any limit on the retry. When we copy files from one directory in readdir order, we are copying in hash order, one leaf block at a time. Which means that if the leaf block in source directory has expanded 6 times, and you copy those entries in that block, by the time you need to expand the leaf in destination directory, you need to expand it 6 times in one go. So any limit on the retry will result in error where it shouldn't.\n Recommendations for Users from Ryan Yao:\n The regression makes it so that creating a new file could fail with ENOSPC after which files created in that directory could become orphaned. Existing files seem okay, but I have yet to confirm that myself and I cannot speak for what others know. It is incredibly difficult to reproduce on systems running coreutils 8.23 or later. So far, reports have only come from people using coreutils 8.22 or older. The directory size actually gets incremented for each orphaned file, which makes it wrong after orphan files happen.\n We will likely have some way to recover the orphaned files (like ext4’s lost+found) and fix the directory sizes in the very near future. Snapshots of the damaged datasets are problematic though. Until we have a subcommand to fix it (not including the snapshots, which we would have to list), the damage can be removed from a system that has it either by rolling back to a snapshot before it happened or creating a new dataset with 0.7.6 (or another release other than 0.7.7), moving everything to the new dataset and destroying the old. That will restore things to pristine condition.\n It should also be possible to check for pools that are affected, but I have yet to finish my analysis to be certain that no false negatives occur when checking, so I will avoid saying how for now.\n Writes to existing files cannot trigger this bug, only adding new files to a directory in bulk\n \n \n\n\nNews Roundup\n\n\n\ndes@’s thoughts on being a FreeBSD committer for 20 years\n\n\n\n\n Yesterday was the twentieth anniversary of my FreeBSD commit bit, and tomorrow will be the twentieth anniversary of my first commit. I figured I’d split the difference and write a few words about it today.\n \n My level of engagement with the FreeBSD project has varied greatly over the twenty years I’ve been a committer. There have been times when I worked on it full-time, and times when I did not touch it for months. The last few years, health issues and life events have consumed my time and sapped my energy, and my contributions have come in bursts. Commit statistics do not tell the whole story, though: even when not working on FreeBSD directly, I have worked on side projects which, like OpenPAM, may one day find their way into FreeBSD.\n \n My contributions have not been limited to code. I was the project’s first Bugmeister; I’ve served on the Security Team for a long time, and have been both Security Officer and Deputy Security Officer; I managed the last four Core Team elections and am doing so again this year.\n \n In return, the project has taught me much about programming and software engineering. It taught me code hygiene and the importance of clarity over cleverness; it taught me the ins and outs of revision control; it taught me the importance of good documentation, and how to write it; and it taught me good release engineering practices.\n \n Last but not least, it has provided me with the opportunity to work with some of the best people in the field. I have the privilege today to count several of them among my friends.\n \n For better or worse, the FreeBSD project has shaped my career and my life. It set me on the path to information security in general and IAA in particular, and opened many a door for me. I would not be where I am now without it.\n \n I won’t pretend to be able to tell the future. I don’t know how long I will remain active in the FreeBSD project and community. It could be another twenty years; or it could be ten, or five, or less. All I know is that FreeBSD and I still have things to teach each other, and I don’t intend to call it quits any time soon.\n\n\n\n\n\n\n\n\n\niXsystems unveils new TrueNAS M-Series Unified Storage Line\n\n\n\n\n San Jose, Calif., April 10, 2018 — iXsystems, the leader in Enterprise Open Source servers and software-defined storage, announced the TrueNAS M40 and M50 as the newest high-performance models in its hybrid, unified storage product line. The TrueNAS M-Series harnesses NVMe and NVDIMM to bring all-flash array performance to the award-winning TrueNAS hybrid arrays. It also includes the Intel® Xeon® Scalable Family of Processors and supports up to 100GbE and 32Gb Fibre Channel networking. Sitting between the all-flash TrueNAS Z50 and the hybrid TrueNAS X-Series in the product line, the TrueNAS M-Series delivers up to 10 Petabytes of highly-available and flash-powered network attached storage and rounds out a comprehensive product set that has a capacity and performance option for every storage budget.\n\n\n\nDesigned for On-Premises & Enterprise Cloud Environments\n\n\n\n As a unified file, block, and object sharing solution, TrueNAS can meet the needs of file serving, backup, virtualization, media production, and private cloud users thanks to its support for the SMB, NFS, AFP, iSCSI, Fibre Channel, and S3 protocols.\n \n At the heart of the TrueNAS M-Series is a custom 4U, dual-controller head unit that supports up to 24 3.5” drives and comes in two models, the M40 and M50, for maximum flexibility and scalability. The TrueNAS M40 uses NVDIMMs for write cache, SSDs for read cache, and up to two external 60-bay expansion shelves that unlock up to 2PB in capacity. The TrueNAS M50 uses NVDIMMs for write caching, NVMe drives for read caching, and up to twelve external 60-bay expansion shelves to scale upwards of 10PB. The dual-controller design provides high-availability failover and non-disruptive upgrades for mission-critical enterprise environments.\n \n By design, the TrueNAS M-Series unleashes cutting-edge persistent memory technology for demanding performance and capacity workloads, enabling businesses to accelerate enterprise applications and deploy enterprise private clouds that are twice the capacity of previous TrueNAS models. It also supports replication to the Amazon S3, BackBlaze B2, Google Cloud, and Microsoft Azure cloud platforms and can deliver an object store using the ubiquitous S3 object storage protocol at a fraction of the cost of the public cloud.\n\n\n\nFast\n\n\n\n As a true enterprise storage platform, the TrueNAS M50 supports very demanding performance workloads with up to four active 100GbE ports, 3TB of RAM, 32GB of NVDIMM write cache and up to 15TB of NVMe flash read cache. The TrueNAS M40 and M50 include up to 24/7 and global next-business-day support, putting IT at ease. The modular and tool-less design of the M-Series allows for easy, non-disruptive servicing and upgrading by end-users and support technicians for guaranteed uptime. TrueNAS has US-Based support provided by the engineering team that developed it, offering the rapid response that every enterprise needs.\n\n\n\nAward-Winning TrueNAS Features\n\nEnterprise: Perfectly suited for private clouds and enterprise workloads such as file sharing, backups, M&E, surveillance, and hosting virtual machines.\nUnified: Utilizes SMB, AFP, NFS for file storage, iSCSI, Fibre Channel and OpenStack Cinder for block storage, and S3-compatible APIs for object storage. Supports every common operating system, hypervisor, and application.\nEconomical: Deploy an enterprise private cloud and reduce storage TCO by 70% over AWS with built-in enterprise-class features such as in-line compression, deduplication, clones, and thin-provisioning.\nSafe: The OpenZFS file system ensures data integrity with best-in-class replication and snapshotting. Customers can replicate data to the rest of the iXsystems storage lineup and to the public cloud.\nReliable: High Availability option with dual hot-swappable controllers for continuous data availability and 99.999% uptime.\nFamiliar: Provision and manage storage with the same simple and powerful WebUI and REST APIs used in all iXsystems storage products, as well as iXsystems’ FreeNAS Software.\nCertified: TrueNAS has passed the Citrix Ready, VMware Ready, and Veeam Ready certifications, reducing the risk of deploying a virtualized infrastructure.\nOpen: By using industry-standard sharing protocols, the OpenZFS Open Source enterprise file system and FreeNAS, the world’s #1 Open Source storage operating system (and also engineered by iXsystems), TrueNAS is the most open enterprise storage solution on the market.\nAvailability\n\n\n\n The TrueNAS M40 and M50 will be generally available in April 2018 through the iXsystems global channel partner network. The TrueNAS M-Series starts at under $20,000 USD and can be easily expanded using a linear “per terabyte” pricing model. With typical compression, a Petabtye can be stored for under $100,000 USD. TrueNAS comes with an all-inclusive software suite that provides NFS, Windows SMB, iSCSI, snapshots, clones and replication.\n\n\n\nFor more information, visit www.ixsystems.com/TrueNAS \nTrueNAS M-Series What's New Video\n\n\n\n\nUnderstanding and tuning the FreeBSD Scheduler \n\n```\nOccasionally I noticed that the system would not quickly process the\ntasks i need done, but instead prefer other, longrunning tasks. I\nfigured it must be related to the scheduler, and decided it hates me.\n\nA closer look shows the behaviour as follows (single CPU):\n\nLets run an I/O-active task, e.g, postgres VACUUM that would\ncontinuously read from big files (while doing compute as well [1]):\n\n\n pool alloc free read write read write\n cache - - - - - -\n ada1s4 7.08G 10.9G 1.58K 0 12.9M 0\n\n\nNow start an endless loop:\n\nwhile true; do :; done\n\nAnd the effect is:\n\n\n pool alloc free read write read write\n cache - - - - - -\n ada1s4 7.08G 10.9G 9 0 76.8K 0\n\n\nThe VACUUM gets almost stuck! This figures with WCPU in \"top\":\n\n\n PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND\n 85583 root 99 0 7044K 1944K RUN 1:06 92.21% bash\n 53005 pgsql 52 0 620M 91856K RUN 5:47 0.50% postgres\n\n\nHacking on kern.sched.quantum makes it quite a bit better:\n\nsysctl kern.sched.quantum=1\n\nkern.sched.quantum: 94488 -> 7874\n\n\n pool alloc free read write read write\n cache - - - - - -\n ada1s4 7.08G 10.9G 395 0 3.12M 0\n \n PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND\n 85583 root 94 0 7044K 1944K RUN 4:13 70.80% bash\n 53005 pgsql 52 0 276M 91856K RUN 5:52 11.83% postgres\n\n\nNow, as usual, the \"root-cause\" questions arise: What exactly does\nthis \"quantum\"? Is this solution a workaround, i.e. actually something\nelse is wrong, and has it tradeoff in other situations? Or otherwise,\nwhy is such a default value chosen, which appears to be ill-deceived?\n\nThe docs for the quantum parameter are a bit unsatisfying - they say\nits the max num of ticks a process gets - and what happens when\nthey're exhausted? If by default the endless loop is actually allowed\nto continue running for 94k ticks (or 94ms, more likely) uninterrupted,\nthen that explains the perceived behaviour - buts thats certainly not\nwhat a scheduler should do when other procs are ready to run.\n\n11.1-RELEASE-p7, kern.hz=200. Switching tickless mode on or off does\nnot influence the matter. Starting the endless loop with \"nice\" does\nnot influence the matter.\n\n[1]\nA pure-I/O job without compute load, like \"dd\", does not show\nthis behaviour. Also, when other tasks are running, the unjust\nbehaviour is not so stongly pronounced.\n```\n\n\n\naarch64 support added\n\n\n I have committed about adding initial support for aarch64.\n\n\n\nbooting log on RaspberryPI3:\n\n\n```\n boot NetBSD/evbarm (aarch64)\n Drop to EL1...OK\n Creating VA=PA tables\n Creating KSEG tables\n Creating KVA=PA tables\n Creating devmap tables\n MMU Enable...OK\n VSTART = ffffffc000001ff4\n FDT<3ab46000> devmap cpufunc bootstrap consinit ok\n uboot: args 0x3ab46000, 0, 0, 0\n\nNetBSD/evbarm (fdt) booting ...\nFDT /memory [0] @ 0x0 size 0x3b000000\nMEM: add 0-3b000000\nMEM: res 0-1000\nMEM: res 3ab46000-3ab4a000\nUsable memory:\n 1000 - 3ab45fff\n 3ab4a000 - 3affffff\ninitarm: kernel phys start 1000000 end 17bd000\nMEM: res 1000000-17bd000\nbootargs: root=axe0\n 1000 - ffffff\n 17bd000 - 3ab45fff\n 3ab4a000 - 3affffff\n------------------------------------------\nkern_vtopdiff = 0xffffffbfff000000\nphysical_start = 0x0000000000001000\nkernel_start_phys = 0x0000000001000000\nkernel_end_phys = 0x00000000017bd000\nphysical_end = 0x000000003ab45000\nVM_MIN_KERNEL_ADDRESS = 0xffffffc000000000\nkernel_start_l2 = 0xffffffc000000000\nkernel_start = 0xffffffc000000000\nkernel_end = 0xffffffc0007bd000\nkernel_end_l2 = 0xffffffc000800000\n(kernel va area)\n(devmap va area)\nVM_MAX_KERNEL_ADDRESS = 0xffffffffffe00000\n------------------------------------------\nCopyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,\n 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,\n 2018 The NetBSD Foundation, Inc. All rights reserved.\nCopyright (c) 1982, 1986, 1989, 1991, 1993\n The Regents of the University of California. All rights reserved.\n\nNetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018\n ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64\ntotal memory = 936 MB\navail memory = 877 MB\n\n\n…\n\nStarting local daemons:.\nUpdating motd.\nStarting sshd.\nStarting inetd.\nStarting cron.\nThe following components reported failures:\n /etc/rc.d/swap2\nSee /var/run/rc.log for more information.\nFri Mar 30 12:35:31 JST 2018\n\nNetBSD/evbarm (rpi3) (console)\n\nlogin: root\nLast login: Fri Mar 30 12:30:24 2018 on console\n\nrpi3# uname -ap\nNetBSD rpi3 8.99.14 NetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018 ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64 evbarm aarch64\nrpi3#\n\n\n```\n\n\n Now, multiuser mode works stably on fdt based boards (RPI3,SUNXI,TEGRA). But there are still some problems, more time is required for release. also SMP is not yet. See sys/arch/aarch64/aarch64/TODO for more detail. Especially the problems around TLS of rtld, and C++ stack unwindings are too difficult for me to solve, I give up and need someone's help (^o^)/ Since C++ doesn't work, ATF also doesn't work. If the ATF works, it will clarify more issues.\n \n sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. One evbarm/conf/GENERIC64 kernel binary supports all fdt (bcm2837,sunxi,tegra) based boards. While on 32bit, sys/arch/evbarm/conf/GENERIC will support all fdt based boards...but doesn't work yet. (WIP)\n \n My deepest appreciation goes to Tohru Nishimura (nisimura@) whose writes vector handlers, context switchings, and so on. and his comments and suggestions were innumerably valuable. I would also like to thank Nick Hudson (skrll@) and Jared McNeill (jmcneill@) whose added support FDT and integrated into evbarm. Finally, I would like to thank Matt Thomas (matt@) whose commited aarch64\n toolchains and preliminary support for aarch64.\n\n\n\n\nBeastie Bits\n\n\n5 Reasons to Use FreeBSD in 2018\nRewriting Intel gigabit network driver in Rust\nRecruiting to make Elastic Search on FreeBSD better\nWindows Server 2019 Preview, in bhyve on FreeBSD\n“SSH Mastery, 2nd ed” in hardcover\n\n\n\n\nFeedback/Questions\n\n\nJason - ZFS Transfer option\nLuis - ZFS Pools\nClonOS \nMichael - Tech Conferences\nanonymous - BSD trash on removable drives\n\n\n\n\n\nSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv\n\n\n","content_html":"OpenBSD 6.3 and DragonflyBSD 5.2 are released, bug fix for disappearing files in OpenZFS on Linux (and only Linux), understanding the FreeBSD CPU scheduler, NetBSD on RPI3, thoughts on being a committer for 20 years, and 5 reasons to use FreeBSD in 2018.
\n\n\n Improved HW support, including:\n SMP support on OpenBSD/arm64 platforms\n vmm/vmd improvements:\n IEEE 802.11 wireless stack improvements\n Generic network stack improvements\n Installer improvements\n Routing daemons and other userland network improvements\n Security improvements\n dhclient(8) improvements\n Assorted improvements\n OpenSMTPD 6.0.4\n OpenSSH 7.7\n LibreSSL 2.7.2
\n\n\nBig-ticket items\n Meltdown and Spectre mitigation support\n Meltdown isolation and spectre mitigation support added. Meltdown mitigation is automatically enabled for all Intel cpus. Spectre mitigation must be enabled manually via sysctl if desired, using sysctls machdep.spectremitigation and machdep.meltdownmitigation.\n HAMMER2\n H2 has received a very large number of bug fixes and performance improvements. We can now recommend H2 as the default root filesystem in non-clustered mode.\n Clustered support is not yet available.\n ipfw Updates\n Implement state based \"redirect\", i.e. without using libalias.\n ipfw now supports all possible ICMP types.\n Fix ICMPMAXTYPE assumptions (now 40 as of this release).\n Improved graphics support\n The drm/i915 kernel driver has been updated to support Intel Coffeelake GPUs\n Add 24-bit pixel format support to the EFI frame buffer code.\n Significantly improve fbio support for the \"scfb\" XOrg driver. This allows EFI frame buffers to be used by X in situations where we do not otherwise support the GPU.\n Partly implement the FBIOBLANK ioctl for display powersaving.\n Syscons waits for drm modesetting at appropriate places, avoiding races.\n + For more details, check out the “All changes since DragonFly 5.0” section.
\n
\n Commit cc63068 caused ENOSPC error when copy a large amount of files between two directories. The reason is that the patch limits zap leaf expansion to 2 retries, and return ENOSPC when failed.
\n\n\nYesterday was the twentieth anniversary of my FreeBSD commit bit, and tomorrow will be the twentieth anniversary of my first commit. I figured I’d split the difference and write a few words about it today.
\n \nMy level of engagement with the FreeBSD project has varied greatly over the twenty years I’ve been a committer. There have been times when I worked on it full-time, and times when I did not touch it for months. The last few years, health issues and life events have consumed my time and sapped my energy, and my contributions have come in bursts. Commit statistics do not tell the whole story, though: even when not working on FreeBSD directly, I have worked on side projects which, like OpenPAM, may one day find their way into FreeBSD.
\n \nMy contributions have not been limited to code. I was the project’s first Bugmeister; I’ve served on the Security Team for a long time, and have been both Security Officer and Deputy Security Officer; I managed the last four Core Team elections and am doing so again this year.
\n \nIn return, the project has taught me much about programming and software engineering. It taught me code hygiene and the importance of clarity over cleverness; it taught me the ins and outs of revision control; it taught me the importance of good documentation, and how to write it; and it taught me good release engineering practices.
\n \nLast but not least, it has provided me with the opportunity to work with some of the best people in the field. I have the privilege today to count several of them among my friends.
\n \nFor better or worse, the FreeBSD project has shaped my career and my life. It set me on the path to information security in general and IAA in particular, and opened many a door for me. I would not be where I am now without it.
\n \nI won’t pretend to be able to tell the future. I don’t know how long I will remain active in the FreeBSD project and community. It could be another twenty years; or it could be ten, or five, or less. All I know is that FreeBSD and I still have things to teach each other, and I don’t intend to call it quits any time soon.
\n
\n\n\nSan Jose, Calif., April 10, 2018 — iXsystems, the leader in Enterprise Open Source servers and software-defined storage, announced the TrueNAS M40 and M50 as the newest high-performance models in its hybrid, unified storage product line. The TrueNAS M-Series harnesses NVMe and NVDIMM to bring all-flash array performance to the award-winning TrueNAS hybrid arrays. It also includes the Intel® Xeon® Scalable Family of Processors and supports up to 100GbE and 32Gb Fibre Channel networking. Sitting between the all-flash TrueNAS Z50 and the hybrid TrueNAS X-Series in the product line, the TrueNAS M-Series delivers up to 10 Petabytes of highly-available and flash-powered network attached storage and rounds out a comprehensive product set that has a capacity and performance option for every storage budget.
\n
\n\n\nAs a unified file, block, and object sharing solution, TrueNAS can meet the needs of file serving, backup, virtualization, media production, and private cloud users thanks to its support for the SMB, NFS, AFP, iSCSI, Fibre Channel, and S3 protocols.
\n \nAt the heart of the TrueNAS M-Series is a custom 4U, dual-controller head unit that supports up to 24 3.5” drives and comes in two models, the M40 and M50, for maximum flexibility and scalability. The TrueNAS M40 uses NVDIMMs for write cache, SSDs for read cache, and up to two external 60-bay expansion shelves that unlock up to 2PB in capacity. The TrueNAS M50 uses NVDIMMs for write caching, NVMe drives for read caching, and up to twelve external 60-bay expansion shelves to scale upwards of 10PB. The dual-controller design provides high-availability failover and non-disruptive upgrades for mission-critical enterprise environments.
\n \nBy design, the TrueNAS M-Series unleashes cutting-edge persistent memory technology for demanding performance and capacity workloads, enabling businesses to accelerate enterprise applications and deploy enterprise private clouds that are twice the capacity of previous TrueNAS models. It also supports replication to the Amazon S3, BackBlaze B2, Google Cloud, and Microsoft Azure cloud platforms and can deliver an object store using the ubiquitous S3 object storage protocol at a fraction of the cost of the public cloud.
\n
\n\n\nAs a true enterprise storage platform, the TrueNAS M50 supports very demanding performance workloads with up to four active 100GbE ports, 3TB of RAM, 32GB of NVDIMM write cache and up to 15TB of NVMe flash read cache. The TrueNAS M40 and M50 include up to 24/7 and global next-business-day support, putting IT at ease. The modular and tool-less design of the M-Series allows for easy, non-disruptive servicing and upgrading by end-users and support technicians for guaranteed uptime. TrueNAS has US-Based support provided by the engineering team that developed it, offering the rapid response that every enterprise needs.
\n
Award-Winning TrueNAS Features
\n\nAvailability
\n\n\nThe TrueNAS M40 and M50 will be generally available in April 2018 through the iXsystems global channel partner network. The TrueNAS M-Series starts at under $20,000 USD and can be easily expanded using a linear “per terabyte” pricing model. With typical compression, a Petabtye can be stored for under $100,000 USD. TrueNAS comes with an all-inclusive software suite that provides NFS, Windows SMB, iSCSI, snapshots, clones and replication.
\n
```\nOccasionally I noticed that the system would not quickly process the\ntasks i need done, but instead prefer other, longrunning tasks. I\nfigured it must be related to the scheduler, and decided it hates me.
\n\nA closer look shows the behaviour as follows (single CPU):
\n\nLets run an I/O-active task, e.g, postgres VACUUM that would\ncontinuously read from big files (while doing compute as well [1]):
\n\n\n\n\npool alloc free read write read write\n cache - - - - - -\n ada1s4 7.08G 10.9G 1.58K 0 12.9M 0
\n
Now start an endless loop:
\n\nAnd the effect is:
\n\n\n\n\npool alloc free read write read write\n cache - - - - - -\n ada1s4 7.08G 10.9G 9 0 76.8K 0
\n
The VACUUM gets almost stuck! This figures with WCPU in \"top\":
\n\n\n\n\nPID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND\n 85583 root 99 0 7044K 1944K RUN 1:06 92.21% bash\n 53005 pgsql 52 0 620M 91856K RUN 5:47 0.50% postgres
\n
Hacking on kern.sched.quantum makes it quite a bit better:
\n\nkern.sched.quantum: 94488 -> 7874
\n\n\n\n\npool alloc free read write read write\n cache - - - - - -\n ada1s4 7.08G 10.9G 395 0 3.12M 0
\n \nPID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND\n 85583 root 94 0 7044K 1944K RUN 4:13 70.80% bash\n 53005 pgsql 52 0 276M 91856K RUN 5:52 11.83% postgres
\n
Now, as usual, the \"root-cause\" questions arise: What exactly does\nthis \"quantum\"? Is this solution a workaround, i.e. actually something\nelse is wrong, and has it tradeoff in other situations? Or otherwise,\nwhy is such a default value chosen, which appears to be ill-deceived?
\n\nThe docs for the quantum parameter are a bit unsatisfying - they say\nits the max num of ticks a process gets - and what happens when\nthey're exhausted? If by default the endless loop is actually allowed\nto continue running for 94k ticks (or 94ms, more likely) uninterrupted,\nthen that explains the perceived behaviour - buts thats certainly not\nwhat a scheduler should do when other procs are ready to run.
\n\n11.1-RELEASE-p7, kern.hz=200. Switching tickless mode on or off does\nnot influence the matter. Starting the endless loop with \"nice\" does\nnot influence the matter.
\n\n[1]\nA pure-I/O job without compute load, like \"dd\", does not show\nthis behaviour. Also, when other tasks are running, the unjust\nbehaviour is not so stongly pronounced.\n```
\n\n\n\n\nI have committed about adding initial support for aarch64.
\n
```\n boot NetBSD/evbarm (aarch64)\n Drop to EL1...OK\n Creating VA=PA tables\n Creating KSEG tables\n Creating KVA=PA tables\n Creating devmap tables\n MMU Enable...OK\n VSTART = ffffffc000001ff4\n FDT<3ab46000> devmap cpufunc bootstrap consinit ok\n uboot: args 0x3ab46000, 0, 0, 0
\n\nNetBSD/evbarm (fdt) booting ...\nFDT /memory [0] @ 0x0 size 0x3b000000\nMEM: add 0-3b000000\nMEM: res 0-1000\nMEM: res 3ab46000-3ab4a000\nUsable memory:\n 1000 - 3ab45fff\n 3ab4a000 - 3affffff\ninitarm: kernel phys start 1000000 end 17bd000\nMEM: res 1000000-17bd000\nbootargs: root=axe0\n 1000 - ffffff\n 17bd000 - 3ab45fff\n 3ab4a000 - 3affffff\n------------------------------------------\nkern_vtopdiff = 0xffffffbfff000000\nphysical_start = 0x0000000000001000\nkernel_start_phys = 0x0000000001000000\nkernel_end_phys = 0x00000000017bd000\nphysical_end = 0x000000003ab45000\nVM_MIN_KERNEL_ADDRESS = 0xffffffc000000000\nkernel_start_l2 = 0xffffffc000000000\nkernel_start = 0xffffffc000000000\nkernel_end = 0xffffffc0007bd000\nkernel_end_l2 = 0xffffffc000800000\n(kernel va area)\n(devmap va area)\nVM_MAX_KERNEL_ADDRESS = 0xffffffffffe00000\n------------------------------------------\nCopyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,\n 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,\n 2018 The NetBSD Foundation, Inc. All rights reserved.\nCopyright (c) 1982, 1986, 1989, 1991, 1993\n The Regents of the University of California. All rights reserved.\n\nNetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018\n ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64\ntotal memory = 936 MB\navail memory = 877 MB\n
\n\n…
\n\nStarting local daemons:.\nUpdating motd.\nStarting sshd.\nStarting inetd.\nStarting cron.\nThe following components reported failures:\n /etc/rc.d/swap2\nSee /var/run/rc.log for more information.\nFri Mar 30 12:35:31 JST 2018\n\nNetBSD/evbarm (rpi3) (console)\n\nlogin: root\nLast login: Fri Mar 30 12:30:24 2018 on console\n\nrpi3# uname -ap\nNetBSD rpi3 8.99.14 NetBSD 8.99.14 (RPI64) #11: Fri Mar 30 12:34:19 JST 2018 ryo@moveq:/usr/home/ryo/tmp/netbsd-src-ryo-wip/sys/arch/evbarm/compile/RPI64 evbarm aarch64\nrpi3#\n
\n\n```
\n\n\n\n\nNow, multiuser mode works stably on fdt based boards (RPI3,SUNXI,TEGRA). But there are still some problems, more time is required for release. also SMP is not yet. See sys/arch/aarch64/aarch64/TODO for more detail. Especially the problems around TLS of rtld, and C++ stack unwindings are too difficult for me to solve, I give up and need someone's help (^o^)/ Since C++ doesn't work, ATF also doesn't work. If the ATF works, it will clarify more issues.
\n \nsys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. One evbarm/conf/GENERIC64 kernel binary supports all fdt (bcm2837,sunxi,tegra) based boards. While on 32bit, sys/arch/evbarm/conf/GENERIC will support all fdt based boards...but doesn't work yet. (WIP)
\n \nMy deepest appreciation goes to Tohru Nishimura (nisimura@) whose writes vector handlers, context switchings, and so on. and his comments and suggestions were innumerably valuable. I would also like to thank Nick Hudson (skrll@) and Jared McNeill (jmcneill@) whose added support FDT and integrated into evbarm. Finally, I would like to thank Matt Thomas (matt@) whose commited aarch64\n toolchains and preliminary support for aarch64.
\n
TrueOS Stable 18.03 released, a look at F-stack, the secret to an open source business model, intro to jails and jail networking, FreeBSD Foundation March update, and the ipsec Errata.
\n\n\n\n\nThe TrueOS team is pleased to announce the availability of a new STABLE release of the TrueOS project (version 18.03). This is a special release due to the security issues impacting the computing world since the beginning of 2018. In particular, mitigating the “Meltdown” and “Spectre” system exploits make it necessary to update the entire package ecosystem for TrueOS. This release does not replace the scheduled June STABLE update, but provides the necessary and expected security updates for the STABLE release branch of TrueOS, even though this is part-way through our normal release cycle.
\n
Important changes between version 17.12 and 18.03
\n\n\n\n\nMost systems will need microcode updates for additional Spectre mitigations. The microcode updates are not enabled by default. This work is considered experimental because it is in active development by the upstream vendors. If desired, the microcode updates are available with the new devcpu-data package, which is available in the Appcafe. Install this package and enable the new microcode_update service to apply the latest runtime code when booting the system.
\n
Important security-based package updates
\n\nAll pre-compiled packages for this release are built with the latest versions of LLVM/Clang, unless the package explicitly requires GCC. These packages also utilize the latest compile-time mitigations for memory-access security concerns.
\n\n\nF-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API. http://www.f-stack.org
\n
Introduction\nWith the rapid development of NIC, the poor performance of data packets processing with Linux kernel has become the bottleneck. However, the rapid development of the Internet needs high performance of network processing, kernel bypass has caught more and more attentions. There are various similar technologies appear, such as DPDK, NETMAP and PF_RING. The main idea of kernel bypass is that Linux is only used to deal with control flow, all data streams are processed in user space. Therefore, kernel bypass can avoid performance bottlenecks caused by kernel packet copying, thread scheduling, system calls and interrupts. Furthermore, kernel bypass can achieve higher performance with multi optimizing methods. Within various techniques, DPDK has been widely used because of its more thorough isolation from kernel scheduling and active community support.
F-Stack is an open source network framework with high performance based on DPDK. With following characteristics
\n\nHistory
\n\n\nIn order to deal with the increasingly severe DDoS attacks, authorized DNS server of Tencent Cloud DNSPod switched from Gigabit Ethernet to 10-Gigabit at the end of 2012. We faced several options, one is to continue to use the original model another is to use kernel bypass technology. After several rounds of investigation, we finally chose to develop our next generation of DNS server based on DPDK. The reason is DPDK provides ultra-high performance and can be seamlessly extended to 40G, or even 100G NIC in the future.
\n \nAfter several months of development and testing, DKDNS, high-performance DNS server based on DPDK officially released in October 2013. It's capable of achieving up to 11 million QPS with a single 10GE port and 18.2 million QPS with two 10GE ports. And then we developed a user-space TCP/IP stack called F-Stack that can process 0.6 million RPS with a single 10GE port.
\n \nWith the fast growth of Tencent Cloud, more and more services need higher network access performance. Meanwhile, F-Stack was continuous improving driven by the business growth, and ultimately developed into a general network access framework. But this TCP/IP stack couldn't meet the needs of these services while continue to develop and maintain a complete network stack will cost high, we've tried several plans and finally determined to port FreeBSD(11.0 stable) TCP/IP stack into F-Stack. Thus, we can reduce the cost of maintenance and follow up the improvement from community quickly.Thanks to libplebnet and libuinet, this work becomes a lot easier.
\n \nWith the rapid development of all kinds of application, in order to help different APPs quick and easily use F-Stack, F-Stack has integrated Nginx, Redis and other commonly used APPs, and a micro thread framework, and provides a standard Epoll/Kqueue interface.
\n \nCurrently, besides authorized DNS server of DNSPod, there are various products in Tencent Cloud has used the F-Stack, such as HttpDNS (D+), COS access module, CDN access module, etc..
\n
iXsystems
\n\n\n There is a good chance you’ve never heard of open source software and an even greater one that you’re using it every day without even realizing it. Open source software is computer software that is available under a variety of licenses that all encourage the sharing of the software and its underlying source code. Open source has powered the internet from day one and today powers the cloud and just about everything connected to it from your mobile phone to virtually every internet of things device.\n FreeNAS is one of two open source operating systems that my company, iXsystems, develops and distributes free of charge and is at the heart of our line of TrueNAS enterprise storage products. While some of our competitors sell storage software similar to FreeNAS, we not only give it away but also do so with truly no strings attached -- competitors can and do take FreeNAS and build products based on it with zero obligation to share their changes. The freedom to do so is the fundamental tenet of permissively licensed open source software, and while it sounds self-defeating to be this generous, we’ve proven that leadership, not licensing, is the true secret to a successful open source business model.\n We each have our own personal definition of what is fair when it comes to open source. At iXsystems, we made a conscious decision to base FreeNAS and TrueOS on the FreeBSD operating system developed by the FreeBSD project. We stand on the shoulders of giants by using FreeBSD and we consider it quite reasonable to give back on the same generous terms that the FreeBSD project offers us. We could be selective in what we provide free of charge, but we believe that doing so would be short-sighted. In the long game we’re playing, the leadership we provide over the open source projects we produce is infinitely more important than any restrictions provided by the licenses of those and other open source projects.\n Twenty years in, we have no reason to change our free-software-on-great-hardware business model and giving away the software has brought an unexpected side-benefit: the largest Q/A department in the world, staffed by our passionate users who volunteer to let us know every thought they have about our software. We wouldn’t change a thing, and I encourage you to find exactly what win-win goodwill you and your company can provide to your constituents to make them not just a customer base but a community.
\n\n\nJails basically partition a FreeBSD system into various isolated sub-systems called jails. The syscall and userspace tools first appeared in FreeBSD 4.0 (~ March 2000) with subsequent releases expanding functionality and improving existing features as well as usability.\n + For Linux users, jails are similar to LXC, used for resource/process isolation. Unlike LXC however, jails are a first-class concept and are well integrated into the base system. Essentially however, both offer a chroot-with-extra-separation feeling.\n Setting up a jail is a fairly simple process, which can essentially be split into three steps:\n + Place the stuff you want to run and the stuff it needs to run somewhere on your filesystem.\n + Add some basic configuration for the jail in jail.conf.\n + Fire up the jail.\n To confirm that the jail started successfully we can use the jls utility:\n We can now enter the jailed environment by using jexec, which will by default execute a root shell inside the named jail\n A jail can only see and use addresses that have been passed down to it by the parent system. This creates a slight problem with the loopback address: The host would probably like to keep that address to itself and not share it with any jail.\n Because of this, the loopback-address inside a jail is emulated by the system:\n + 127.0.0.1 is an alias for the first IPv4-address assigned to the jail.\n + ::1 is an alias for the first IPv6-address assigned to the jail.\n While this looks simple enough and usually works just fine[tm], it is also a source of many problems. Just imagine if your jail has only one single global IPv4 assigned to it. A daemon binding its (possibly unsecured) control port to the loopback-address would then unwillingly be exposed to the rest of the internet, which is hardly ever a good idea.\n + So, create an extra loopback adapter, and make the first IP in each jail a private loopback address\n + The tutorial goes on to cover making multiple jails share a single public IP address using NAT\n + It also covers more advanced concepts like ‘thin’ jails, to save some disk space if you are going to create a large number of jails, and how to upgrade them after the fact\n + Finally, it covers the integration with a lot of common tools, like identifying and filter jailed processes using top and ps, or using the package managers support for jails to install packages in a jail from the outside.
\n
curl -C - -O https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest.iso\ncurl -C - -O https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest-USB.img.bz2\ncurl -C - -O https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest.vmwarevm.tar.bz2\n
\n\nA generated changelog is here:\n\nhttps://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos.html#20180329T002644Z\n
\n\nThe full build bits directory, for those interested, is here in Manta:\n\n/Joyent_Dev/public/SmartOS/20180329T002644Z\n
\n\n\nNext year, we have the opportunity to have a BSD track, similar to the BSD Devroom at FOSDEM. We are looking for some volunteers in Southern California who can help organize this one or two-day event and help us educate more people about the BSDs. Let us know if you\n would like to help with this effort.
\n
\nRoll Call: #WhoUsesFreeBSD
\n
\nMany of you probably saw our post on social media asking Who Uses FreeBSD. Please help us answer this question to assist us in determining FreeBSD market share data, promote how companies are successfully using FreeBSD to encourage more companies to embrace\n FreeBSD, and to update the list of users on our website. Knowing who uses FreeBSD helps our contributors know where to look for jobs; knowing what universities teach with FreeBSD, helps companies know where to recruit, and knowing what products use FreeBSD helps us determine what features and technologies to support.
\n
\nNew Hosting Partner: Oregon State University Open Source Lab
\n
Tarsnap
\n\nSecond round of ZFS improvements in FreeBSD, Postgres finds that non-FreeBSD/non-Illumos systems are corrupting data, interview with Kevin Bowling, BSDCan list of talks, and cryptographic right answers.
\n\n\nOne of the first tasks during the pool load process is to parse a config provided from userland that describes what devices the pool is composed of. A vdev tree is generated from that config, and then all the vdevs are opened.\n The Meta Object Set (MOS) of the pool is accessed, and several metadata objects that are necessary to load the pool are read. The exact configuration of the pool is also stored inside the MOS. Since the configuration provided from userland is external and might not accurately describe the vdev tree of the pool at the txg that is being loaded, it cannot be relied upon to safely operate the pool. For that reason, the configuration in the MOS is read early on. In the past, the two configurations were compared together and if there was a mismatch then the load process was aborted and an error was returned.\n The latter was a good way to ensure a pool does not get corrupted, however it made the pool load process needlessly fragile in cases where the vdev configuration changed or the userland configuration was outdated. Since the MOS is stored in 3 copies, the configuration provided by userland doesn't have to be perfect in order to read its contents. Hence, a new approach has been adopted: The pool is first opened with the untrusted userland configuration just so that the real configuration can be read from the MOS. The trusted MOS configuration is then used to generate a new vdev tree and the pool is re-opened.\n When the pool is opened with an untrusted configuration, writes are disabled to avoid accidentally damaging it. During reads, some sanity checks are performed on block pointers to see if each DVA points to a known vdev; when the configuration is untrusted, instead of panicking the system if those checks fail we simply avoid issuing reads to the invalid DVAs.\n This new two-step pool load process now allows rewinding pools across vdev tree changes such as device replacement, addition, etc. Loading a pool from an external config file in a clustering environment also becomes much safer now since the pool will import even if the config is outdated and didn't, for instance, register a recent device addition.\n With this code in place, it became relatively easy to implement a long-sought-after feature: the ability to import a pool with missing top level (i.e. non-redundant) devices. Note that since this almost guarantees some loss Of data, this feature is for now restricted to a read-only import.
\n\n\nSome time ago I ran into an issue where a user encountered data corruption after a storage error. PostgreSQL played a part in that corruption by allowing checkpoint what should've been a fatal error.\n TL;DR: Pg should PANIC on fsync() EIO return. Retrying fsync() is not OK at least on Linux. When fsync() returns success it means \"all writes since the last fsync have hit disk\" but we assume it means \"all writes since the last SUCCESSFUL fsync have hit disk\".\n Pg wrote some blocks, which went to OS dirty buffers for writeback. Writeback failed due to an underlying storage error. The block I/O layer and XFS marked the writeback page as failed (ASEIO), but had no way to tell the app about the failure. When Pg called fsync() on the FD during the next checkpoint, fsync() returned EIO because of the flagged page, to tell Pg that a previous async write failed. Pg treated the checkpoint as failed and didn't advance the redo start position in the control file.\n + All good so far.\n But then we retried the checkpoint, which retried the fsync(). The retry succeeded, because the prior fsync() *cleared the ASEIO bad page flag*.\n The write never made it to disk, but we completed the checkpoint, and merrily carried on our way. Whoops, data loss.\n The clear-error-and-continue behaviour of fsync is not documented as far as I can tell. Nor is fsync() returning EIO unless you have a very new linux man-pages with the patch I wrote to add it. But from what I can see in the POSIX standard we are not given any guarantees about what happens on fsync() failure at all, so we're probably wrong to assume that retrying fsync() is safe.\n We already PANIC on fsync() failure for WAL segments. We just need to do the same for data forks at least for EIO. This isn't as bad as it seems because AFAICS fsync only returns EIO in cases where we should be stopping the world anyway, and many FSes will do that for us.\n + Upon further looking, it turns out it is not just Linux brain damage:\n Apparently I was too optimistic. I had looked only at FreeBSD, which keeps the page around and dirties it so we can retry, but the other BSDs apparently don't (FreeBSD changed that in 1999).\n From what I can tell from the sources below, we have: Linux, OpenBSD, NetBSD: retrying fsync() after EIO lies\n FreeBSD, Illumos: retrying fsync() after EIO tells the truth\n + NetBSD PR to solve the issues \n + I/O errors are not reported back to fsync at all.\n + Write errors during genfs_putpages that fail for any reason other than ENOMEM cause the data to be semi-silently discarded.\n + It appears that UVM pages are marked clean when they're selected to be written out, not after the write succeeds; so there are a bunch of potential races when writes fail.\n + It appears that write errors for buffercache buffers are semi-silently discarded as well.
\n
iXsystems
\n\n\nWe’re less interested in empowering developers and a lot more pessimistic about the prospects of getting this stuff right.
But if you’re a developer and not a cryptography engineer, you shouldn’t do any of that. You should keep things simple and conventional and easy to analyze; “boring”, as the Google TLS people would say.
\n\n\nCryptographic Right Answers
Encrypting Data
\n\n\nPercival, 2009: AES-CTR with HMAC.\n Ptacek, 2015: (1) NaCl/libsodium’s default, (2) ChaCha20-Poly1305, or (3) AES-GCM.\n Latacora, 2018: KMS or XSalsa20+Poly1305
\n
\n\n\nPercival, 2009: Use 256-bit keys.\n Ptacek, 2015: Use 256-bit keys.\n Latacora, 2018: Go ahead and use 256 bit keys.
\n
\n\n\nPercival, 2009: Use HMAC.\n Ptacek, 2015: Yep, use HMAC.\n Latacora, 2018: Still HMAC.
\n
\n\n\nPercival, 2009: Use SHA256 (SHA-2).\n Ptacek, 2015: Use SHA-2.\n Latacora, 2018: Still SHA-2.
\n
\n\n\nPercival, 2009: Use 256-bit random numbers.\n Ptacek, 2015: Use 256-bit random numbers.\n Latacora, 2018: Use 256-bit random numbers.
\n
\n\n\nPercival, 2009: scrypt or PBKDF2.\n Ptacek, 2015: In order of preference, use scrypt, bcrypt, and then if nothing else is available PBKDF2.\n Latacora, 2018: In order of preference, use scrypt, argon2, bcrypt, and then if nothing else is available PBKDF2.
\n
\n\n\nPercival, 2009: Use RSAES-OAEP with SHA256 and MGF1+SHA256 bzzrt pop ffssssssst exponent 65537.\n Ptacek, 2015: Use NaCl/libsodium (box / cryptobox).\n Latacora, 2018: Use Nacl/libsodium (box / cryptobox).
\n
\n\n\nPercival, 2009: Use RSASSA-PSS with SHA256 then MGF1+SHA256 in tricolor systemic silicate orientation.\n Ptacek, 2015: Use Nacl, Ed25519, or RFC6979.\n Latacora, 2018: Use Nacl or Ed25519.
\n
\n\n\nPercival, 2009: Operate over the 2048-bit Group #14 with a generator of 2.\n Ptacek, 2015: Probably still DH-2048, or Nacl.\n Latacora, 2018: Probably nothing. Or use Curve25519.
\n
\n\n\nPercival, 2009: Use OpenSSL.\n Ptacek, 2015: Remains: OpenSSL, or BoringSSL if you can. Or just use AWS ELBs\n Latacora, 2018: Use AWS ALB/ELB or OpenSSL, with LetsEncrypt
\n
\n\n\nPercival, 2009: Distribute the server’s public RSA key with the client code, and do not use SSL.\n Ptacek, 2015: Use OpenSSL, or BoringSSL if you can. Or just use AWS ELBs\n Latacora, 2018: Use AWS ALB/ELB or OpenSSL, with LetsEncrypt
\n
\n\n\nPercival, 2009: Use Tarsnap.\n Ptacek, 2015: Use Tarsnap.\n Latacora, 2018: Store PMAC-SIV-encrypted arc files to S3 and save fingerprints of your backups to an ERC20-compatible blockchain. Just kidding. You should still use Tarsnap.
\n
\n\n\nI am adding IPv6 addresses to each of my servers. This post assumes the server is up and running FreeBSD 11.1 and you already have an IPv6 address block. This does not cover the creation of an IPv6 tunnel, such as that provided by HE.net. This assumes native IPv6.
\n \nIn this post, I am using the IPv6 addresses from the IPv6 Address Prefix Reserved for Documentation (i.e. 2001:DB8::/32). You should use your own addresses.
\n \nThe IPv6 block I have been assigned is 2001:DB8:1001:8d00/64.
\n \nI added this to /etc/rc.conf:
\n
\nipv6_activate_all_interfaces=\"YES\"\nipv6_defaultrouter=\"2001:DB8:1001:8d00::1\"\nifconfig_em1_ipv6=\"inet6 2001:DB8:1001:8d00:d389:119c:9b57:396b prefixlen 64 accept_rtadv\" # ns1\n
\n\n\nThe IPv6 address I have assigned to this host is completely random (with the given block). I found a random IPv6 address generator and used it to select d389:119c:9b57:396b as the address for this service within my address block.
\n \nI don’t have the reference, but I did read that randomly selecting addresses within your block is a better approach.
\n \nIn order to invoke these changes without rebooting, I issued these commands:
\n
```\n[dan@tallboy:~] $ sudo ifconfig em1 inet6 2001:DB8:1001:8d00:d389:119c:9b57:396b prefixlen 64 accept_rtadv\n[dan@tallboy:~] $
\n\n[dan@tallboy:~] $ sudo route add -inet6 default 2001:DB8:1001:8d00::1\nadd net default: gateway 2001:DB8:1001:8d00::1\n```
\n\n\n\n\nIf you do the route add first, you will get this error:
\n
\n[dan@tallboy:~] $ sudo route add -inet6 default 2001:DB8:1001:8d00::1\nroute: writing to routing socket: Network is unreachable\nadd net default: gateway 2001:DB8:1001:8d00::1 fib 0: Network is unreachable\n
Tarsnap
\n\nNew ZFS features landing in FreeBSD, MAP_STACK for OpenBSD, how to write safer C code with Clang’s address sanitizer, Michael W. Lucas on sponsor gifts, TCP blackbox recorder, and Dell disk system hacking.
\n\n9188 increase size of dbuf cache to reduce indirect block decompression
\n\n\nWith compressed ARC (6950) we use up to 25% of our CPU to decompress indirect blocks, under a workload of random cached reads. To reduce this decompression cost, we would like to increase the size of the dbuf cache so that more indirect blocks can be stored uncompressed.\n If we are caching entire large files of recordsize=8K, the indirect blocks use 1/64th as much memory as the data blocks (assuming they have the same compression ratio). We suggest making the dbuf cache be 1/32nd of all memory, so that in this scenario we should be able to keep all the indirect blocks decompressed in the dbuf cache. (We want it to be more than the 1/64th that the indirect blocks would use because we need to cache other stuff in the dbuf cache as well.)\n In real world workloads, this won't help as dramatically as the example above, but we think it's still worth it because the risk of decreasing performance is low. The potential negative performance impact is that we will be slightly reducing the size of the ARC (by ~3%).
\n
9166 zfs storage pool checkpoint
\n\n\nThe idea of Storage Pool Checkpoint (aka zpool checkpoint) deals with exactly that. It can be thought of as a “pool-wide snapshot” (or a variation of extreme rewind that doesn’t corrupt your data). It remembers the entire state of the pool at the point that it was taken and the user can revert back to it later or discard it. Its generic use case is an administrator that is about to perform a set of destructive actions to ZFS as part of a critical procedure. She takes a checkpoint of the pool before performing the actions, then rewinds back to it if one of them fails or puts the pool into an unexpected state. Otherwise, she discards it. With the assumption that no one else is making modifications to ZFS, she basically wraps all these actions into a “high-level transaction”.
\n
8484 Implement aggregate sum and use for arc counters
\n\n\nIn pursuit of improving performance on multi-core systems, we should implements fanned out counters and use them to improve the performance of some of the arc statistics. These stats are updated extremely frequently, and can consume a significant amount of CPU time.
\n
And a small bug fix authored by me:
\n arcloancompressedbuf() increments arcloanedbytes by psize unconditionally In the case of zfscompressedarcenabled=0, when the buf is returned via arcreturnbuf(), if ARCBUFCOMPRESSED(buf) is false, then arcloanedbytes is decremented by lsize, not psize.\n Switch to using arcbufsize(buf), instead of psize, which will return psize or lsize, depending on the result of ARCBUF_COMPRESSED(buf).
\n\n\nAlmost 2 decades ago we started work on W^X. The concept was simple. Pages that are writable, should not be executable. We applied this concept object by object, trying to seperate objects with different qualities to different pages. The first one we handled was the signal trampoline at the top of the stack. We just kept making changes in the same vein. Eventually W^X came to some of our kernel address spaces also.\n The fundamental concept is that an object should only have the\n permissions necessary, and any other operation should fault. The only permission separations we have are kernel vs userland, and then read, write, and execute.\n How about we add another new permission! This is not a hardware permission, but a software permission. It is opportunistically enforced by the kernel.\n the permission is MAPSTACK. If you want to use memory as a stack, you must mmap it with that flag bit. The kernel does so automatically for the stack region of a process's stack. Two other types of stack occur: thread stacks, and alternate signal stacks. Those are handled in clever ways.\n When a system call happens, we check if the stack-pointer register points to such a page. If it doesn't, the program is killed. We have tightened the ABI. You may no longer point your stack register at non-stack memory. You'll be killed. This checking code is MI, so it works for all platforms.\n Since page-permissions are generally done on page boundaries, there is caveat that thread and altstacks must now be page-sized and page-aligned, so that we can enforce the MAPSTACK attribute correctly. It is possible that a few ports need some massaging to satisfy this condition, but we haven't found any which break yet. A syslog_r has been added so that we can identify these failure cases. Also, the faulting cases are quite verbose for now, to help identify the programs we need to repair.
\n
\n\n\nWe wanted to improve our password strength algorithm, and decided to go for the industry-standard zxcvbn, from the people at Dropbox. Our web front-end would use the default Javascript library, and for mobile and desktop, we chose to use the C implementation as it was the lowest common denominator for all platforms.\n Bootstrapping all of this together was done pretty fast. I had toyed around with a few sample passwords so I decided to run it through the test suite we had for the previous password strength evaluator. The test generates a large number of random passwords according to different rules and expects the strength to be in a given range. But the test runner kept crashing with segmentation faults.\n It turns out the library has a lot of buffer overflow cases that are usually \"harmless\", but eventually crash your program when you run the evaluator function too much. I started fixing the cases I could see, but reading someone else's algorithms to track down tiny memory errors got old pretty fast. I needed a tool to help me.\n That's when I thought of Clang's Address Sanitizer.\n AddressSanitizer is a fast memory error detector. It consists of a compiler instrumentation module and a run-time library\n Let's try the sanitizer on a simple program. We'll allocate a buffer on the heap, copy each character of a string into it, and print it to standard output.\n + The site walks through a simple example which contains an error, it writes past the end of a buffer\n + The code works as expected, and nothing bad happens. It must be fine…\n + Then they compile it again with the address sanitizer actived\n So what can we gather from that pile of hex? Let's go through it line by line.\n AddressSanitizer found a heap buffer overflow at 0x60200000ef3d, a seemingly valid address (not NULL or any other clearly faulty value).\n + ASAN points directly to the line of code that is causing the problem\n We're writing outside of the heap in this instruction. And AddressSanitizer isn't having it.\n This is definitely one of my favorite indications. In addition to telling which line in the code failed and where in the memory the failure happened, you get a complete description of the closest allocated region in memory (which is probably the region you were trying to access).\n + They then walk through combining this with lldb, the Clang debugger, to actually interactively inspect the state of the problem when an invalid memory access happens\n Back to my practical case, how did I put the address sanitizer to good use? I simply ran the test suite, compiled with the sanitizer, with lldb. Sure enough, it stopped on every line that could cause a crash. It turns out there were many cases where zxcvbn-c wrote past the end of allocated buffers, on the heap and on the stack. I fixed those cases in the C library and ran the tests again. Not a segfault in sight!\n I've used memory tools in the past, but they were usually unwieldy, or put such a toll on performance that they were useless in any real-life case. Clang's address sanitizer turned out to be detailed, reliable, and surprisingly easy to use. I've heard of the miracles of Valgrind but macOS hardly supports it, making it a pain to use on my MacBook Pro.\n Coupled with Clang's static analyzer, AddressSanitizer is going to become a mandatory stop for evaluating code quality. It's also going to be the first tool I grab when facing confusing memory issues. There are many more case where I could use early failure and memory history to debug my code. For example, if a program crashes when accessing member of a deallocated object, we could easily trace the event that caused the deallocation, saving hours of adding and reading logs to retrace just what happened.
\n
\n\n\nNote the little stack of customs forms off to the side. It’s like I’ve learned a lesson from standing at the post office counter filling out those stupid forms. Sponsors should get their books soon.
\n \nThis seems like an apropos moment to talk about what I do for print sponsors. I say I send them “a gift,” but what does that really mean? The obvious thing to ship them is a copy of the book I’ve written. Flat-out selling print books online has tax implications, though.
\n \nSponsors might have guessed that they’d get a copy of the book. But I shipped them the hardcover, which isn’t my usual practice.
\n \nThat’s because I send sponsors a gift. As it’s a gift, I get to choose what I send. I want to send them something nice, to encourage them to sponsor another book. It makes no sense for me to send a sponsor a Singing Wedgie-O-Gram. (Well, maybe a couple sponsors. You know who you are.)
\n \nThe poor bastards who bought into my scam–er, sponsored my untitled book–have no idea what’s coming. As of right now, their sensible guesses are woefully incomplete.
\n \nFuture books? They might get a copy of the book. They might get book plus something. They might just get the something. Folks who sponsor the jails book might get a cake with a file in it. Who knows?
\n \nIt’s a gift. It’s my job to make that gift worthwhile.
\n \nAnd to amuse myself. Because otherwise, what’s the point?
\n
\n\n\nKDE4 has been rudely moved aside on FreeBSD. It still installs (use x11/kde4) and should update without a problem, but this is another step towards adding modern KDE (Plasma 5 and Applications) to the official FreeBSD Ports tree.\n This has taken a long time mostly for administrative reasons, getting all the bits lined up so that people sticking with KDE4 (which, right now, would be everyone using KDE from official ports and packages on FreeBSD) don’t end up with a broken desktop. We don’t want that. But now that everything Qt4 and kdelibs4-based has been moved aside by suffixing it with -kde4, we have the unsuffixed names free to indicate the latest-and-greatest from upstream.
\n \nKDE4 users will see a lot of packages moving around and being renamed, but no functional changes. Curiously, the KDE4 desktop depends on Qt5 and KDE Frameworks 5 — and it has for quite some time already, because the Oxygen icons are shared with KDE Frameworks, but primarily because FileLight was updated to the modern KDE Applications version some time ago (the KDE4 version had some serious bugs, although I can not remember what they were). Now that the names are cleaned up, we could consider giving KDE4 users the buggy version back.
\n \nFrom here on, we’ve got the following things lined up:
\n
\n\n\nSo we’ve been saying Real Soon Now ™ for years, but things are Realer Sooner Nower ™ now.
\n
\n\n\nA while back I reviewed the Dell FS12-NV7 – a 2U rack server being sold cheap by all and sundry. It’s a powerful box, even by modern standards, but one of its big drawbacks is the disk system it comes with. But it needn’t be.
\n \nThere are two viable solutions, depending on what you want to do. You can make use of the SAS backplane, using SAS and/or SATA drives, or you can go for fewer SATA drives and free up one or more PCIe slots as Plan B. You probably have an FS12 because it looks good for building a drive array (or even FreeNAS) so I’ll deal with Plan A first.
\n \nLike most Dell servers, this comes with a Dell PERC RAID SAS controller – a PERC6/i to be precise. This ‘I’ means it has internal connectors; the /E is the same but its sockets are external.
\n \nThe PERC connects to a twelve-slot backplane forming a drive array at the front of the box. More on the backplane later; it’s the PERCs you need to worry about.
\n \nThe PERC6 is actually an LSI Megaraid 1078 card, which is just the thing you need if you’re running an operating system like Windows that doesn’t support a volume manager, striping and other grown-up stuff. Or if your OS does have these features, but you just don’t trust it. If you are running such an OS you may as well stick to the PERC6, and good luck to you. If you’re using BSD (including FreeNAS), Solaris or a Linux distribution that handles disk arrays, read on. The PERC6 is a solution to a problem you probably don’t have, but in all other respects its a turkey. You really want a straightforward HBA (Host Bus Adapter) that allows your clever operating system to talk directly with the drives.
\n \nAny SAS card based on the 1078 (such as the PERC6) is likely to have problems with drives larger than 2Tb. I’m not completely sure why, but I suspect it only applies to SATA. Unfortunately I don’t have any very large SAS drives to test this theory. A 2Tb limit isn’t really such a problem when you’re talking about a high performance array, as lots of small drives are a better option anyway. But it does matter if you’re building a very large datastore and don’t mind slower access and very significant resilvering times when you replace a drive. And for large datastores, very large SATA drives save you a whole lot of cash. The best capacity/cost ratio is for 5Gb SATA drives
\n \nSome Dell PERCs can be re-flashed with LSI firmware and used as a normal HBA. Unfortunately the PERC6 isn’t one of them. I believe the PERC6/R can be, but those I’ve seen in a FS12 are just a bit too old. So the first thing you’ll need to do is dump them in the recycling or try and sell them on eBay.
\n \nThere are actually two PERC6 cards in most machine, and they each support eight SAS channels through two SFF-8484 connectors on each card. Given there are twelve drives slots, one of the PERCs is only half used. Sometimes they have a cable going off to a battery located near the fans. This is used in a desperate attempt to keep the data in the card’s cache safe in order to avoid write holes corrupting NTFS during a power failure, although the data on the on-drive caches won’t be so lucky. If you’re using a file system like that, make sure you have a UPS for the whole lot.
\n \nBut we’re going to put the PERCs out of our misery and replace them with some nice new LSI HBAs that will do our operating system’s bidding and let it talk to the drives as it knows best. But which to pick? First we need to know what we’re connecting.
\n \nMoving to the front of the case there are twelve metal drive slots with a backplane behind. Dell makes machines with either backplanes or expanders. A backplane has a 1:1 SAS channel to drive connection; an expander takes one SAS channel and multiplexes it to (usually) four drives. You could always swap the blackplane with an expander, but I like the 1:1 nature of a backplane. It’s faster, especially if you’re configured as an array. And besides, we don’t want to spend more money than we need to, otherwise we wouldn’t be hot-rodding a cheap 2U server in the first place – expanders are expensive. Bizarrely, HBAs are cheap in comparison. So we need twelve channels of SAS that will connect to the sockets on the backplane.
\n \nThe HBA you will probably want to go with is an LSI, as these have great OS support. Other cards are available, but check that the drivers are also available. The obvious choice for SAS aficionados is the LSI 9211-8i, which has eight internal channels. This is based on an LSI 2000 series chip, the 2008, which is the de-facto standard. There’s also four-channel -4i version, so you could get your twelve channels using one of each – but the price difference is small these days, so you might as well go for two -8i cards. If you want cheaper there are 1068-based equivalent cards, and these work just fine at about half the price. They probably won’t work with larger disks, only operate at 3Gb and the original SAS standard. However, the 2000 series is only about £25 extra and gives you more options for the future. A good investment. Conversely, the latest 3000 series cards can do some extra stuff (particularly to do with active cables) but I can’t see any great advantage in paying megabucks for one unless you’re going really high-end – in which case the NV12 isn’t the box for you anyway. And you’d need some very fast drives and a faster backplane to see any speed advantage. And probably a new motherboard….
\n \nWhether the 6Gb SAS2 of the 9211-8i is any use on the backplane, which was designed for 3Gb, I don’t know. If it matters that much to you you probably need to spend a lot more money. A drive array with a direct 3Gb to each drive is going to shift fast enough for most purposes.
\n \nOnce you have removed the PERCs and plugged in your modern-ish 9211 HBAs, your next problem is going to be the cable. Both the PERCs and the backplane have SFF-8484 multi-lane connectors, which you might not recognise. SAS is a point-to-point system, the same as SATA, and a multi-lane cable is simply four single cables in a bundle with one plug. (Newer versions of SAS have more). SFF-8484 multi-lane connectors are somewhat rare, (but unfortunately this doesn’t make them valuable if you were hoping to flog them on eBay). The world switched quickly to the SFF-8087 for multi-lane SAS. The signals are electrically the same, but the connector is not.
\n \nPlease generate and paste your ad code here. If left empty, the ad location will be highlighted on your blog pages with a reminder to enter your code. Mid-Post\n So there are two snags with this backplane. Firstly it’s designed to work with PERC controllers; secondly it has the old SFF-8484 connectors on the back, and any SAS cables you find are likely to have SFF-8087.
\n \nFirst things first – there is actually a jumper on the backplane to tell it whether it’s talking to a PERC or a standard LSI HBA. All you need to do is find it and change it. Fortunately there are very few jumpers to choose from (i.e. two), and you know the link is already in the wrong place. So try them one at a time until it works. The one you want may be labelled J15, but I wouldn’t like to say this was the same on every variant.
\n \nSecond problem: the cable. You can get cables with an SFF-8087 on one end and an SFF-8484 on the other. These should work. But they’re usually rather expensive. If you want to make your own, it’s a PITA but at least you have the connectors already (assuming you didn’t bin the ones on the PERC cables).
\n \nI don’t know what committee designed SAS cable connectors, but ease of construction wasn’t foremost in their collective minds. You’re basically soldering twisted pair to a tiny PCB. This is mechanically rubbish, of course, as the slightest force on the cable will lift the track. Therefore its usual to cover the whole joint in solidified gunk (technical term) to protect it. Rewiring SAS connectors is definitely not easy.
\n \nI’ve tried various ways of soldering to them, none of which were satisfactory or rewarding. One method is to clip the all bare wires you wish to solder using something like a bulldog clip so they’re at lined up horizontally and then press then adjust the clamp so they’re gently pressed to the tracks on the board, making final adjustments with a strong magnifying glass and a fine tweezers. You can then either solder them with a fine temperature-controlled iron, or have pre-coated the pads with solder paste and flash across it with an SMD rework station. I’d love to know how they’re actually manufactured – using a precision jig I assume.
\n \nThe “easy” way is to avoid soldering the connectors at all; simply cut existing cables in half and join one to the other. I’ve used prototyping matrix board for this. Strip and twist the conductors, push them through a hole and solder. This keeps things compact but manageable. We’re dealing with twisted pair here, so maintain the twists as close as possible to the board – it actually works quite well.
\n \nHowever, I’ve now found a reasonably-priced source of the appropriate cable so I don’t do this any more. Contact me if you need some in the UK.
\n \nSo all that remains is to plug your HBAs to the backplane, shove in some drives and you’re away. If you’re at this stage, it “just works”. The access lights for all the drives do their thing as they should. The only mystery is how you can get the ident LED to come on; this may be controlled by the PERC when it detects a failure using the so-called sideband channel, or it may be operated by the electronics on the backplane. It’s workings are, I’m afraid, something of a mystery still – it’s got too much electronics on board to be a completely passive backplane.
\n \nPlan B: SATA
\n \nIf you plan to use only SATA drives, especially if you don’t intend using more than six, it makes little sense to bother with SAS at all. The Gigabyte motherboard comes with half a dozen perfectly good 3Gb SATA channels, and if you need more you can always put another controller in a PCIe slot, or even USB. The advantages are lower cost and you get to free up two PCIe slots for more interesting things.
\n \nThe down-side is that you can’t use the SAS backplane, but you can still use the mounting bays.
\n \nRemoving the backplane looks tricky, but it really isn’t when you look a bit closer. Take out the fans first (held in place by rubber blocks), undo a couple of screws and it just lifts and slides out. You can then slot and lock in the drives and connect the SATA connectors directly to the back of the drives. You could even slide them out again without opening the case, as long as the cable was long enough and you manually detached the cable it when it was withdrawn. And let’s face it – drives are likely to last for years so even with half a dozen it’s not that great a hardship to open the case occasionally.
\n \nNext comes power. The PSU has a special connector for the backplane and two standard SATA power plugs. You could split these three ways using an adapter, but if you have a lot of drives you might want to re-wire the cables going to the backplane plug. It can definitely power twelve drives.
\n \nAnd that’s almost all there is to it. Unfortunately the main fans are connected to the backplane, which you’ve just removed. You can power them from an adapter on the drive power cables, but there are unused fan connectors on the motherboard. I’m doing a bit more research on cooling options, but this approach has promising possibilities for noise reduction.
\n
Tarsnap
\n\nOpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available.
\n\nMP3 Feed | iTunes Feed | HD Vid Feed | HD Torrent Feed
\n\n\n\n\nWhilst setting up one of my development laptops to port some software to Windows I noticed Windows 10 doing crazy things like installing or updating apps and games by default after initial setup. The one I noticed in particular was Candy Crush Soda Saga which for those who don't know of it is some cheesy little puzzle game originally for consumer devices. I honestly did not want software like this near to a development machine. It has also been reported that Windows 10 now also updates core system software without notifying the user. Surely this destroys any vaguely deterministic behaviour, in my opinion making Windows 10 by default almost useless for development testbeds.
\n \nDeciding instead to start from scratch but this time to set the inbuilt Windows Firewall to be very restrictive and only allow a few select programs to communicate. In this case all I really needed to be online was Firefox, Subversion and Putty. To my amusement (and astonishment) I found out that the Windows firewall could be modified to give access very easily by programs during installation (usually because this task needs to be done with admin privileges). It also seems that Windows store Apps can change the windows firewall settings at any point. One way to get around this issue could be to install a 3rd party firewall that most software will not have knowledge about and thus not attempt to break through. However the only decent firewall I have used was Sygate Pro which unfortunately is no longer supported by recent operating systems. The last supported versions was 2003, XP and 2000. In short, I avoid 3rd party firewalls.
\n \nInstead I decided to trap Windows 10 (and all of it's rogue updaters) behind a virtual machine running OpenBSD. This effectively provided me with a full blown firewall appliance. From here I could then allow specific software I trusted through the firewall (via a proxy) in a safe, controlled and deterministic manner. For other interested developers (and security conscious users) and for my own reference, I have listed the steps taken here:
\n
1) First and foremost disable the Windows DHCP service - this is so no IP can be obtained on any interface. This effectively stops any communication with any network on the host system. This can be done by running services.msc with admin privileges and stopping and disabling the service called DHCP Client.
2) Install or enable your favorite virtualization software - I have tested this with both VirtualBox and Hyper-V. Note that on non-server versions of Windows, in order to get Hyper-V working, your processor also needs to support SLAT which is daft so to avoid faffing about, I recommend using VirtualBox to get round this seemingly arbitrary restriction.
3) Install OpenBSD on the VM - Note, if you decide to use Hyper-V, its hardware support isn't 100% perfect to run OpenBSD and you will need to disable a couple of things in the kernel. At the initial boot prompt, run the following commands.
\nconfig -e -o /bsd /bsd\ndisable acpi\ndisable mpbios\n
4) Add a host only virtual adapter to the VM - This is the one which we are going to connect through the VM with. Look at the IP that VirtualBox assigns this in network manager on the host machine. Mine was [b]192.168.56.1[/b]. Set up the adapter in the OpenBSD VM to have a static address on the same subnet. For example [b]192.168.56.2[/b]. If you are using Hyper-V and OpenBSD, make sure you add a \"Legacy Interface\" because no guest additions are available. Then set up a virtual switch which is host only.
5) Add a bridged adapter to the VM - then assign it to whichever interface you wanted to connect to the external network with. Note that if using Wireless, set the bridged adapters MAC address to the same as your physical device or the access point will reject it. This is not needed (or possible) on Hyper-V because the actual device is \"shared\" rather than bridged so the same MAC address is used. Again, if you use Hyper-V, then add another virtual switch and attach it to your chosen external interface. VMs in Hyper-V \"share\" an adapter within a virtual switch and there is the option to also disable the hosts ability to use this interface at the same time which is fine for an additional level of security if those pesky rogue apps and updaters can also enable / disable DHCP service one day which wouldn't be too surprising.
6) Connect to your network in the host OS - In case of Wireless, select the correct network from the list and type in a password if needed. Windows will probably say \"no internet available\", it also does not assign an IP address which is fine.
7) Install the Squid proxy package on the OpenBSD guest and enable the daemon
```
\n\n```
\n\n\n\n\nWe will use this service for a limited selection of \"safe and trusted\" programs to connect to the outside world from within the Windows 10 host. You can also use putty on the host to connect to the VM via SSH and create a SOCKS proxy which software like Firefox can also use to connect externally.
\n
8) Configure the software you want to be able to access the external network with
\n\n\n--proxy-server=\"socks5://<VM IP>:<SOCKS PORT>\"\n--host-resolver-rules=\"MAP * 0.0.0.0 , EXCLUDE <VM IP>\"\n
\n\n\nI've managed to unbreak the LLDB debugger as much as possible with the current kernel and hit problems with ptrace(2) that are causing issues with further work on proper NetBSD support. Meanwhile, I've upstreamed all the planned NetBSD patches to sanitizers and helped other BSDs to gain better or initial support.
\n
\n\n\nSince the last time I worked on LLDB, we have introduced many changes to the kernel interfaces (most notably related to signals) that apparently fixed some bugs in Go and introduced regressions in ptrace(2). Part of the regressions were noted by the existing ATF tests. However, the breakage was only marked as a new problem to resolve. For completeness, the ptrace(2) code was also cleaned up by Christos Zoulas, and we fixed some bugs with compat32.
\n \nI've fixed a crash in *NetBSD::Factory::Launch(), triggered on startup of the lldb-server application.
\n \nHere is the commit message:
\n
```\nWe cannot call process_up->SetState() inside\nthe NativeProcessNetBSD::Factory::Launch\nfunction because it triggers a NULL pointer\ndeference.
\n\nThe generic code for launching a process in:\nGDBRemoteCommunicationServerLLGS::LaunchProcess\nsets the mdebuggedprocessup pointer after\na successful call to mprocessfactory.Launch().\nIf we attempt to call processup->SetState()\ninside a platform specific Launch function we\nend up dereferencing a NULL pointer in\nNativeProcessProtocol::GetCurrentThreadID().
\n\nUse the proper call processup->SetState(,false)\nthat sets notifydelegates to false.\n```
\n\n\n\n\nI suspended development of new features in sanitizers last month, but I was still in the process of upstreaming of local patches. This process was time-consuming as it required rebasing patches, adding dedicated tests, and addressing all other requests and comments from the upstream developers.
\n \nI'm not counting hot fixes, as some changes were triggering build or test issues on !NetBSD hosts. Thankfully all these issues were addressed quickly. The final result is a reduction of local delta size of almost 1MB to less than 100KB (1205 lines of diff). The remaining patches are rescheduled for later, mostly because they depend on extra work with cross-OS tests and prior integration of sanitizers with the basesystem distribution. I didn't want to put extra work here in the current state of affairs and, I've registered as a mentor for Google Summer of Code for the NetBSD Foundation and prepared Software Quality improvement tasks in order to outsource part of the labour.
\n
\n\n\nI've also improved documentation for some of the features of NetBSD, described in man-pages. These pieces of information were sometimes wrong or incomplete, and this makes covering the NetBSD system with features such as sanitizers harder as there is a mismatch between the actual code and the documented code.
\n \nSome pieces of software also require better namespacing support, these days mostly for the POSIX standard. I've fixed few low-hanging fruits there and requested pullups to NetBSD-8(BETA).
\n \nI thank the developers for improving the landed code in order to ship the best solutions for users.
\n
\n\n\nA One-man-show in human activity is usually less fun and productive than collaboration in a team. This is also true in software development. Last month I was helping as a reviewer to port LLVM features to FreeBSD and when possible to OpenBSD. This included MSan/FreeBSD, libFuzzer/FreeBSD, XRay/FreeBSD and UBSan/OpenBSD.
\n \nI've landed most of the submitted and reviewed code to the mainstream LLVM tree.
\n \nPart of the code also verified the correctness of NetBSD routes in the existing porting efforts and showed new options for improvement. This is the reason why I've landed preliminary XRay/NetBSD code and added missing NetBSD bits to ToolChain::getOSLibName(). The latter produced setup issues with the prebuilt LLVM toolchain, as the directory name with compiler-rt goodies were located in a path like ./lib/clang/7.0.0/lib/netbsd8.99.12 with a varying OS version. This could stop working after upgrades, so I've simplified it to \"netbsd\", similar to FreeBSD and Solaris.
\n
\n\n\nI've prepared a build of Clang/LLVM with LLDB and compiler-rt features prebuilt on NetBSD/amd64 v. 8.99.12:
\n
llvm-clang-compilerrt-lldb-7.0.0beta_2018-02-28.tar.bz2
\n\n\nWith the approaching NetBSD 8.0 release I plan to finish backporting a few changes there from HEAD:
\n
\n\n\nOnce done, I will return to ptrace(2) debugging and corrections.
\n
DigitalOcean
\n\n\n\n\nWhen working on complex systems, such as OS kernels, your attention span and cognitive energy are too valuable to be wasted on inefficiencies pertaining to ancillary tasks. After experimenting with different environmental setups for kernel debugging, some of which were awkward and distracting from my main objectives, I have arrived to my current workflow, which is described here. This approach is mainly oriented towards security research and the study of kernel internals.
\n \nBefore delving into the details, this is the general outline of my environment:
\n \nMy host system runs Linux. My target system is a QEMU guest.
\n \nI’m tracing and debugging on my host system by attaching GDB (with NetBSD x86-64 ABI support) to QEMU’s built-in GDB server.\n I work with NetBSD-current. All sources are built on my host system with the cross-compilation toolchain produced by build.sh.\n I use NFS to share the source tree and the build artifacts between the target and the host.\n I find IDEs awkward, so for codebase navigation I mainly rely on vim, tmux and ctags.\n For non-intrusive instrumentation, such as figuring out control flow, I’m using dtrace.
\n
Preparing the host system
\n\nBuilding NetBSD-current
A word of warning
\n\n\n -r Remove contents of TOOLDIR and DESTDIR before building.\n -u Set MKUPDATE=yes; do not run \"make clean\" first.\n Without this, everything is rebuilt, including the tools.\n
\n\n\nChance are, you do not want to use these options once you’ve successfully built the cross-compilation toolchain and your entire userland, because building those takes time and there aren’t many good reasons to recompile them from scratch. Here’s what to expect:
\n \nOn my desktop, running a quad-core Intel i5-3470 at 3.20GHz with 24GB of RAM and underlying directory structure residing on a SSD drive, the entire process took about 55 minutes. I was running make with -j12, so the machine was quite busy.\n On an old Dell D630 laptop, running Intel Core 2 Duo T7500 at 2.20GHz with 4GB of RAM and a slow hard drive (5400RPM), the process took approximatelly 2.5 hours. I was running make with -j4. Based on the temperature alerts and CPU clock throttling messages, it was quite a struggle.
\n
Compiling the sources
\n\n```\nAdd support for the experimental Internet-Draft \"TCP Alternative Backoff with\nECN (ABE)\" proposal to the New Reno congestion control algorithm module.\nABE reduces the amount of congestion window reduction in response to\nECN-signalled congestion relative to the loss-inferred congestion response.
\n\nMore details about ABE can be found in the Internet-Draft:\nhttps://tools.ietf.org/html/draft-ietf-tcpm-alternativebackoff-ecn
\n\nThe implementation introduces four new sysctls:
\n\nnet.inet.tcp.cc.abe defaults to 0 (disabled) and can be set to non-zero to\nenable ABE for ECN-enabled TCP connections.
net.inet.tcp.cc.newreno.beta and net.inet.tcp.cc.newreno.betaecn set the\nmultiplicative window decrease factor, specified as a percentage, applied to\nthe congestion window in response to a loss-based or ECN-based congestion\nsignal respectively. They default to the values specified in the draft i.e.\nbeta=50 and betaecn=80.
net.inet.tcp.cc.abe_frlossreduce defaults to 0 (disabled) and can be set to\nnon-zero to enable the use of standard beta (50% by default) when repairing\nloss during an ECN-signalled congestion recovery episode. It enables a more\nconservative congestion response and is provided for the purposes of\nexperimentation as a result of some discussion at IETF 100 in Singapore.
The values of beta and betaecn can also be set per-connection by way of the\nTCPCCALGOOPT TCP-level socket option and the new CCNEWRENOBETA or\nCCNEWRENOBETA_ECN CC algo sub-options.
\n\nSubmitted by: Tom Jones tj@enoti.me\nTested by: Tom Jones tj@enoti.me, Grenville Armitage garmitage@swin.edu.au\nRelnotes: Yes\nDifferential Revision: https://reviews.freebsd.org/D11616\n```
\n\n\n\n\nThe recent changes in -current mitigating the Meltdown vulnerability have been backported to the 6.1 and 6.2 (amd64) releases, and the syspatch update (for 6.2) is now available.
\n
```\nChanges by: bluhm@cvs.openbsd.org 2018/02/26 05:36:18\nLog message:\nImplement a workaround against the Meltdown flaw in Intel CPUs.\nThe following changes have been backported from OpenBSD -current.
\n\nChanges by: guenther@cvs.openbsd.org 2018/01/06 15:03:13\nLog message:\nHandle %gs like %[def]s and reset set it in cpu_switchto() instead of on\nevery return to userspace.
\n\nChanges by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20\nLog message:\nAdd identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs\nthat should help mitigate spectre. This is just the detection piece, these\nfeatures are not yet used.\nPart of a larger ongoing effort to mitigate meltdown/spectre. i386 will\ncome later; it needs some machdep.c cleanup first.
\n\nChanges by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19\nLog message:\nremove all PG_G global page mappings from the kernel when running on\nIntel CPUs. Part of an ongoing set of commits to mitigate the Intel\n\"meltdown\" CVE. This diff does not confer any immunity to that\nvulnerability - subsequent commits are still needed and are being\nworked on presently.\nok guenther, deraadt
\n\nChanges by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30\nLog message:\nIBRS -> IBRS,IBPB in identifycpu lines
\n\nChanges by: guenther@cvs.openbsd.org 2018/02/21 12:24:15\nLog message:\nMeltdown: implement user/kernel page table separation.\nOn Intel CPUs which speculate past user/supervisor page permission checks,\nuse a separate page table for userspace with only the minimum of kernel code\nand data required for the transitions to/from the kernel (still marked as\nsupervisor-only, of course):\n- the IDT (RO)\n- three pages of kernel text in the .kutext section for interrupt, trap,\nand syscall trampoline code (RX)\n- one page of kernel data in the .kudata section for TLB flush IPIs (RW)\n- the lapic page (RW, uncachable)\n- per CPU: one page for the TSS+GDT (RO) and one page for trampoline\nstacks (RW)\nWhen a syscall, trap, or interrupt takes a CPU from userspace to kernel the\ntrampoline code switches page tables, switches stacks to the thread's real\nkernel stack, then copies over the necessary bits from the trampoline stack.\nOn return to userspace the opposite occurs: recreate the iretq frame on the\ntrampoline stack, switch stack, switch page tables, and return to userspace.\nmlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing\nissues on MP in particular, and drove the final push to completion.\nMany rounds of testing by naddy@, sthen@, and others\nThanks to Alex Wilson from Joyent for early discussions about trampolines\nand their data requirements.\nPer-CPU page layout mostly inspired by DragonFlyBSD.\nok mlarkin@ deraadt@
\n\nChanges by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59\nLog message:\nThe GNU assembler does not understand 1ULL, so replace the constant\nwith 1. Then it compiles with gcc, sign and size do not matter\nhere.
\n\nChanges by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14\nLog message:\nThe compile time assertion for cpu info did not work with gcc.\nRephrase the condition in a way that both gcc and clang accept it.
\n\nChanges by: guenther@cvs.openbsd.org 2018/02/22 13:36:40\nLog message:\nSet the PG_G (global) bit on the special page table entries that are shared\nbetween the u-k and u+k tables, because they're actually in all tables.
\n\nOpenBSD 6.1 errata 037\n```
\n\n```\nChanges by: bluhm@cvs.openbsd.org 2018/02/26 05:29:48\nLog message:\nImplement a workaround against the Meltdown flaw in Intel CPUs.\nThe following changes have been backported from OpenBSD -current.
\n\nChanges by: guenther@cvs.openbsd.org 2018/01/06 15:03:13\nLog message:\nHandle %gs like %[def]s and reset set it in cpu_switchto() instead of on\nevery return to userspace.
\n\nChanges by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20\nLog message:\nAdd identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs\nthat should help mitigate spectre. This is just the detection piece, these\nfeatures are not yet used.\nPart of a larger ongoing effort to mitigate meltdown/spectre. i386 will\ncome later; it needs some machdep.c cleanup first.
\n\nChanges by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19\nLog message:\nremove all PG_G global page mappings from the kernel when running on\nIntel CPUs. Part of an ongoing set of commits to mitigate the Intel\n\"meltdown\" CVE. This diff does not confer any immunity to that\nvulnerability - subsequent commits are still needed and are being\nworked on presently.
\n\nChanges by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30\nLog message:\nIBRS -> IBRS,IBPB in identifycpu lines
\n\nChanges by: guenther@cvs.openbsd.org 2018/02/21 12:24:15\nLog message:\nMeltdown: implement user/kernel page table separation.\nOn Intel CPUs which speculate past user/supervisor page permission checks,\nuse a separate page table for userspace with only the minimum of kernel code\nand data required for the transitions to/from the kernel (still marked as\nsupervisor-only, of course):\n- the IDT (RO)\n- three pages of kernel text in the .kutext section for interrupt, trap,\nand syscall trampoline code (RX)\n- one page of kernel data in the .kudata section for TLB flush IPIs (RW)\n- the lapic page (RW, uncachable)\n- per CPU: one page for the TSS+GDT (RO) and one page for trampoline\nstacks (RW)\nWhen a syscall, trap, or interrupt takes a CPU from userspace to kernel the\ntrampoline code switches page tables, switches stacks to the thread's real\nkernel stack, then copies over the necessary bits from the trampoline stack.\nOn return to userspace the opposite occurs: recreate the iretq frame on the\ntrampoline stack, switch stack, switch page tables, and return to userspace.\nmlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing\nissues on MP in particular, and drove the final push to completion.\nMany rounds of testing by naddy@, sthen@, and others\nThanks to Alex Wilson from Joyent for early discussions about trampolines\nand their data requirements.\nPer-CPU page layout mostly inspired by DragonFlyBSD.
\n\nChanges by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59\nLog message:\nThe GNU assembler does not understand 1ULL, so replace the constant\nwith 1. Then it compiles with gcc, sign and size do not matter\nhere.
\n\nChanges by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14\nLog message:\nThe compile time assertion for cpu info did not work with gcc.\nRephrase the condition in a way that both gcc and clang accept it.
\n\nChanges by: guenther@cvs.openbsd.org 2018/02/22 13:36:40\nLog message:\nSet the PG_G (global) bit on the special page table entries that are shared\nbetween the u-k and u+k tables, because they're actually in all tables.
\n\nOpenBSD 6.2 errata 009\n```
\n\niXsystems
\n\n\n\n\nKen Westerback (krw@) has sent in the first report from the (recently concluded) a2k18 hackathon:
\n
YYZ -> YVR -> MEL -> ZQN -> CHC -> DUD -> WLG -> AKL -> SYD -> BNE -> YVR -> YYZ
For those of you who don’t speak Airport code:
```
\n\nWhew.
\n\nOnce in Dunedin the hacking commenced. The background was a regular tick of new meltdown diffs to test in addition to whatever work one was actually engaged in. I was lucky (?) in that none of the problems with the various versions cropped up on my laptop.\n```
\n\n```\nI worked with rpe@ and tb@ to make the install script create the 'correct' FQDN when dhclient was involved. I worked with tb@ on some code cleanup in various bits of the base. dhclient(8) got some nice cleanup, further pruning/improving log messages in particular. In addition the oddball -q option was flipped into the more normal -v. I.e. be quiet by default and verbose on request.
\n\nMore substantially the use of recorded leases was made less intrusive by avoiding continual reconfiguration of the interface with the same information. The 'request', 'require' and 'ignore' dhclient.conf(5) statement were changed so they are cumulative, making it easier to build longer lists of affected options.
\n\nI tweaked softraid(4) to remove a handrolled version of duid_format().
\n\nI sprinkled a couple of M_WAITOK into amd64 and i386 mpbios to document that there is really no need to check for NULL being returned from some malloc() calls.
\n\nI continued to help test the new filesystem quiescing logic that deraadt@ committed during the hackathon.
\n\nI only locked myself out of my room once!
\n\nFueled by the excellent coffee from local institutions The Good Earth Cafe and The Good Oil Cafe, and the excellent hacking facilities and accommodations at the University of Otago it was another enjoyable and productive hackathon south of the equator. And I even saw penguins.
\n\nThanks to Jim Cheetham and the support from the project and the OpenBSD Foundation that made it all possible\n```
\n\n\n\n\nI found this when going through old documents. It looks like I wrote it and never posted it. Perhaps I didn’t consider it finished at the time. But looking at it now, I think it’s good enough to share. It’s a redrafting of the BSD licence, in poetic form. Maybe I had plans to do other licences one day; I can’t remember.
\n \nI’ve interleaved it with the original license text so you can see how true, or otherwise, I’ve been to it. Enjoy :-)
\n
```\nCopyright (c)
Redistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions\nare met:\n```
\n\n\n\n\nYou may redistribute and use –\n as source or binary, as you choose,\n and with some changes or without –\n this software; let there be no doubt.\n But you must meet conditions three,\n if in compliance you wish to be.
\n
\n1. Redistributions of source code must retain the above copyright\n notice, this list of conditions and the following disclaimer.\n2. Redistributions in binary form must reproduce the above copyright\n notice, this list of conditions and the following disclaimer in the\n documentation and/or other materials provided with the distribution.\n3. Neither the name of the nor the names of its\n contributors may be used to endorse or promote products derived\n from this software without specific prior written permission.\n
\n\n\nThe first is obvious, of course –\n To keep this text within the source.\n The second is for binaries\n Place in the docs a copy, please.\n A moral lesson from this ode –\n Don’t strip the copyright on code.
\n \nThe third applies when you promote:\n You must not take, from us who wrote,\n our names and make it seem as true\n we like or love your version too.\n (Unless, of course, you contact us\n And get our written assensus.)
\n
\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n\"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS\nFOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE\nCOPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,\nINCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\nBUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\nLOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\nCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\nLIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN\nANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\nPOSSIBILITY OF SUCH DAMAGE.\n
\n\n\nOne final point to be laid out\n (You must forgive my need to shout):\n THERE IS NO WARRANTY FOR THIS\n WHATEVER THING MAY GO AMISS.\n EXPRESS, IMPLIED, IT’S ALL THE SAME –\n RESPONSIBILITY DISCLAIMED.
\n \nWE ARE NOT LIABLE FOR LOSS\n NO MATTER HOW INCURRED THE COST\n THE TYPE OR STYLE OF DAMAGE DONE\n WHATE’ER THE LEGAL THEORY SPUN.\n THIS STILL REMAINS AS TRUE IF YOU\n INFORM US WHAT YOU PLAN TO DO.
\n \nWhen all is told, we sum up thus –\n Do what you like, just don’t sue us.
\n
Tarsnap ad
\n\nWe kick off the first episode with the latest BSD news, show you how to avoid intrusion detection systems and talk to Peter Hessler about BGP spam blacklists!
\n\nUsing BGP to distribute spam blacklists and whitelists
\n\n