ZetaWatch Authorization · 2019-10-13 00:16 by Black in ZetaWatch
The ZetaWatch helper tool uses the Security framework to authorize users before performing privileged operations. It currently supports the following permissions.
net.the-color-black.ZetaWatch.import- allowed by default, required for importing a pool.
net.the-color-black.ZetaWatch.export- allowed by default, required for exporting a pool.
net.the-color-black.ZetaWatch.mount- allowed by default, required for mounting a dataset.
net.the-color-black.ZetaWatch.unmount- allowed by default, required for unmounting a dataset.
net.the-color-black.ZetaWatch.snapshot- allowed by default, required for creating a snapshot.
net.the-color-black.ZetaWatch.rollback- requires admin authentication by default, required for rolling back a filesystem.
net.the-color-black.ZetaWatch.clone- requires admin authentication by default, required for cloning a filesystem.
net.the-color-black.ZetaWatch.create- requires admin authentication by default, required for creating a new filesystem.
net.the-color-black.ZetaWatch.destroy- requires admin authentication by default, required for destroying a filesystem or snapshot.
net.the-color-black.ZetaWatch.key- allowed by default, required for loading or unloading a key for a dataset. This also includes the ability to auto mount / unmount them.
net.the-color-black.ZetaWatch.scrub- allowed by default, required for starting, stopping or pausing scrubs.
These permissions can be manipulated via the security command line program. To inspect the current dataset creation permissions, and switching it to allow this to all users:
security authorizationdb read net.the-color-black.ZetaWatch.create security authorizationdb write net.the-color-black.ZetaWatch.create allow
Permissions include allow, deny or authenticate-admin.
More detailed information about this topic can be found in the article apples documentation about AuthorizationServices and Managing the Authorization Database in OS X Mavericks
Using Sparkle Framework · 2019-10-12 00:53 by Black in ZetaWatch
Sparkle is a very nice and popular framework for self-updating applications on Mac OS X. But using it in ZetaWatch turned out to be slightly problematic.
Code Signing / Notarization
Mac OS X is increasingly cautious with allowing code to run. Notarization is required to create binaries that can be run without without warning on the newest Mac OS X. Notarization is mostly supported by Skarkle itself, but apple’s validation doesn’t accept the original Sparkle.framework due to the unsigned helper binaries it contains.
The easiest way to code sign it seemed to simply build it myself in Xcode, and let it do all the work.
ZFS Binary Compatibility
Since ZetaWatch directly links to the zfs libraries, it only works if those are compatible. And while Sparkle has built-in support for OS compatibility checking, it doesn’t have the same for other dependencies. There is support for custom appcast filtering, to select a suitable version, but since the ZFS version and the ZetaWatch version are kind of orthogonal, this didn’t seem fitting.
The chosen solution was to have a ZFS version specific appcast URL, and make ZetaWatch query the appropriate appcast. This allows updating ZetaWatch when the used ZFS version changes, but also have several supported parallel builds. Currently, the only supported ZFS version is 1.9, so this is not an issue yet.
Localization
Sparkle supports dozens of languages, and all of them take up space. Combined, Sparkle.framework alone is already 3.5 MB. Since ZetaWatch itself is a very small program, and is currently not localized in any language other than English, this seems very wasteful. I decided therefore to strip it down, to English only and 1.4 MB.
ZetaWatch / ZFS Snapshoting · 2019-10-12 00:43 by Black in ZetaWatch
ZetaWatch recently received the capability for snapshot management. This includes displaying snapshots, but their creation and destruction. Snapshots can also be cloned, rolled back to, and mounted. This article describes how ZetaWatch can interact with snapshots, the shortcomings of it, and how zfs provides this capability. And what is planed to be improved.
Problems
Mounting snapshots is different from normal datasets, their mount point is fixed into the hidden .zfs/snapshot directory at the root of every dataset. (It can be made visible by setting zfs set snapdir=visible pool/dataset). On OS X, this seems to make finder, and all other Cocoa applications, unable to see it. Posix applications such as shells can use it though. It is therefore of rather limited usefulness.
Rolling back to a snapshot is problematic, because it applies to the whole dataset. It is not something that is useful in a lot of situation, since dataset granularity is not very fine grained. Rolling back to a snapshot will destroy all snapshots that are newer than it, and also all dependent clones. This is why ZetaWatch requires authentication before performing a rollback.
Cloning is more generally useful. It creates a new dependent file system that can be used like the original, including modifications. But it still depends on the original snapshot, and the original dataset. In ZetaWatch, this also requires authentication, since it allows creation of a new dataset.
Destroying snapshots is easy, but will fail if a dependent clone exists. Recursive deletion or readable errors are not yet provided.
Snapshot related API
The ZFS API for snapshot interaction is a bit inconsistent, combining handles, names and even nvlists in places, and sometimes offering higher level convenience, and in other places lacking. This isn’t too surprising though, the ZFS libraries seem intended only for internal use by the command line tools.
zfs_snapshot- takes a
libzfs_handle_t*handle for the library itself, the desired name of the snapshot (including pool and dataset name, a flag for recursive and annvlist_t*for additional properties for the created snapshots. The snapshot name is validated, and split into a dataset and snapshot portion. A handle to the dataset is created from the name, and used to iterate over all child filesystems recursively, if requested. The names of all to-be-created snapshots are added to an nvlist, andzfs_snapshot_nvlis invoked. It in turn validates the properties, and callslzc_snapshotto actually create the snapshots.
zfs_clone- is similar, but it takes an
zfs_handle_t*to the snapshot to be cloned, the desired full name and an nvlist with properties. It validates the operation, verifying among other things that both the source snapshot (passed via handle) and the destination dataset (passes as part of the path) are valid and exist, then passes control on tolzc_clone. Which again accepts strings instead of handles for everything.
zfs_rollback- takes a
zfs_handle_t*for the filesystem, azfs_handle_t*for the snapshot and a force flag as argument. This is strange, since snapshots only apply to one filesystem. Internally, this function first destroys all bookmarks and snapshots that are newer, and then callslzc_rollback_to, passing it the names of the base dataset and the snapshot.
zfs_destroy- is much lower level than the previous functions. It can destroy datasets and snapshots, which are passed in via a handle. But it doesn’t allow any recursion, unlike
zfs_rollback, or even unmount the dataset. And unlikezfs_clone, it doesn’t verify if the operation makes sense either. And it directly interacts with zfs_ioctl. If it wasn’t for taking a handle as parameter, it’d feel right at home with the lzc family of functions.
Planed Improvements to ZetaWatch
- Capability for Clone Promotion
- Ask for confirmation before destroying snapshots / clones on dataset rollback
- Recursive destruction with confirmation.
- Better error message on non-recursive destruction.
ZetaWatch for Developers · 2019-05-05 17:37 by Black in ZetaWatch
ZFS Interaction
ZetaWatch communicates with zfs using libzfs.dylib, libzfs_core.dylib, libzpool.dylib and libnvpair.dylib, just like the command line tools do. This gives it all the flexibility of the command line tools, at the cost of having to reimplement functionality that is found in the tools and not the library. And since the libraries are
explicitly not meant to provide a stable ABI, ZetaWatch is also closely coupled to the ZFS version it is built and written for.
All the ZFS interaction is wrapped in the ZFSWrapper library. This C++ library isolates the issues mentioned above and provides a more convenient and safe API than the original C interface does. The library is used both by the helper tool and the frontend app. This is the most reusable part of ZetaWatch, and might be split out as separate project later.
- ZFSUtils
- contains most of the advanced functionality, such as C++ Wrappers around the library, pool, vdev or file system handles. Those classes also have functionality to query state and iterate over members.
- ZFSNVList
- provides a wrapper around the
nvpair_t/nvlist_tdata structure that is used in ZFS for a lot of userland / kernel communication. It manages resources in both owning and non-owning fashion, and allows for easier iteration over sequences.
- ZFSStrings
- translate ZFS status enums into the user facing emoji or string description, optionally with localization. (Localization is not well tested or supported at the moment.)
Helper Tool
The implementation of the helper tool follows apple’s EvenBetterAuthorizationSample
The helper tool communicates with the user application via AuthorizationService and NSXPCConnection. The application side of code for this is in ZetaAuthorization.m. The RPC protocol can be found in ZetaAuthorizationHelperProtocol.h, and is implemented in ZetaAuthorizationHelper.mm. The CommonAuthorization.m file contains the supported commands and associated default permissions.
The helper tool can be uninstalled with the `uninstall-helper.sh` script. This is useful for debugging the installation of the helper, or updating the helper without increasing the bundle version.
Security & Code Signing
Official release builds are signed and notarized, and should run without issues even on newer Mac OS X. But there are still issues with authentication reported with the program not being recognized as signed. To verify security manually, the following commands can be used:
codesign -v -v -d ZetaWatch.app
xcrun stapler validate -v ZetaWatch.appBuilding ZetaWatch requires an apple developer account with DeveloperID signing
capabilities, since it uses SMJobBless to run a helper service as root. This service
executes actions on behalf of the user, such as mounting, unmounting or loading a key.
Notarization is required to create binaries that can be run without without warning on
the newest Mac OS X.
ZetaWatch · 2019-05-05 01:54 by Black in ZetaWatch
ZetaWatch is a small OS X program that displays the zfs status in the menu bar, similar to what iStat Menus does for other information. It is fairly well tested, but due to the current state of libzfs and libzfs_core, changes will be required until the API
stabilizes. ZetaWatch is usually compiled for the latest available ZFS release for Mac OS, and might not be compatible with other releases.
Currently supported features are:
- Show pool and vdev status including scrub progress
- Show pool / filesystem properties
- Show filesystem and vdev stats
- Import pools and Export pools
- Mount / unmount datasets
- Load encryption keys for encrypted datasets
- Scrub pools
- Report errors in notification center
Check it out :)
CaptureOne for Sony · 2019-03-31 14:54 by Black in
Apple’s Photos isn’t too great, Adobe’s Lightroom requries a subscription, Sony’s Edit is quite lacking in feature. After some searching around, I settled for PhaseOne’s Capture One, and it’s free, for Sony (or Fujifilm) cameras.
Some features are only available in the pro version, which is available as pay-once or subscription, but the free version is already very feature rich, and includes asset management, raw editing and more. And it’s quite easy to use.
This picture was taken under dim sodium light, handheld, with ISO 6400, and edited slightly (and not very skillfully) in CaptureOne Express.
Here a 1:1 cutout of the original, converted to JPEG with Apple’s Preview.
One click to adjust white balance, a bit of playing with auto-exposure and the curves to change the light, and adjust noise reduction a bit.