Go to content Go to navigation

ZetaWatch Authorization · 2019-10-13 00:16 by Black in

The ZetaWatch helper tool uses the Security framework to authorize users before performing privileged operations. It currently supports the following permissions.

net.the-color-black.ZetaWatch.import
allowed by default, required for importing a pool.
net.the-color-black.ZetaWatch.export
allowed by default, required for exporting a pool.
net.the-color-black.ZetaWatch.mount
allowed by default, required for mounting a dataset.
net.the-color-black.ZetaWatch.unmount
allowed by default, required for unmounting a dataset.
net.the-color-black.ZetaWatch.snapshot
allowed by default, required for creating a snapshot.
net.the-color-black.ZetaWatch.rollback
requires admin authentication by default, required for rolling back a filesystem.
net.the-color-black.ZetaWatch.clone
requires admin authentication by default, required for cloning a filesystem.
net.the-color-black.ZetaWatch.create
requires admin authentication by default, required for creating a new filesystem.
net.the-color-black.ZetaWatch.destroy
requires admin authentication by default, required for destroying a filesystem or snapshot.
net.the-color-black.ZetaWatch.key
allowed by default, required for loading or unloading a key for a dataset. This also includes the ability to auto mount / unmount them.
net.the-color-black.ZetaWatch.scrub
allowed by default, required for starting, stopping or pausing scrubs.

These permissions can be manipulated via the security command line program. To inspect the current dataset creation permissions, and switching it to allow this to all users:

security authorizationdb read net.the-color-black.ZetaWatch.create
security authorizationdb write net.the-color-black.ZetaWatch.create allow

Permissions include allow, deny or authenticate-admin.

More detailed information about this topic can be found in the article apples documentation about AuthorizationServices and Managing the Authorization Database in OS X Mavericks

Comment

Using Sparkle Framework · 2019-10-12 00:53 by Black in

Sparkle is a very nice and popular framework for self-updating applications on Mac OS X. But using it in ZetaWatch turned out to be slightly problematic.

Code Signing / Notarization

Mac OS X is increasingly cautious with allowing code to run. Notarization is required to create binaries that can be run without without warning on the newest Mac OS X. Notarization is mostly supported by Skarkle itself, but apple’s validation doesn’t accept the original Sparkle.framework due to the unsigned helper binaries it contains.

The easiest way to code sign it seemed to simply build it myself in Xcode, and let it do all the work.

ZFS Binary Compatibility

Since ZetaWatch directly links to the zfs libraries, it only works if those are compatible. And while Sparkle has built-in support for OS compatibility checking, it doesn’t have the same for other dependencies. There is support for custom appcast filtering, to select a suitable version, but since the ZFS version and the ZetaWatch version are kind of orthogonal, this didn’t seem fitting.

The chosen solution was to have a ZFS version specific appcast URL, and make ZetaWatch query the appropriate appcast. This allows updating ZetaWatch when the used ZFS version changes, but also have several supported parallel builds. Currently, the only supported ZFS version is 1.9, so this is not an issue yet.

Localization

Sparkle supports dozens of languages, and all of them take up space. Combined, Sparkle.framework alone is already 3.5 MB. Since ZetaWatch itself is a very small program, and is currently not localized in any language other than English, this seems very wasteful. I decided therefore to strip it down, to English only and 1.4 MB.

Comment

ZetaWatch / ZFS Snapshoting · 2019-10-12 00:43 by Black in

ZetaWatch recently received the capability for snapshot management. This includes displaying snapshots, but their creation and destruction. Snapshots can also be cloned, rolled back to, and mounted. This article describes how ZetaWatch can interact with snapshots, the shortcomings of it, and how zfs provides this capability. And what is planed to be improved.

Problems

Mounting snapshots is different from normal datasets, their mount point is fixed into the hidden .zfs/snapshot directory at the root of every dataset. (It can be made visible by setting zfs set snapdir=visible pool/dataset). On OS X, this seems to make finder, and all other Cocoa applications, unable to see it. Posix applications such as shells can use it though. It is therefore of rather limited usefulness.

Rolling back to a snapshot is problematic, because it applies to the whole dataset. It is not something that is useful in a lot of situation, since dataset granularity is not very fine grained. Rolling back to a snapshot will destroy all snapshots that are newer than it, and also all dependent clones. This is why ZetaWatch requires authentication before performing a rollback.

Cloning is more generally useful. It creates a new dependent file system that can be used like the original, including modifications. But it still depends on the original snapshot, and the original dataset. In ZetaWatch, this also requires authentication, since it allows creation of a new dataset.

Destroying snapshots is easy, but will fail if a dependent clone exists. Recursive deletion or readable errors are not yet provided.

Snapshot related API

The ZFS API for snapshot interaction is a bit inconsistent, combining handles, names and even nvlists in places, and sometimes offering higher level convenience, and in other places lacking. This isn’t too surprising though, the ZFS libraries seem intended only for internal use by the command line tools.

zfs_snapshot
takes a libzfs_handle_t* handle for the library itself, the desired name of the snapshot (including pool and dataset name, a flag for recursive and an nvlist_t* for additional properties for the created snapshots. The snapshot name is validated, and split into a dataset and snapshot portion. A handle to the dataset is created from the name, and used to iterate over all child filesystems recursively, if requested. The names of all to-be-created snapshots are added to an nvlist, and zfs_snapshot_nvl is invoked. It in turn validates the properties, and calls lzc_snapshot to actually create the snapshots.
zfs_clone
is similar, but it takes an zfs_handle_t* to the snapshot to be cloned, the desired full name and an nvlist with properties. It validates the operation, verifying among other things that both the source snapshot (passed via handle) and the destination dataset (passes as part of the path) are valid and exist, then passes control on to lzc_clone. Which again accepts strings instead of handles for everything.
zfs_rollback
takes a zfs_handle_t* for the filesystem, a zfs_handle_t* for the snapshot and a force flag as argument. This is strange, since snapshots only apply to one filesystem. Internally, this function first destroys all bookmarks and snapshots that are newer, and then calls lzc_rollback_to, passing it the names of the base dataset and the snapshot.
zfs_destroy
is much lower level than the previous functions. It can destroy datasets and snapshots, which are passed in via a handle. But it doesn’t allow any recursion, unlike zfs_rollback, or even unmount the dataset. And unlike zfs_clone, it doesn’t verify if the operation makes sense either. And it directly interacts with zfs_ioctl. If it wasn’t for taking a handle as parameter, it’d feel right at home with the lzc family of functions.

Planed Improvements to ZetaWatch

  • Capability for Clone Promotion
  • Ask for confirmation before destroying snapshots / clones on dataset rollback
  • Recursive destruction with confirmation.
  • Better error message on non-recursive destruction.

Comment

ZetaWatch for Developers · 2019-05-05 17:37 by Black in

ZFS Interaction

ZetaWatch communicates with zfs using libzfs.dylib, libzfs_core.dylib, libzpool.dylib and libnvpair.dylib, just like the command line tools do. This gives it all the flexibility of the command line tools, at the cost of having to reimplement functionality that is found in the tools and not the library. And since the libraries are
explicitly not meant to provide a stable ABI, ZetaWatch is also closely coupled to the ZFS version it is built and written for.

All the ZFS interaction is wrapped in the ZFSWrapper library. This C++ library isolates the issues mentioned above and provides a more convenient and safe API than the original C interface does. The library is used both by the helper tool and the frontend app. This is the most reusable part of ZetaWatch, and might be split out as separate project later.

ZFSUtils
contains most of the advanced functionality, such as C++ Wrappers around the library, pool, vdev or file system handles. Those classes also have functionality to query state and iterate over members.
ZFSNVList
provides a wrapper around the nvpair_t / nvlist_t data structure that is used in ZFS for a lot of userland / kernel communication. It manages resources in both owning and non-owning fashion, and allows for easier iteration over sequences.
ZFSStrings
translate ZFS status enums into the user facing emoji or string description, optionally with localization. (Localization is not well tested or supported at the moment.)

Helper Tool

The implementation of the helper tool follows apple’s EvenBetterAuthorizationSample

The helper tool communicates with the user application via AuthorizationService and NSXPCConnection. The application side of code for this is in ZetaAuthorization.m. The RPC protocol can be found in ZetaAuthorizationHelperProtocol.h, and is implemented in ZetaAuthorizationHelper.mm. The CommonAuthorization.m file contains the supported commands and associated default permissions.

The helper tool can be uninstalled with the `uninstall-helper.sh` script. This is useful for debugging the installation of the helper, or updating the helper without increasing the bundle version.

Security & Code Signing

Official release builds are signed and notarized, and should run without issues even on newer Mac OS X. But there are still issues with authentication reported with the program not being recognized as signed. To verify security manually, the following commands can be used:

codesign -v -v -d ZetaWatch.app
xcrun stapler validate -v ZetaWatch.app

Building ZetaWatch requires an apple developer account with DeveloperID signing
capabilities, since it uses SMJobBless to run a helper service as root. This service
executes actions on behalf of the user, such as mounting, unmounting or loading a key.
Notarization is required to create binaries that can be run without without warning on
the newest Mac OS X.

Comment

ZetaWatch · 2019-05-05 01:54 by Black in

ZetaWatch is a small OS X program that displays the zfs status in the menu bar, similar to what iStat Menus does for other information. It is fairly well tested, but due to the current state of libzfs and libzfs_core, changes will be required until the API
stabilizes. ZetaWatch is usually compiled for the latest available ZFS release for Mac OS, and might not be compatible with other releases.

Currently supported features are:

  • Show pool and vdev status including scrub progress
  • Show pool / filesystem properties
  • Show filesystem and vdev stats
  • Import pools and Export pools
  • Mount / unmount datasets
  • Load encryption keys for encrypted datasets
  • Scrub pools
  • Report errors in notification center

Check it out :)

Comment

CaptureOne for Sony · 2019-03-31 14:54 by Black in

Apple’s Photos isn’t too great, Adobe’s Lightroom requries a subscription, Sony’s Edit is quite lacking in feature. After some searching around, I settled for PhaseOne’s Capture One, and it’s free, for Sony (or Fujifilm) cameras.

Some features are only available in the pro version, which is available as pay-once or subscription, but the free version is already very feature rich, and includes asset management, raw editing and more. And it’s quite easy to use.

This picture was taken under dim sodium light, handheld, with ISO 6400, and edited slightly (and not very skillfully) in CaptureOne Express.

Here a 1:1 cutout of the original, converted to JPEG with Apple’s Preview.

One click to adjust white balance, a bit of playing with auto-exposure and the curves to change the light, and adjust noise reduction a bit.

Comment