[virt-tools-list] [Libguestfs] ANNOUNCE: libnbd 1.2 & nbdkit 1.16 - high performance NBD client and server
Brett Thurber
bthurber at redhat.com
Thu Nov 14 19:17:32 UTC 2019
Very nice and congrats on getting this in.
Brett
On Thu, Nov 14, 2019 at 5:53 AM Richard W.M. Jones <rjones at redhat.com>
wrote:
> I'm pleased to announce the releases of libnbd 1.2 and nbdkit 1.16.
> These are a high performance Network Block Device (NBD) client library
> and server.
>
>
> Key features of libnbd:
>
> * Synchronous API for ease of use.
> * Asynchronous API for writing non-blocking, multithreaded clients.
> You can mix both APIs freely.
> * High performance.
> * Minimal dependencies for the basic library.
> * Well-documented, stable API.
> * Bindings in several programming languages.
> * Shell (nbdsh) for command line and scripting.
>
> Git: https://github.com/libguestfs/libnbd
> Download: http://download.libguestfs.org/libnbd/1.2-stable/
> Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=28807
>
>
> Key features of nbdkit:
>
> * Multithreaded NBD server written in C with good performance.
> * Minimal dependencies for the basic server.
> * Liberal license (BSD) allows nbdkit to be linked to proprietary
> libraries or included in proprietary code.
> * Well-documented, simple plugin API with a stable ABI guarantee.
> Lets you export “unconventional” block devices easily.
> * You can write plugins in C, Lua, Perl, Python, OCaml, Ruby, Rust,
> shell script or Tcl.
> * Filters can be stacked in front of plugins to transform the output.
>
> Git: https://github.com/libguestfs/nbdkit
> Download: http://download.libguestfs.org/nbdkit/1.16-stable/
> Fedora: https://koji.fedoraproject.org/koji/packageinfo?packageID=16469
>
>
> *** Release notes for libnbd 1.2 ***
>
> These are the release notes for libnbd stable release 1.2. This
> describes the major changes since 1.0.
>
> libnbd 1.2.0 was released on 14th November 2019.
>
> Security
> Two security problems were found during development of libnbd 1.2.
> Both were backported to the 1.0 stable branch. Upgrading is highly
> advisable.
>
> CVE-2019-14842 protocol downgrade attack when using
> "LIBNBD_TLS_REQUIRE"
>
> See the full announcement and links to mitigation, tests and fixes
> here:
>
> https://www.redhat.com/archives/libguestfs/2019-September/msg00128.html
>
> remote code execution vulnerability
>
> See the full announcement here:
>
> https://www.redhat.com/archives/libguestfs/2019-October/msg00060.html
>
> New APIs
> nbd_can_fast_zero(3)
> Test support by the server for fast zeroing (Eric Blake).
>
> nbd_connect_socket(3)
> nbd_aio_connect_socket(3)
> Connect to a local connected socket which you create in your
> main
> program using your own chosen method.
>
> nbd_connect_systemd_socket_activation(3)
> nbd_aio_connect_systemd_socket_activation(3)
> Connect to local processes that support systemd socket
> activation.
>
> nbd_connect_vsock(3)
> nbd_aio_connect_vsock(3)
> Used to connect to servers over "AF_VSOCK".
>
> nbd_get_handshake_flags(3)
> nbd_set_handshake_flags(3)
> nbd_get_request_structured_replies(3)
> nbd_set_request_structured_replies(3)
> nbd_get_structured_replies_negotiated(3)
> Can be used when testing NBD servers to avoid various NBD
> features
> (Eric Blake).
>
> nbd_get_protocol(3)
> Get the NBD protocol variant that the server supports.
>
> nbd_get_tls_negotiated(3)
> Did we actually negotiate a TLS connection?
>
> nbd_set_uri_allow_local_file(3)
> nbd_set_uri_allow_tls(3)
> nbd_set_uri_allow_transports(3)
> These can be used to filter NBD URIs before calling
> nbd_connect_uri(3).
>
> New features
> New tool nbdfuse(1) lets you create a loop-mounted file backed by an
> NBD server without needing root.
>
> "AF_VSOCK" is now a supported protocol (thanks Stefan Hajnoczi and
> Stefano Garzarella).
>
> Support for the "FAST_ZERO" flag (Eric Blake).
>
> Allow disabling certain protocol features, to make it easier to test
> servers (Eric Blake).
>
> Stack-allocated Variable Length Arrays (VLAs) are now banned
> throughout
> the library, making the library easier to consume from threads and
> other small stack situations.
>
> Reproducible builds (Chris Lamb).
>
> Support for filtering potentially dangerous or undesirable NBD URI
> features.
>
> Documentation
> Many improvements to the generated manual pages, including:
>
> · Separate "RETURN VALUE" and "ERRORS" sections for each API
> function.
>
> · Example code.
>
> · Relevant links can be added to the "SEE ALSO" section.
>
> · Link to NBD URI specification where relevant, and improve
> documentation around what URIs libnbd supports.
>
> · Document libnbd version number scheme.
>
> · Document limits on export name length, encoding etc.
>
> New libnbd-security(3) man page listing past security issues and
> remediations (Eric Blake).
>
> Tools
> nbdsh(1) has a new --base-allocation option which can be used to
> request "base:allocation" metadata context.
>
> New nbdsh(1) --uri (-u) option to connect to URIs.
>
> Tests
> You can now fuzz libnbd using either American Fuzzy Lop or clang’s
> libFuzzer.
>
> Add unit tests for nbdsh(1) (Eric Blake).
>
> Improved interop testing with various NBD servers and features.
>
> Other improvements and bug fixes
> nbd_connect_tcp(3) now tries to return the correct errno(3) from the
> underlying connect(2) call when that fails.
>
> The nbd-protocol.h header file is now shared between libnbd and
> nbdkit.
>
> Better fork-safety in "nbd_connect_*" APIs.
>
> The code was analyzed with Coverity and various problems identified
> and
> fixed.
>
>
> *** Release notes for nbdkit 1.16 ***
>
> These are the release notes for nbdkit stable release 1.16. This
> describes the major changes since 1.14.
>
> nbdkit 1.16.0 was released on 14th November 2019.
>
> Security
> Two security issues were found during development of nbdkit 1.16.
> Fixes for these were backported to older stable branches.
> Upgrading to
> the fixed versions is highly recommended. The new
> nbdkit-security(1)
> man page contains an up to date list of past security issues.
>
> CVE-2019-14850 denial of service due to premature opening of
> back-end
> connection
>
> See the full announcement and links to mitigation, tests and fixes
> here:
>
> https://www.redhat.com/archives/libguestfs/2019-September/msg00084.html
>
> CVE-2019-14851 assertion failure by issuing commands in the wrong
> order
>
> This CVE was caused by the fix to the previous issue.
>
> See the full announcement and links to mitigation, tests and fixes
> here:
>
> https://www.redhat.com/archives/libguestfs/2019-September/msg00272.html
>
> New features
> Add support for fast zeroing. Plugins can expose this using the new
> ".can_fast_zero" method (Eric Blake).
>
> nbdkit-partitioning-plugin(1) allows use of "mbr-id=default" or
> "type-guid=default" to go back to the default MBR byte or partition
> type GUID.
>
> New --mask-handshake server flag can be used for testing client
> feature
> negotiation (Eric Blake).
>
> The client export name is passed to nbdkit-captive(1) --run
> parameter
> as $exportname (Eric Blake).
>
> Captive --run commands which fail (eg. aborting) now cause nbdkit to
> exit with an error instead of errors being silently ignored (Eric
> Blake).
>
> File descriptors can be passed to password parameters, eg:
> "password=-3" which means that the password should be read from file
> descriptor 3.
>
> nbdkit can now serve over the "AF_VSOCK" protocol (thanks Stefan
> Hajnoczi).
>
> New --log=null option discards error messages.
>
> Plugins
> Python 2 support has been dropped from nbdkit-python-plugin(3) in
> line
> with Python 2 end of life at the beginning of 2020. Python ≥ 3.3 is
> required by this plugin. If you wish to continue to use Python 2
> then
> you will need to use nbdkit 1.14.
>
> New nbdkit-info-plugin(1) which returns various server information
> back
> to the client. It can be used for testing server latency amongst
> other
> things.
>
> nbdkit-data-plugin(1) now allows you to write "BYTE*N" to get
> repeated
> bytes (eg. nbdkit data data="0x55*4096").
>
> nbdkit-ssh-plugin(1) new parameter "compression=true|false" to
> control
> transport compression.
>
> nbdkit-vddk-plugin(1) is no longer compiled on non-x86 platforms
> since
> VMware has only ever shipped VDDK on x86.
>
> nbdkit-sh-plugin(1) scripts can now see the client exportname and
> can
> use the "magic_config_key" feature.
>
> Filters
> New nbdkit-retry-filter(1) which can reopen the plugin
> transparently on
> certain types of failures (lots of help from Eric Blake).
>
> API
> Macros "NBDKIT_VERSION_MAJOR", "NBDKIT_VERSION_MINOR",
> "NBDKIT_VERSION_MICRO" expose the compile-time version of nbdkit to
> plugins and filters (Eric Blake).
>
> Filters (which unlike plugins do not have a public stable API) must
> now
> exactly match the version of nbdkit when loaded (Eric Blake).
>
> New ".can_fast_zero" method (Eric Blake).
>
> New "nbdkit_export_name" server function for reading the export name
> passed by the client.
>
> New "nbdkit_peer_name" server function to return the client address
> (like getpeername(2)).
>
> New server functions for safely parsing integers:
> "nbdkit_parse_int",
> "nbdkit_parse_unsigned", "nbdkit_parse_int8_t",
> "nbdkit_parse_uint8_t",
> "nbdkit_parse_int16_t", "nbdkit_parse_uint16_t",
> "nbdkit_parse_int32_t", "nbdkit_parse_uint32_t",
> "nbdkit_parse_int64_t", "nbdkit_parse_uint64_t".
>
> Bug fixes
> ".trim" with FUA flag set now works (Eric Blake).
>
> Documentation
> The previous release notes have been turned into man pages.
>
> Tests
> Several tests now optionally use nbdsh(1) instead of qemu-io.
>
> You can now fuzz nbdkit using either American Fuzzy Lop or clang’s
> libFuzzer.
>
> Several tests have had sleep times increased to make them more
> stable
> when run on slow or heavily loaded machines.
>
> Internals
> Reproducible builds (Chris Lamb).
>
> Compile code with -Wshadow warning (Eric Blake).
>
> The internal backend system has been extensively overhauled. In
> particular this means that we now validate request ranges as
> requests
> are passed between filters and down to the plugin, making it easier
> to
> find bugs in filters early (Eric Blake).
>
> Plugin size and "can_*" flags are cached more aggressively by the
> server (Eric Blake).
>
> Variable Length Arrays (VLAs) on stack are now banned throughout the
> code.
>
> The nbd-protocol.h header describing the NBD protocol is now shared
> with libnbd(3).
>
> Plugin ".unload" method is now called after all worker threads have
> exited, avoiding races at server shutdown.
>
> Code was audited using Coverity and various problems were fixed.
>
>
>
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-p2v converts physical machines to virtual machines. Boot with a
> live CD or over the network (PXE) and turn machines into KVM guests.
> http://libguestfs.org/virt-v2v
>
> _______________________________________________
> Libguestfs mailing list
> Libguestfs at redhat.com
> https://www.redhat.com/mailman/listinfo/libguestfs
--
Brett Thurber - RHCA, RHCVA
Distinguished Engineer and Engineering Manager, Migration Engineering
Products & Technologies Group, Red Hat
Mobile: +1 (512) 547-9282
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/virt-tools-list/attachments/20191114/1aabe367/attachment-0001.htm>
More information about the virt-tools-list
mailing list