OpenZFS Developer Summit 2014

From OpenZFS
Jump to navigation Jump to search
Open-zfs-san-francisco-logo.png The second annual OpenZFS Developer Summit was held in San Francisco, November 10-11, 2014. All OpenZFS developers were invited to participate.



The goal of the event was to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. This 2-day event consisted of 1 day of presentations (Nov 10) and a 1-day hackathon (Nov 11).

Details about registration, presentations, and sponsorship opportunities were provided on this page.

Event

Photos

Photos of the event can be viewed on the OpenZFS Google+ page.

Presentations

Title Speaker Slides Video
Keynote Matt Ahrens from Delphix slides video
Platform Overview: ZFS on OSv Raphael Carvalho from Cloudius slides video
Platform Overview: ZFS on Linux Brian Behlendorf from LLNL slides video
Platform Overview: illumos Prakash Surya from Delphix slides video
Platform Overview: FreeBSD Xin Li from iXsystems slides video
Platform Panel Q&A Above speakers - video
Manta Dave Pacheco from Joyent slides video
Compression Saso Kiselkov from Nexenta slides video
Performance George Wilson from Delphix slides video
Host-Aware SMR Tim Feldman from Seagate slides video
Fast File Cloning Pavel Zakharov from Imation slides video
Device Removal Alex Reece & Matt Ahrens from Delphix slides video
libzfs_core and ioctl stabalization Richard Yao from ClusterHQ slides video
Channel Programs Chris Siden from Delphix slides video
Company Lightning Talk: DDRdrive Chris George slides video
Company Lightning Talk: ClusterHQ Luke Marsden slides video
Company Lightning Talk: Delphix Matt Ahrens slides video
Company Lightning Talk: Coraid Richard Elling slides video
Company Lightning Talk: Storiant John Layman slides video
Company Lightning Talk: Syneto Dan Vatca slides video
Company Lightning Talk: Nexenta Tony Nguyen slides video
Company Lightning Talk: iX Systems Josh Paetzel slides video
Company Lightning Talk: Zacodi Labs Maxim Martynov slides video
Day 1 Closing Matt Ahrens from Delphix - video
Hackathon and Day 2 Closing Hackathon Participants - video


Projects started at the hackathon

  • Channel Programs - continuation of work from last year's hackathon
    • See project page, slides and 2013 and 2014 videos describing project
    • Chris and Paul - (Best in show winner) Demo of several working channel programs, lots of testing yet to do.
  • Upstream zfs-make - tool to quickly build zfs on illumos
    • Justin and Andiry - 5 new issues upstreamed for review, including a performance enhancement aggregating sync and async operations
    • Dan McDonald brought zmake to illumos
    • Steve, Chip, Brian B. - Building zfs on illumos
  • Poor mans compressed send (decomp when recv)
    • Matt Ahrens - Working prototype, some work remains (e.g. send flags)
  • Sha 512/256
    • Saso and John Mark - Implemented Blake2 hash instead. Saso also revived a SCSI unmap patch
  • Rip out scrub throttle, queue mb not ios
    • George Wilson - Mostly finished, needs testing
  • One man page per subcommand
    • Manoj - Completed this work
  • Use GRUB2 to boot Linux from a root pool with multiple vdevs (
  • vdev kstats (Richard)
    • Richard and Dan - Added several new per dataset zil kstats
  • object count per user in DMU
    • Isaac and Manoj - Fully vetted design, and implementation begun.
  • improve arc_kmem_reap()
    • Tony and George - Made progress on this, as well as improving the amount of memory consumed by arc buf headers when using l2arc.
  • quickly creating fragmented pools
    • Tony and George - Use a passed in histogram to create desired fragmentation level - near code complete
  • reconstruct space_maps
    • Albert - Has this nearly working
  • large dnodes
    • Ned worked on this, solving several crashes - still testing for edge cases
  • open-zfs infrastructure in docker (flocker)
    • Luke - made progress on the mailman and website infrastructure
  • fix OS-2313
    • Bryan Dan and Robert worked on a zil_close/spa_sync race that causes intermittent panics and hangs
  • fix zpool commands hanging when we have suspended pools
    • Don and Brian B. - Created a fix for this bug
  • Data collection using statistical methods for weighting recorded data by arbitrary time range
    • Alex and Prakash - A working demo measuring spa activity
  • Add ziltest to zfstest
    • John and Don - got this working, also added a zpool subcommand to list unused disks.

A huge THANK YOU to the sponsors of the 2014 OpenZFS Developer Summit:

Delphix logo.png Ddrdrive logo dev 2013.png Logo freebsdfoundation.png
Storiant.png Ixsystems.jpg New hac logo.jpg
Nexenta-GL-logo-tagline.jpg Syneto-logo-Sept14.svg Seagate 2c pos.jpg
OmniTI-logo.jpeg Clusterhq.png


Ideas for next hackathon

  • preferred allocation alignment (george)
  • JSON command output
  • separate manpages for each zfs subcommand
  • update build instructions
  • iokstats for sync vs async
  • iokstats for each dataset
  • fix race condition in zpool import device detection
  • zfs set fsid_guid= (and/or include fsid_guid in send stream)
  • new CLI option to enable send features: zfs send -x embedded,largetblock or -x all
  • JSON command output
  • File cloning with implicit indirection
  • print usage message for specific property
    • zfs help command
  • compile ZFS on Linux with -std=gnu99; does it work?
  • storage pool aging
    • specify histogram of free segment sizes
    • allocator chooses locations to satisfy histogram
  • "get next file" syscall
    • iterates in object # order, opening next file; zfs prefetches dnodes
    • could speed up operations that need to operate on every file in a filesystem, e.g. "chmod -r"
  • Porting Party - learn how to contribute changes to a new platform
  • performance analytics framework - record performance, store & display (dtrace + graphite?)
  • USE method data collection framework for performance analysis (http://www.brendangregg.com/USEmethod/use-solaris.html)
  • Illumos grub2
  • Clean up embedded zil code
  • UTF-8 dataset & poolname (Richard)
  • zfs_core collaboration (Richard)
  • collectd agents (Richard)
  • scrub/resilver progress (Richard)
  • promote prefetch reads to sync reads
  • make zfs recv less trusting
  • raidz1 for single device
  • static analysis tools for zfs
  • dynamic analysis for ztest
  • improved documentation for developing zfs on linux
  • runtime lock analyzer
  • make dtrace probes in linux to work with lttng
  • per zone I/O QoS
  • convert AVL trees to B-trees or hash tables
  • cross platform xattr
  • use refcount_t for v_count

Harassment Policy

OpenZFS Developer Summit is dedicated to providing a harassment-free conference experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion. We do not tolerate harassment of conference participants in any form. Sexual language and imagery is not appropriate for any conference venue, including talks. Conference participants violating these rules may be sanctioned or expelled from the conference at the discretion of the conference organizers.


For information about last year's event, see the OpenZFS Developer Summit 2013 page.