erlang/zfs
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
niamtokik
2018-01-04 22:11:17 +01:00
parent 6657d1dcd4
commit 2dc61a3463
21 changed files with 4603 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@@ -8,3 +8,7 @@ ebin/*.beam
rel/example_project
.concrete/DEV_MODE
.rebar
*.fossil
*~
_build
.rebar.lock

43
LICENSE
View File

@@ -1,29 +1,28 @@
BSD 3-Clause License
Copyright (c) 2018,
All rights reserved.
Copyright (c) 2017 Mathieu Kerjouan <contact@steepath.eu>
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

40
README.md Normal file
View File

@@ -0,0 +1,40 @@
# zfs
Erlang ZFS High and Low Level implementation.
## Usage
This is an Erlang Library, you can use it on any project
based on Licensing.
## Documentation
This project is born from one idea, reverse engineering ZFS
as gray box, from the hard way. We want to document every
ZFS aspect and offer a high level implementation to check if
specification and current existing documentation fit well.
You can find more documentation in doc directory at the root
of this project.
## Build
$ rebar3 compile
## Test
Erlang ZFS implementation is also a test battery for ZFS, you
can run different test to check if our implementation is well
supported on your hardware, but also directly on your OS.
For running unit test:
$ rebar3 eunit
For running common_test:
$ rebar3 ct
For running local low level test:
$ cd c_src && make test

0
c_src/Makefile Normal file
View File

26
doc/README.md Normal file
View File

@@ -0,0 +1,26 @@
# Documentation
## Why implement ZFS in Erlang?
Erlang is natively distributed, high level and stable language.
ZFS is complexe file-system requiring lot of knowledge. This
file-system is also an old-one, developped in the 2000's, it
wasn't developped with security in mind. ZFS stream for example
should be trusted.
Erlang is particularly efficient to parse binary string, ZFS
is a binary data-structure...
## What can we do with this library?
Currently, this library is only used to parse and check
ZFS stream send from `zfs send` command.
## Some idea of concrete work?
* authenticated flow from external customers
* authenticated flow from internal servers
* zfs stream split
* zfs stream fuzzing
* zfs stream distributed check
* and much more...

126
doc/notes.md Normal file
View File

@@ -0,0 +1,126 @@
# Notes
This is a kind of specification documentation
base on what I need and how to use it.
## ZFS Stream Parsing
First thing first. ZFS library for Erlang was made
to easily parse ZFS send/receive stream and split
it in small piece.
Parsing a zfs stream should be simple and return
standard and comprehensible data-structure.
zfs_stream:parse(Stream).
Sometime, our ZFS stream is a bit larger than expected,
so, we can filter it. This filtering will be done with
Erlang Match syntax.
zfs_stream:parse(Stream, [{'object', '_'}]).
### Specific Stream Parsing Feature
zfs_stream:parse(Stream, []).
ZFS Stream is a complex data-structure containing
different type of objects. We can parse it independantly
of each others.
% dmu replay record is a full
% zfs stream sent from zfs send command
dmu_replay_record:parse(Stream).
dmu_replay_record:parse(Stream, _Opts).
% drr_begin is the first data structure
% used in ZFS Stream and contain common
% information about this stream
drr_begin:parse(Stream).
drr_begin:parse(Stream; _Opts).
% drr_end
drr_end:parse(Stream).
drr_end:parse(Stream, _Opts).
% drr_object
drr_object:parse(Stream).
drr_object:parse(Stream, _Opts).
% drr_freeobjects
drr_freeobjects:parse(Stream).
drr_freeobjects:parse(Stream, _Opts).
% drr_write
drr_write:parse(Stream).
drr_write:parse(Stream, _Opts).
% drr_write_byref
drr_write_byref:parse(Stream).
drr_write_byref:parse(Stream, _Opts).
% drr_free
drr_free:parse(Stream).
drr_free:parse(Stream, _Opts).
% drr_spill
drr_spill:parse(Stream).
drr_spill:parse(Stream, _Opts).
Those functions return erlang map data-structure only
if the stream or part of the stream is the right object.
dmu_replay_record:parse/1 parse an entire ZFS stream and is
equivalent with zfs_stream:parse/1 function.
## ZFS Stream Splitting
A ZFS Stream is a range of ordered transactional ZFS objects.
This library is made to split stream in small pieces.
zfs_stream:split(Stream).
zfs_stream:split(Stream, _Opts).
You can split it in different way, just split all logical part
in bitstring or parse each part with erlang abstraction.
zfs_stream:split(Stream, [bitstring]).
zfs_stream:split(Stream, [map]).
## ZFS Stream Analysing
All ZFS data-structure are delivered with checksum, analyzing it
give us the change to validate every piece of a stream based
on checksum.
zfs_stream:analyze(Stream).
zfs_stream:analyze(Stream, _Opts).
## ZFS Stream Crafting
If we have the possibility to analyze or read a stream, we have
also the possibility to craft any kind of ZFS stream from scratch.
zfs_stream:craft(Type, Content).
## ZFS Stream Fuzzing
ZFS is executed in Kernel Land, if something goes wrong, in the
best case, system crash, in the worst case, an attacker can
install a rootkit or gain privilege access to our server. Fuzzing
is a technique used widely on security to check program inputs.
This high level library give us the possibility to randomly craft
ZFS stream and check them but also alterate an existing one.
% create a totally random zfs stream from scratch
zfs_stream:fuzz().
% generete an object stream based on Type
zfs_stream:fuzz(Type).
% extra options
zfs_stream:fuzz(Type, _Opts).
zfs_stream:fuzz(Type, [{destination, Destination}
,{source, Source}
,{alter, []}
,{format, Format}]).

25
doc/resources.md Normal file
View File

@@ -0,0 +1,25 @@
# Resources
== Video
* https://www.youtube.com/channel/UC0IK6Y4Go2KtRueHDiQcxow
== Website
* http://www.open-zfs.org/
== Papers
* http://www.giis.co.in/Zfs_ondiskformat.pdf
* https://www.usenix.org/legacy/events/fast10/tech/full_papers/zhang.pdf
* https://www.usenix.org/legacy/event/lisa07/htgr_files/bonwick_htgr.pdf
* https://www.usenix.org/system/files/conference/inflow14/inflow14-zuck.pdf
* http://supertech.csail.mit.edu/papers/YuanZhJa16.pdf
* https://cs.uwaterloo.ca/~mashti/cs854-f17/papers/zfs.pdf
* http://www.usenix.net/legacy/events/fast10/tech/full_papers/zhang.pdf
== Source code
* http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h
* http://bxr.su/FreeBSD/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h
* http://bxr.su/FreeBSD/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_sendrecv.c

2
rebar.config Normal file
View File

@@ -0,0 +1,2 @@
{erl_opts, [debug_info]}.
{deps, []}.

48
src/dmu.erl Normal file
View File

@@ -0,0 +1,48 @@
%%%===================================================================
%%%
%%%===================================================================
-module(dmu).
-export([replay_record/0, replay_record/1, replay_record/2]).
-include("zfs.hrl").
-include_lib("eunit/include/eunit.hrl").
-ifdef(debug).
-compile(export_all).
-endif.
%%%===================================================================
%%% API SECTION - EXPORTED FUNCTION
%%%===================================================================
-spec replay_record() -> list().
replay_record() ->
[drr_type, drr_payloadlen, drr_begin].
-spec replay_record(bitstring()) -> {ok, map(), bitstring()}.
replay_record(Bitstring) ->
replay_record(Bitstring, []).
-spec replay_record(bitstring(), list()) -> {ok, map(), bitstring()}.
replay_record(<<0:32, PayloadLen:32, Rest/bitstring>>, _Opts) ->
<< Begin:(304*8)/bitstring, _/bitstring>> = Rest,
{ok, Struct, <<>>} = drr:struct_begin(Begin),
Init = #{ drr_type => 'begin'
, drr_payloadlen => PayloadLen
, content => Struct },
{ok, Init, Rest};
replay_record(<<1:32, PayloadLen:32, Rest/bitstring>>, _Opts) ->
ok.
%%%===================================================================
%%% PRIVATE SECTION
%%%===================================================================
-spec drr_payloadlen(bitstring()) -> {ok, map(), bitstring()}.
drr_payloadlen(<<PayloadLen:32/little, Rest/bitstring>>) ->
{ok, #{ payloadlen => PayloadLen }, Rest}.
drr_payloadlen_test() ->
[ drr_payloadlen_test(X) || X <- lists:seq(1,1) ].
drr_payloadlen_test(1) ->
IN = <<1,0,0,0>>,
OUT = {ok, #{ payloadlen => 1 }, <<>>},
?assertEqual(OUT, drr_payloadlen(IN)).

812
src/drr.erl Normal file
View File

@@ -0,0 +1,812 @@
%%%===================================================================
%%% @author Mathieu Kerjouan
%%% @copyright 2017
%%% @version 0.1.0
%%% @title drr data structure implementation
%%% @doc
%%% @end
%%%===================================================================
-module(drr).
-export([struct_begin/0, struct_begin/1, struct_begin/2]).
-export([struct_end/0, struct_end/1, struct_end/2]).
-export([struct_object/0]).
-export([struct_freeobjects/0]).
-export([struct_write/0]).
-export([struct_free/0]).
-export([struct_write_byref/0]).
-export([struct_spill/0]).
-include("zfs.hrl").
-include_lib("eunit/include/eunit.hrl").
-ifdef(debug).
-compile(export_all).
-endif.
%%%===================================================================
%%% API SECTION - EXPORTED FUNCTION
%%%===================================================================
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_begin() -> list().
struct_begin() ->
[ drr_magic, drr_versioninfo, drr_creation_time, drr_type
, drr_flags, drr_toguid, drr_fromguid, drr_toname ].
-spec struct_begin(bitstring())
-> {ok, map()} |
{ok, map(), bitstring()} |
{error, term()}.
struct_begin(Bitstring) ->
struct_begin(Bitstring, []).
-spec struct_begin(bitstring(), list())
-> {ok, map()} |
{ok, map(), bitstring()} |
{error, term()}.
struct_begin(Bitstring, _Opts) ->
Return = #{},
{ok, Magic, MagicRest} = drr_magic(Bitstring),
Return1 = maps:merge(Return, Magic),
{ok, VersionInfo, VersionInfoRest} = drr_versioninfo(MagicRest),
Return2 = maps:merge(Return1, VersionInfo),
{ok, CreationTime, CreationTimeRest} = drr_creation_time(VersionInfoRest),
Return3 = maps:merge(Return2, CreationTime),
{ok, Type, TypeRest} = objset_type(CreationTimeRest),
Return4 = maps:merge(Return3, Type),
{ok, Flags, FlagsRest} = drr_flags(TypeRest),
Return5 = maps:merge(Return4, Flags),
{ok, ToGuid, ToGuidRest} = drr_toguid(FlagsRest),
Return6 = maps:merge(Return5, ToGuid),
{ok, FromGuid, FromGuidRest} = drr_fromguid(ToGuidRest),
Return7 = maps:merge(Return6, FromGuid),
{ok, ToName, ToNameRest} = drr_toname(FromGuidRest),
Return8 = maps:merge(Return7, ToName),
{ok, Return8, ToNameRest}.
struct_begin_test() ->
[ struct_begin_test(X) || X <- [1] ].
struct_begin_test(1) ->
IN = <<172, 203, 186, 245, 2, 0, 0, 0, 17, 0, 0, 0, 0, 0, 0, 0, 102,
47, 41, 90, 0, 0, 0, 0, 2, 0, 0, 0, 4, 0, 0, 0, 54, 80, 15,
33, 249, 114, 233, 135, 0, 0, 0, 0, 0, 0, 0, 0, 112, 111,
111, 108, 64, 116, 101, 115, 116, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 >>,
OUT = {ok
, #{ magic => <<16#2F5bacbac:64/little>>
, creation_time => 16#5a292f66
, type => zfs
, featuresflags => 4
, flags => <<4,0,0,0>>
, hdrtype => substream
, toguid => <<54,80,15,33,249,114,233,135>>
, fromguid => <<0,0,0,0,0,0,0,0>>
, toname => <<"pool@test">>
% , versioninfo => <<17,0,0,0,0,0,0,0>>
}
, <<>>
},
?assertEqual(OUT, struct_begin(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_end() -> list().
struct_end() ->
[ drr_checksum, drr_toguid ].
-spec struct_end(bitstring())
-> {ok, map()} |
{ok, map(), bitstring()} |
{error, term()}.
struct_end(Bitstring) ->
struct_end(Bitstring, []).
-spec struct_end(bitstring(), list())
-> {ok, map()} |
{ok, map(), bitstring()} |
{error, term()}.
struct_end(Bitstring, _Opts) ->
Init = #{},
{ok, Checksum, ChecksumRest} = drr_checksum(Bitstring),
Return1 = maps:merge(Checksum, Init),
{ok, ToGuid, ToGuidRest} = drr_toguid(ChecksumRest),
Return2 = maps:merge(ToGuid, Return1),
{ok, Return2, ToGuidRest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_object() -> list().
struct_object() ->
[ drr_object, drr_type, drr_bonustype, drr_blksz, drr_bonuslen
, drr_checksumtype, drr_compress, {drr_pad, 6}, drr_toguid ].
% -spec object(bitstring())
% -> {ok, drr_object()} |
% {ok, drr_object(), bitstring()} |
% {error, term()}.
% -spec object(bitstring(), list())
% -> {ok, drr_object()} |
% {ok, drr_object(), bitstring()} |
% {error, term()}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_freeobjects() -> list().
struct_freeobjects() ->
[ drr_firstobj, drr_numobjs, drr_toguid ].
-spec struct_freeobjects(bitstring())
-> {ok, map()} |
{ok, map(), bitstring()} |
{error, term()}.
struct_freeobjects(Bitstring) ->
struct_freeobjects(Bitstring, []).
-spec struct_freeobjects(bitstring(), list())
-> {ok, map()} |
{ok, map(), bitstring()} |
{error, term()}.
struct_freeobjects(Bitstring, _Opts) ->
{wip, Bitstring}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_write() -> list().
struct_write() ->
[ drr_object, drr_type, {drr_pad, 4}, drr_offset, drr_length
, drr_toguid, drr_checksumtype, drr_checksumflags, {drr_pad, 2}
, drr_key ].
% -spec write(bitstring())
% -> {ok, drr_write()} |
% {ok, drr_write(), bitstring()} |
% {error, term()}.
% -spec write(bitstring(), list())
% -> {ok, drr_write()} |
% {ok, drr_write(), bitstring()} |
% {error, term()}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_free() -> list().
struct_free() ->
[ drr_object, drr_offset, drr_length, drr_toguid ].
% -spec free(bitstring())
% -> {ok, drr_free()} |
% {ok, drr_free(), bitstring()} |
% {error, term()}.
% -spec free(bitstring(), list())
% -> {ok, drr_free()} |
% {ok, drr_free(), bitstring()} |
% {error, term()}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_write_byref() -> list().
struct_write_byref() ->
[ drr_object, drr_offset, drr_length, drr_toguid, drr_refguid
, drr_refobject, drr_refoffset, drr_checksumtype, drr_checksumflags
, {drr_pad, 6}, drr_key ].
% -spec byref(bitstring())
% -> {ok, drr_byref()} |
% {ok, drr_byref(), bitstring()} |
% {error, term()}.
% -spec byref(bitstring(), list())
% -> {ok, drr_byref()} |
% {ok, drr_byref(), bitstring()} |
% {error, term()}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec struct_spill() -> list().
struct_spill() ->
[ drr_object, drr_length, drr_toguid, {drr_pad, 4} ].
% -spec struct_spill(bitstring())
% -> {ok, map()} |
% {ok, map(), bitstring()} |
% {error, term()}.
% -spec struct_spill(bitstring(), list())
% -> {ok, map()} |
% {ok, map(), bitstring()} |
% {error, term()}.
%%%===================================================================
%%% INTERNAL SECTION - PRIVATE FUNCTION
%%%===================================================================
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_blksz(bitstring())
-> {ok, map(), bitstring()}.
drr_blksz(<<BlkSz:32/little, Rest/bitstring>>) ->
{ok, #{ blksz => BlkSz }, Rest}.
drr_blksz_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_bonuslen(bitstring())
-> {ok, map(), bitstring()}.
drr_bonuslen(<<BonusLen:32/little, Rest/bitstring>>) ->
{ok, #{bonuslen => BonusLen}, Rest}.
drr_bonuslen_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_bonustype(bitstring())
-> {ok, atom(), bitstring()}.
drr_bonustype(<<BonusType:64/little, Rest/bitstring>>) ->
{ok, bonustype(BonusType), Rest/bitstring}.
drr_bonustype_test() ->
[ drr_bonustype_test(X) || X <- lists:seq(1,1) ].
drr_bonustype_test(1) ->
?assertEqual(ok, ok).
% http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h#126
-define(BONUSTYPE(X,A), bonustype(X) when is_integer(X) -> A;
bonustype(A) when is_atom(A) -> X).
?BONUSTYPE(0, none);
?BONUSTYPE(1, object_directory);
?BONUSTYPE(2, object_array);
?BONUSTYPE(3, packed_nvlist);
?BONUSTYPE(4, packed_nvlist_size);
?BONUSTYPE(5, pbobj);
?BONUSTYPE(6, pbobj_hdr);
?BONUSTYPE(7, space_map_header);
?BONUSTYPE(8, space_map);
?BONUSTYPE(9, intent_log);
?BONUSTYPE(10, dnode);
?BONUSTYPE(11, objset);
?BONUSTYPE(12, dsl_dir);
?BONUSTYPE(13, dsl_dir_child_map);
?BONUSTYPE(14, dsl_ds_snap_map);
?BONUSTYPE(15, dsl_ds_props);
?BONUSTYPE(16, dsl_dataset);
?BONUSTYPE(17, znode);
?BONUSTYPE(18, oldacl);
?BONUSTYPE(19, plain_file_contents);
?BONUSTYPE(20, directory_contents);
?BONUSTYPE(21, master_node);
?BONUSTYPE(22, unlinked_set);
?BONUSTYPE(23, zvol);
?BONUSTYPE(25, zvol_prop);
?BONUSTYPE(26, plain_other);
?BONUSTYPE(27, uint64_other);
?BONUSTYPE(28, zap_other);
?BONUSTYPE(29, error_log);
?BONUSTYPE(30, spa_history);
?BONUSTYPE(31, spa_history_offsets);
?BONUSTYPE(32, pool_props);
?BONUSTYPE(33, dsl_perms);
?BONUSTYPE(34, acl);
?BONUSTYPE(35, sysacl);
?BONUSTYPE(36, fuid);
?BONUSTYPE(37, fuid_size);
?BONUSTYPE(38, next_clones);
?BONUSTYPE(39, scan_queue);
?BONUSTYPE(40, usergroup_used);
?BONUSTYPE(41, usergroup_quota);
?BONUSTYPE(42, userrefs);
?BONUSTYPE(43, ddt_zap);
?BONUSTYPE(44, ddt_stats);
?BONUSTYPE(45, sa);
?BONUSTYPE(46, sa_master_node);
?BONUSTYPE(47, sa_attr_registration);
?BONUSTYPE(48, sa_attr_layouts);
?BONUSTYPE(49, scan_xlate);
?BONUSTYPE(50, dedup);
?BONUSTYPE(51, deadlist);
?BONUSTYPE(52, deadlist_hdr);
?BONUSTYPE(53, dsl_clones);
?BONUSTYPE(54, bpobj_subobj);
?BONUSTYPE(55, numtypes).
% DMU_OTN_UINT8_DATA = DMU_OT(DMU_BSWAP_UINT8, B_FALSE),
% DMU_OTN_UINT8_METADATA = DMU_OT(DMU_BSWAP_UINT8, B_TRUE),
% DMU_OTN_UINT16_DATA = DMU_OT(DMU_BSWAP_UINT16, B_FALSE),
% DMU_OTN_UINT16_METADATA = DMU_OT(DMU_BSWAP_UINT16, B_TRUE),
% DMU_OTN_UINT32_DATA = DMU_OT(DMU_BSWAP_UINT32, B_FALSE),
% DMU_OTN_UINT32_METADATA = DMU_OT(DMU_BSWAP_UINT32, B_TRUE),
% DMU_OTN_UINT64_DATA = DMU_OT(DMU_BSWAP_UINT64, B_FALSE),
% DMU_OTN_UINT64_METADATA = DMU_OT(DMU_BSWAP_UINT64, B_TRUE),
% DMU_OTN_ZAP_DATA = DMU_OT(DMU_BSWAP_ZAP, B_FALSE),
% DMU_OTN_ZAP_METADATA = DMU_OT(DMU_BSWAP_ZAP, B_TRUE),
%%--------------------------------------------------------------------
%% @doc
%% @end
%% @todo check data checksum
%%--------------------------------------------------------------------
-spec drr_checksum(bitstring)
-> {ok, bitstring(), bitstring}.
drr_checksum(<<Checksum:256/bitstring, Rest/bitstring>>) ->
?debugFmt("drr_checksum: ~p", [Checksum]),
{ok, Checksum, Rest}.
drr_checksum_test() ->
[ drr_checksum_test(X) || X <- [1] ].
drr_checksum_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_checksumflags(bitstring()) -> {ok, bitstring(), bitstring}.
drr_checksumflags(<<ChecksumFlags:8, Rest/bitstring>>) ->
?debugFmt("drr_checksumflags: ~p", [ChecksumFlags]),
{ok, ChecksumFlags, Rest}.
drr_checksumflags_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_checksumtype(bitstring()) -> {ok, atom(), bitstring()}.
drr_checksumtype(<<ChecksumType:8/little, Rest/bitstring>>) ->
?debugFmt("drr_checksumtype: ~p", [ChecksumType]),
{ok, ChecksumType, Rest}.
drr_checksumtype_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_compress(bitstring()) -> {ok, atom(), bitstring()}.
drr_compress(<<Compress:8/little, Rest/bitstring>>) ->
?debugFmt("drr_compress: ~p", [Compress]),
{ok, Compress, Rest}.
drr_compress_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_creation_time(bitstring())
-> {ok, map(), bitstring()}.
drr_creation_time(<<CreationTime:64/little, Rest/bitstring>>) ->
?debugFmt("drr_creation_time: ~p", [CreationTime]),
{ok, #{ creation_time => CreationTime }, Rest}.
drr_creation_time_test() ->
[drr_creation_time_test(X) || X <- [1] ].
drr_creation_time_test(1) ->
IN = <<16#66, 16#2f, 16#29, 16#5a,0,0,0,0>>,
OUT = {ok, #{ creation_time => 16#5a292f66 }, <<>>},
?assertEqual(OUT, drr_creation_time(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_firstobj(bitstring())
-> {ok, bitstring(), bitstring()}.
drr_firstobj(<<FirstObj:64/bitstring, Rest/bitstring>>) ->
?debugFmt("drr_firstobj: ~p", [FirstObj]),
{ok, FirstObj, Rest}.
drr_firstobj_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_flags(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_flags(<<Flags:32/bitstring, Rest/bitstring>>) ->
?debugFmt("drr_flags: ~p", [Flags]),
{ok, #{ flags => Flags }, Rest}.
drr_flags_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_fromguid(bitstring()) -> {ok, map(), bitstring()}.
drr_fromguid(<<FromGuid:64/bitstring, Rest/bitstring>>) ->
?debugFmt("drr_fromguid: ~p", [FromGuid]),
{ok, #{ fromguid => FromGuid }, Rest}.
drr_fromguid_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_key(bitstring()) -> {ok, map(), bitstring()}.
drr_key(<<CkSum:256/bitstring, Prop:64/bitstring, Rest/bitstring>>) ->
?debugFmt("drr_key: ~p, ~p", [CkSum, Prop]),
{ok, #{cksum => CkSum, prop => Prop}, Rest}.
drr_key_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_length(bitstring()) -> {ok, map(), bitstring()}.
drr_length(<<Length:64/little, Rest/bitstring>>) ->
?debugFmt("drr_length: ~p", [Length]),
{ok, #{ length => Length }, Rest}.
drr_length_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_magic(bitstring())
-> {ok, map(), bitstring()} |
{error, term()}.
drr_magic(<<16#2F5bacbac:64/little, Rest/bitstring>>) ->
?debugFmt("drr_magic: ok", []),
{ok, #{ magic => <<16#2F5bacbac:64/little>> }, Rest};
drr_magic(<<_:64, _/bitstring>>) ->
{error, magic}.
drr_magic_test() ->
[ drr_magic_test(X) || X <- lists:seq(1,2) ].
drr_magic_test(1) ->
IN = <<172,203,186,245,2,0,0,0>>,
OUT = {ok, #{magic => <<16#2F5bacbac:64/little>>}, <<>>},
?assertEqual(OUT, drr_magic(IN));
drr_magic_test(2) ->
IN = <<1,2,3,4, 5,6,7,8>>,
OUT = {error, magic},
?assertEqual(OUT, drr_magic(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_numobjs(bitstring()) -> {ok, non_neg_integer(), bitstring()}.
drr_numobjs(<<NumObjs:64/little, Rest/bitstring>>) ->
?debugFmt("drr_numobjs: ~p", [NumObjs]),
{ok, NumObjs, Rest}.
drr_numobjs_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_object(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_object(<<Object:64/bitstring, Rest/bitstring>>) ->
{ok, Object, Rest}.
drr_object_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_offset(bitstring()) -> {ok, non_neg_integer(), bitstring()}.
drr_offset(<<Offset:64/little, Rest/bitstring>>) ->
{ok, Offset, Rest}.
drr_offset_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_pad(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_pad(<<Pad:32, Rest/bitstring>>) ->
{ok, Pad, Rest}.
drr_pad_test(1) ->
?assertEqual(ok, ok).
-spec drr_pad2(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_pad2(<<Pad:8, Rest/bitstring>>) ->
{ok, Pad, Rest}.
drr_pad2_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%
%%--------------------------------------------------------------------
-spec drr_refguid(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_refguid(<<RefGuid:64/bitstring, Rest/bitstring>>) ->
{ok, RefGuid, Rest}.
drr_refguid_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%
%%--------------------------------------------------------------------
-spec drr_refobject(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_refobject(<<RefObject:64/bitstring, Rest/bitstring>>) ->
{ok, RefObject, Rest}.
drr_refobject_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%
%%--------------------------------------------------------------------
-spec drr_refoffset(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_refoffset(<<RefOffset:64/bitstring, Rest/bitstring>>) ->
{ok, RefOffset, Rest}.
drr_refoffset_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%
%%--------------------------------------------------------------------
-spec drr_toguid(bitstring())
-> {ok, map(), bitstring()}.
drr_toguid(<<ToGuid:64/bitstring, Rest/bitstring>>) ->
{ok, #{ toguid => ToGuid }, Rest}.
drr_toguid_test() ->
[ drr_toguid_test(X) || X <- [1] ].
drr_toguid_test(1) ->
IN = <<16#36, 16#50, 16#0f, 16#21, 16#f9, 16#72, 16#e9, 16#87>>,
OUT = {ok, #{ toguid => <<54,80,15,33,249,114,233,135>>}, <<>>},
?assertEqual(OUT, drr_toguid(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%% @todo check snapshot name.
%%--------------------------------------------------------------------
-spec drr_toname(bitstring())
-> {ok, map(), bitstring()}.
drr_toname(<<ToName:(256*8)/bitstring, Rest/bitstring>>) ->
?debugFmt("drr_toname: ~p", [ToName]),
{ok, #{ toname => zfs_lib:null_clean(ToName) }, Rest}.
drr_toname_test() ->
[drr_toname_test(X) || X <- lists:seq(1, 2) ].
drr_toname_test(1) ->
IN = <<>>,
?assertException(error, _, drr_toname(IN));
drr_toname_test(2) ->
IN = <<"pool@test", 0:(256*8-72)>>,
OUT = {ok, #{ toname => <<"pool@test">> }, <<>>},
?assertEqual(OUT, drr_toname(IN));
drr_toname_test(3) ->
IN = <<"zpooltest", 0:(256*8-72)>>,
OUT = {error, not_snapshot},
?assertEqual(OUT, drr_toname(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_type(bitstring())
-> {ok, map(), bitstring()}.
drr_type(<<Type:32/little, Rest/bitstring>>) ->
?debugFmt("drr_type: ~p", [Type]),
{ok, #{ type => type(Type) }, Rest}.
drr_type_test() ->
[ drr_type_test(X) || X <- lists:seq(1,9) ].
drr_type_test(1) ->
IN = <<0,0,0,0 >>,
OUT = {ok, #{ type => drr_begin }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(2) ->
IN = <<1,0,0,0>>,
OUT = {ok, #{ type => drr_object }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(3) ->
IN = <<2,0,0,0>>,
OUT = {ok, #{ type => drr_freeobjects }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(4) ->
IN = <<3,0,0,0>>,
OUT = {ok, #{ type => drr_write}, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(5) ->
IN = <<4,0,0,0>>,
OUT = {ok, #{ type => drr_free }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(6) ->
IN = <<5,0,0,0>>,
OUT = {ok, #{ type => drr_end }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(7) ->
IN = <<6,0,0,0>>,
OUT = {ok, #{ type => drr_write_byref }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(8) ->
IN = <<7,0,0,0>>,
OUT = {ok, #{ type => drr_spill }, <<>>},
?assertEqual(OUT, drr_type(IN));
drr_type_test(9) ->
IN = <<8,0,0,0>>,
OUT = {ok, #{ type => drr_umtypes }, <<>>},
?assertEqual(OUT, drr_type(IN)).
-define(TYPE(X,A), type(X) when is_integer(X) -> A;
type(A) when is_atom(A) -> X ).
?TYPE(0, drr_begin);
?TYPE(1, drr_object);
?TYPE(2, drr_freeobjects);
?TYPE(3, drr_write);
?TYPE(4, drr_free);
?TYPE(5, drr_end);
?TYPE(6, drr_write_byref);
?TYPE(7, drr_spill);
?TYPE(8, drr_umtypes).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_versioninfo(bitstring()) -> {ok, map(), bitstring()}.
drr_versioninfo(<<VersionInfo:64/little, Rest/bitstring>>) ->
?debugFmt("drr_versioninfo: ~p", [<<VersionInfo:64>>]),
<<_:32, Features:30, HdrType:2>> = <<VersionInfo:64>>,
{ok, #{ hdrtype => hdrtype(HdrType)
, featuresflags => Features
}, Rest}.
drr_versioninfo_test() ->
drr_versioninfo_test(1),
drr_versioninfo_test(2).
drr_versioninfo_test(1) ->
IN = <<17,0,0,0,0,0,0,0>>,
OUT = {ok, #{ hdrtype => substream
, featuresflags => 4 }, <<>>},
?assertEqual(OUT, drr_versioninfo(IN));
drr_versioninfo_test(2) ->
IN = <<16,0,0,0,0,0,0,0>>,
OUT = {ok, #{ hdrtype => compoundstream
, featuresflags => 4 }, <<>>}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-define(HDRTYPE(X,A), hdrtype(X) -> A; hdrtype(A) -> X).
-spec hdrtype(integer()) -> atom().
?HDRTYPE(1, substream);
?HDRTYPE(2, compoundstream).
hdrtype_test() ->
[ hdrtype_test(X) || X <- [1,2] ].
hdrtype_test(1) ->
IN = 1,
OUT = substream,
?assertEqual(OUT, hdrtype(1));
hdrtype_test(2) ->
IN = 2,
OUT = compoundstream,
?assertEqual(OUT, hdrtype(2)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%% @see http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h#58
%%--------------------------------------------------------------------
-define(OBJSET_TYPE(X,A), dmu_objset_type(X) -> A;
dmu_objset_type(A) -> X).
?OBJSET_TYPE(0, none);
?OBJSET_TYPE(1, meta);
?OBJSET_TYPE(2, zfs);
?OBJSET_TYPE(3, zvol);
?OBJSET_TYPE(4, other);
?OBJSET_TYPE(5, any);
?OBJSET_TYPE(6, numtypes).
%%
%%
%%
-spec objset_type(bitstring()) -> {ok, atom(), bitstring()}.
objset_type(<<ObjsetType:32/little, Rest/bitstring>>) ->
{ok, #{ type => dmu_objset_type(ObjsetType) }, Rest}.
objset_type_test() ->
[ objset_type_test(X) || X <- lists:seq(1,1) ].
objset_type_test(1) ->
IN = <<0,0,0,0>>,
OUT = {ok, #{ type => none }, <<>>},
?assertEqual(OUT, objset_type(IN));
objset_type_test(2) ->
IN = <<1,0,0,0>>,
OUT = {ok, #{ type => meta }, <<>>},
?assertEqual(OUT, objset_type(IN));
objset_type_test(3) ->
IN = <<2,0,0,0>>,
OUT = {ok, #{ type => zfs}, <<>>},
?assertEqual(OUT, objset_type(IN)).
%%--------------------------------------------------------------------
%%
%%--------------------------------------------------------------------
-define(DMU_BSWAP_UINT8, 0).
-define(DMU_BSWAP_UINT16, 1).
-define(DMU_BSWAP_UINT32, 2).
-define(DMU_BSWAP_UINT64, 3).
-define(DMU_BSWAP_ZAP, 4).
-define(DMU_BSWAP_DNODE, 5).
-define(DMU_BSWAP_OBJSET, 6).
-define(DMU_BSWAP_ZNODE, 7).
-define(DMU_BSWAP_OLDACL, 8).
-define(DMU_BSWAP_ACL, 9).
-define(DMU_BSWAP_NUMFUNCS, 10).

154
src/fletcher.erl Normal file
View File

@@ -0,0 +1,154 @@
%%%===================================================================
%%% @author Mathieu Kerjouan
%%% @copyright 2017
%%% @title Fletcher Checksums Implementation
%%% @doc fletcher checksum naive implementation. All this code
%%% is not optimized and was made only for testing purpose.
%%% @end
%%% @see sys/cddl/contrib/opensolaris/common/zfs/zfs_fletcher.c
%%%===================================================================
-module(fletcher).
-export([fletcher16/1]).
-export([fletcher32/1]).
-export([fletcher64/1]).
-export([adler32/1]).
-include_lib("eunit/include/eunit.hrl").
-define(ENDIANESS, little).
%%--------------------------------------------------------------------
%% @doc naive fletcher16 implementation
%% @end
%%--------------------------------------------------------------------
-spec fletcher16(bitstring()) -> integer().
fletcher16(Bitstring) ->
fletcher16(Bitstring, {0, 0}).
-spec fletcher16(bitstring(), {integer(), integer()}) -> integer().
fletcher16(<<>>, {Sum1, Sum2}) ->
(Sum2 bsl 8) bor Sum1;
fletcher16(<<Char:8, Rest/bitstring>>, {Sum1, Sum2}) ->
S1 = (Sum1 + Char) rem 255,
S2 = (Sum2 + S1) rem 255,
fletcher16(Rest, {S1, S2}).
fletcher16_test() ->
[ fletcher16_test(X) || X <- lists:seq(1,3) ].
fletcher16_test(1) ->
IN = <<"abcde">>,
OUT = 51440,
?assertEqual(OUT, fletcher16(IN));
fletcher16_test(2) ->
IN = <<"abcdef">>,
OUT = 8279,
?assertEqual(OUT, fletcher16(IN));
fletcher16_test(3) ->
IN = <<"abcdefgh">>,
OUT = 1575,
?assertEqual(OUT, fletcher16(IN)).
%%--------------------------------------------------------------------
%% @doc naive fletcher32 implementation
%% @end
%%--------------------------------------------------------------------
fletcher32(Bitstring) ->
fletcher32(Bitstring, {0, 0}).
fletcher32(<<>>, {Sum1, Sum2}) ->
S1 = Sum1 rem 65535,
S2 = Sum2 rem 65535,
(S2 bsl 16) bor S1;
fletcher32(<<Char:16/?ENDIANESS, Rest/bitstring>>, {Sum1, Sum2}) ->
S1 = (Sum1 + Char),
S2 = (Sum2 + S1),
fletcher32(Rest, {S1, S2});
fletcher32(<<Char:8/?ENDIANESS, Rest/bitstring>>, {Sum1, Sum2}) ->
S1 = (Sum1 + Char),
S2 = (Sum2 + S1),
fletcher32(Rest, {S1, S2}).
fletcher32_test() ->
[ fletcher32_test(X) || X <- lists:seq(1,3) ].
fletcher32_test(1) ->
IN = <<"abcde">>,
OUT = 4031760169,
?assertEqual(OUT, fletcher32(IN));
fletcher32_test(2) ->
IN = <<"abcdef">>,
OUT = 1448095018,
?assertEqual(OUT, fletcher32(IN));
fletcher32_test(3) ->
IN = <<"abcdefgh">>,
OUT = 3957429649,
?assertEqual(OUT, fletcher32(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
fletcher64(Bitstring) ->
fletcher64(Bitstring, {0, 0}).
fletcher64(<<>>, {Sum1, Sum2}) ->
S1 = Sum1 rem 4294967295,
S2 = Sum2 rem 4294967295,
(S2 bsl 32) bor S1;
fletcher64(<<Char:32/?ENDIANESS, Rest/bitstring>>, {Sum1, Sum2}) ->
S1 = Sum1 + Char,
S2 = Sum2 + S1,
fletcher64(Rest, {S1, S2});
fletcher64(<<Char:16/?ENDIANESS, Rest/bitstring>>, {Sum1, Sum2}) ->
S1 = Sum1 + Char,
S2 = Sum2 + S1,
fletcher64(Rest, {S1, S2});
fletcher64(<<Char:8/?ENDIANESS, Rest/bitstring>>, {Sum1, Sum2}) ->
S1 = Sum1 + Char,
S2 = Sum2 + S1,
fletcher64(Rest, {S1, S2}).
fletcher64_test() ->
[ fletcher64_test(X) || X <- lists:seq(1,3) ].
fletcher64_test(1) ->
IN = <<"abcde">>,
OUT = 14467467625952928454,
?assertEqual(OUT, fletcher64(IN));
fletcher64_test(2) ->
IN = <<"abcdef">>,
OUT = 14467579776138987718,
?assertEqual(OUT, fletcher64(IN));
fletcher64_test(3) ->
IN = <<"abcdefgh">>,
OUT = 3543817411021686982,
?assertEqual(OUT, fletcher64(IN)).
%%--------------------------------------------------------------------
%% @doc naive adler32 implementation. Erlang is delivered with its
%% own adler implementation, see erlang:adler32/1 function.
%% @end
%%--------------------------------------------------------------------
adler32(Bitstring) ->
adler32(Bitstring, {1, 0}).
adler32(<<>>, {C1, C2}) ->
(C2 bsl 16) bor C1;
adler32(<<Char:8, Rest/bitstring>>, {C1, C2}) ->
S1 = C1 + Char rem 65521,
S2 = C2 + S1 rem 65521,
adler32(Rest, {S1, S2}).
adler32_test() ->
[ adler32_test(X) || X <- lists:seq(1,3) ].
adler32_test(1) ->
IN = <<"abcde">>,
OUT = 16#05C801F0,
?assertEqual(OUT, adler32(IN));
adler32_test(2) ->
IN = <<"abcdef">>,
OUT = 16#081E0256,
?assertEqual(OUT, adler32(IN));
adler32_test(3) ->
IN = <<"abcdefgh">>,
OUT = 16#0E000325,
?assertEqual(OUT, adler32(IN)).

14
src/zfs.app.src Normal file
View File

@@ -0,0 +1,14 @@
{application, zfs,
[{description, "Erlang ZFS Implementation"},
{vsn, "0.1.0"},
{registered, []},
{applications,
[kernel,
stdlib
]},
{env,[]},
{modules, []},
{maintainers, ["contact@steepath.eu"]},
{licenses, ["ISC/OpenBSD"]},
{links, []}
]}.

512
src/zfs.erl Normal file
View File

@@ -0,0 +1,512 @@
%%%-------------------------------------------------------------------
%%% @title Erlang ZFS High Level Implementation
%%% @author Mathieu Kerjouan
%%% @copyright 2017
%%% @version 0.1.0
%%% @doc This module is the first step of Erlang ZFS Implementation
%%% and contain all required functions for parsing ZFS streams.
%%% @end
%%% @see http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h
%%%-------------------------------------------------------------------
-module(zfs).
-compile(export_all).
-include("zfs.hrl").
-include_lib("eunit/include/eunit.hrl").
%%--------------------------------------------------------------------
%% @doc clean function remove null char from bitstring.
%% @end
%%--------------------------------------------------------------------
clean(Bitstring) ->
clean(Bitstring, <<>>).
clean(<<A, Rest/bitstring>>, Buf)
when A =/= 0 ->
clean(Rest, <<Buf/bitstring, A>>);
clean(_, Buf) ->
Buf.
%%--------------------------------------------------------------------
%% @doc generic automaton handler function, take a list of action in
%% multiple form. Each action should take at least one argument
%% wich is next action to do after executing the first one.
%% @end
%%--------------------------------------------------------------------
-spec next(list(), list()) -> {ok, list()} | term().
next([], Args) ->
{ok, Args};
next([Function|T], Args)
when is_atom(Function) ->
?debugFmt("Function: ~p", [Function]),
?debugFmt("Rest: ~p", [T]),
?debugFmt("Args: ~p", [Args]),
erlang:apply(?MODULE, Function, [T|Args]);
next([Function|T], Args)
when is_function(Function) ->
?debugFmt("Function: ~p", [Function]),
?debugFmt("Rest: ~p", [T]),
?debugFmt("Args: ~p", [Args]),
erlang:apply(Function, [T|Args]);
next([{Module, Function}|T], Args)
when is_atom(Module), is_atom(Function) ->
?debugFmt("Module: ~p", [Module]),
?debugFmt("Function: ~p", [Function]),
?debugFmt("Rest: ~p", [T]),
?debugFmt("Args: ~p", [Args]),
erlang:apply(Module, Function, [T|Args]).
%%--------------------------------------------------------------------
%% @doc simple function to put value in different structure like
%% map and proplist.
%% @end
%%--------------------------------------------------------------------
put(Key, Value, Proplist)
when is_list(Proplist) ->
[{Key, Value}|Proplist];
put(Key, Value, Map)
when is_map(Map) ->
maps:put(Key, Value, Map).
%%--------------------------------------------------------------------
%% @doc main function to parse zfs stream. This function accept list
%% or bitstring.
%% @end
%%--------------------------------------------------------------------
-spec dmu(bitstring() | list()) -> tuple().
dmu(List)
when is_list(List) ->
dmu(erlang:list_to_bitstring(List));
dmu(Bitstring)
when is_bitstring(Bitstring) ->
drr_header(Bitstring, #{}).
%%--------------------------------------------------------------------
%% @doc drr_end/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_end() -> [atom(), ...].
drr_end() ->
[ drr_checksum, drr_toguid, return ].
%%--------------------------------------------------------------------
%% @doc drr_object/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_object() -> [atom(), ...].
drr_object() ->
[ drr_object, drr_type, drr_bonustype, drr_blksz
, drr_bonuslen, drr_checksumtype, drr_compress
, drr_pad, drr_toguid, return ].
%%--------------------------------------------------------------------
%% @doc drr_freeobjects/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_freeobjects() -> [atom(), ...].
drr_freeobjects() ->
[ drr_firstobj, drr_numobjs, drr_toguid, return ].
%%--------------------------------------------------------------------
%% @doc drr_write/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_write() -> [atom(), ...].
drr_write() ->
[ drr_object, drr_type, drr_pad, drr_offset, drr_length
, drr_toguid, drr_checksumtype, drr_checksumflags
, drr_pad2, drr_key, return ].
%%--------------------------------------------------------------------
%% @doc drr_free/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_free() -> [atom(), ...].
drr_free() ->
[ drr_object, drr_offset, drr_length, drr_toguid ].
%%--------------------------------------------------------------------
%% @doc drr_write_byref/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_write_byref() -> [atom(), ...].
drr_write_byref() ->
[ drr_object, drr_offset, drr_length, drr_toguid
, drr_refguid, drr_refobject, drr_refoffset
, drr_checksumtype, drr_checksumflags, drr_pad2
, drr_key, return ].
%%--------------------------------------------------------------------
%% @doc drr_spill/0.
%% @end
%%--------------------------------------------------------------------
-spec drr_spill() -> [atom(), ...].
drr_spill() ->
[ drr_object, drr_length, drr_toguid, drr_pad, return ].
%%--------------------------------------------------------------------
%% @doc drr_header function will route type based on first 64bits of
%% ZFS stream from send function. First 32bits define ZFS DRR,
%% following 32bits define DRR payload length.
%% Objects has defined size: 312bytes. This is the size of
%% the dmu_replay_record structure from zfs_ioctl header. So,
%% we can retrieve all information with recursive call, but
%% if our structure isn't well sized, we'll need to ensure
%% to remove remaining data. Another method, more imperative,
%% is to extract directly 312bytes from current stream and
%% parse it after. Current implementation use pattern
%% matching method.
%% @end
%%--------------------------------------------------------------------
drr_header(<<DrrType:32/little, DrrPayloadLen:32, Rest/bitstring>>, DmuHeader) ->
?debugFmt("type: ~p", [DrrType]),
?debugFmt("payload: ~p", [DrrPayloadLen]),
DmuUpdate1 = maps:put(type, DrrType, DmuHeader),
DmuUpdate2 = maps:put(payloadlen, DrrPayloadLen, DmuUpdate1),
case DrrType of
?DRR_BEGIN ->
drr_begin(Rest, drr_begin(), #{}, DmuUpdate2);
?DRR_OBJECT ->
wip; % drr_object(Rest, drr_object(), #{}, DmuHeader);
?DRR_FREEOBJECTS ->
drr_freeobjects(Rest, drr_freeobjects(), #{});
?DRR_WRITE ->
wip; % drr_write(Rest, drr_write(), #{}, DmuHeader);
?DRR_FREE ->
wip; % drr_free(Rest, drr_free(), #{}, DmuHeader);
?DRR_END ->
wip; % drr_end(Rest, drr_end(), #{}, DmuHeader);
?DRR_WRITE_BYREF ->
wip; % drr_write_byref(Rest, drr_write_byref(), #{}, DmuHeader);
?DRR_SPILL ->
wip; % drr_spill(Rest, drr_spill(), #{}, DmuHeader);
?DRR_NUMTYPES ->
wip; % drr_numtypes(Rest, drr_numtypes(), #{}, DmuHeader)
_ ->
{error, "unsupported type"}
end.
%%--------------------------------------------------------------------
%% @doc drr_begin/0 function return a list of all function to parse
%% a begin ZFS stream.
%% @end
%%--------------------------------------------------------------------
-spec drr_begin() -> [atom(), ...].
drr_begin() ->
[ drr_magic, drr_versioninfo, drr_creation_time, drr_type
, drr_flags, drr_toguid, drr_fromguid, drr_toname, return ].
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
drr_begin(Bitstring, Order, Buf, DmuHeader) ->
?debugMsg("drr_begin function"),
next(Order, [Bitstring, Buf, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc drr_magic function extract magic ZFS value following and
%% check if this value is well configured.
%% @end
%%--------------------------------------------------------------------
-spec drr_magic(bitstring()) -> {ok, list(), bitstring()}.
drr_magic(<<Magic:64/little, Rest/bitstring>>) ->
?debugFmt("magic: ~p", [Magic]),
M = erlang:integer_to_binary(Magic, 16),
case M =:= <<"2F5BACBAC">> of
true ->
?debugFmt("magic: ~p", [M]),
{ok, M, Rest}
end.
-spec drr_magic(list(), bitstring(), map(), map()) -> tuple().
drr_magic(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, Magic, Rest} = drr_magic(Bitstring),
Return = maps:put(magic, Magic, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc drr_versioninfo extract a 64bits from current bitstring and
%% split it in 2 value: hdrtype and features flags.
%% @end
%%--------------------------------------------------------------------
-spec drr_versioninfo(bitstring) -> {ok, bitstring(), bitstring()}.
drr_versioninfo(<<VersionInfo:64/bitstring, Rest/bitstring>>) ->
?debugFmt("versioninfo: ~p", [VersionInfo]),
{ok, VersionInfo, Rest}.
-spec drr_versioninfo(list(), bitstring(), map(), map()) -> tuple().
drr_versioninfo(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
% we need to extract feature and hdrtype
% but this thing currently doesn't work as expected...
% <<Features:6/little, HdrType:2, _/bitstring>> = VersionInfo,
% <<HdrType:16, Features/bitstring>> = VersionInfo,
% ?debugFmt("feature: ~p:~p", [HdrType, Features]),
% Return1 = maps:put(features, Features, Buf),
% Return2 = maps:put(hdrtype, undef, Return1),
{ok, VersionInfo, Rest} = drr_versioninfo(Bitstring),
Return = maps:put(versioninfo, VersionInfo, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc drr_creation_time extract a 64bits value from current
%% bitstring.
%% @end
%%--------------------------------------------------------------------
-spec drr_creation_time(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_creation_time(<<CreationTime:64/bitstring, Rest/bitstring>>) ->
?debugFmt("creation_time: ~p", [CreationTime]),
{ok, CreationTime, Rest}.
-spec drr_creation_time(list(), bitstring(), map(), map()) -> tuple().
drr_creation_time(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, CreationTime, Rest} = drr_creation_time(Bitstring),
Return = maps:put(creation_time, CreationTime, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc drr_type extract 32bits value from current bitstring.
%% @end
%%--------------------------------------------------------------------
-spec drr_type(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_type(<<Type:32/bitstring, Rest/bitstring>>) ->
?debugFmt("type: ~p", [Type]),
{ok, Type, Rest}.
-spec drr_type(list(), bitstring(), map(), map()) -> tuple().
drr_type(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, Type, Rest} = drr_type(Bitstring),
Return = maps:put(type, Type, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc drr_flags function extract flags from current bitstring.
%% @end
%%--------------------------------------------------------------------
-spec drr_flags(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_flags(<<Flags:32/bitstring, Rest/bitstring>>) ->
?debugFmt("flags: ~p", [Flags]),
{ok, Flags, Rest}.
-spec drr_flags(list(), bitstring(), map(), map()) -> tuple().
drr_flags(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, Flags, Rest} = drr_flags(Bitstring),
Return = maps:put(flags, Flags, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc drr_toguid extract 64bits as bitstring from current
%% bitstring.
%% @end
%%--------------------------------------------------------------------
-spec drr_toguid(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_toguid(<<ToGuid:64/bitstring, Rest/bitstring>>) ->
?debugFmt("toguid: ~p", [ToGuid]),
{ok, ToGuid, Rest}.
drr_toguid(Order, Bitstring, Buf)
when is_bitstring(Bitstring) ->
{ok, ToGuid, Rest} = drr_toguid(Bitstring),
Return = maps:put(toguid, ToGuid, Buf),
next(Order, [Rest, Return]).
-spec drr_toguid(list(), bitstring(), map(), map()) -> tuple().
drr_toguid(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, ToGuid, Rest} = drr_toguid(Bitstring),
Return = maps:put(toguid, ToGuid, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_fromguid(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_fromguid(<<FromGuid:64/bitstring, Rest/bitstring>>) ->
?debugFmt("fromguid: ~p", [FromGuid]),
{ok, FromGuid, Rest}.
drr_fromguid(Order,Bitstring, Buf)
when is_bitstring(Bitstring) ->
{ok, FromGuid, Rest} = drr_fromguid(Bitstring),
Return = maps:put(fromguid, FromGuid, Buf),
next(Order, [Rest, Return]).
-spec drr_fromguid(list(), bitstring(), map(), map()) -> tuple().
drr_fromguid(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, FromGuid, Rest} = drr_fromguid(Bitstring),
Return = maps:put(fromguid, FromGuid, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_toname(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_toname(<<ToName:(256*8)/bitstring, Rest/bitstring>>) ->
?debugFmt("toname: ~p", [clean(ToName)]),
{ok, ToName, Rest}.
-spec drr_toname(list(), bitstring(), map(), map()) -> tuple().
drr_toname(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, ToName, Rest} = drr_toname(Bitstring),
Return = maps:put(toname, clean(ToName), Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
return([], Rest, Buf ) ->
{ok, Buf, Rest}.
return([], Rest, Buf, DmuHeader) ->
{ok, maps:put(drr, Buf, DmuHeader), Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
drr_end(Bitstring, Order, DmuHeader) ->
next(Order, [Bitstring, #{}, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_checksum(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_checksum(<<Checksum:256/bitstring, Rest/bitstring>>) ->
?debugFmt("Checksum: ~p", [Checksum]),
{ok, Checksum, Rest}.
-spec drr_checksum(list(), bitstring(), map(), map()) -> tuple().
drr_checksum(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, Checksum, Rest} = drr_checksum(Bitstring),
Return = maps:put(checksum, Checksum, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_object(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_object(<<Object:64, Rest/bitstring>>) ->
?debugFmt("Object: ~p", [Object]),
{ok, Object, Rest}.
-spec drr_object(list(), bitstring(), map(), map()) -> tuple().
drr_object(Order, Bitstring, Buf, DmuHeader)
when is_bitstring(Bitstring) ->
{ok, Object, Rest} = drr_object(Bitstring),
Return = maps:put(object, Object, Buf),
next(Order, [Rest, Return, DmuHeader]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_bonustype(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_bonustype(<<BonusType:64/bitstring, Rest/bitstring>>) ->
{ok, BonusType, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_blksz(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_blksz(<<BlkSz:32, Rest/bitstring>>) ->
{ok, BlkSz, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_bonuslen(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_bonuslen(<<BonusLen:32/bitstring, Rest/bitstring>>) ->
{ok, BonusLen, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_checksumtype(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_checksumtype(<<ChecksumType:8, Rest/bitstring>>) ->
{ok, ChecksumType, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_compress(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_compress(<<Compress:8, Rest/bitstring>>) ->
{ok, Compress, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_pad(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_pad(<<Pad:8, Rest>>) ->
{ok, Pad, Rest}.
-spec drr_pad(bitstring(), non_neg_integer())
-> {ok, bitstring(), bitstring()}.
drr_pad(Bitstring, Length) ->
L = Length*8,
<<Pad:L, Rest/bitstring>> = Bitstring,
{ok, Pad, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
drr_freeobjects(Bitstring, Order, Buf) ->
?debugMsg("drr_freeobjects function"),
next(Order, [Bitstring, Buf]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_firstobj(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_firstobj(<<FirstObj:64/little, Rest/bitstring>>) ->
{ok, FirstObj, Rest}.
drr_firstobj(Order, Bitstring, Buf) ->
{ok, FirstObj, Rest} = drr_firstobj(Bitstring),
Return = maps:put(firstobj, FirstObj, Buf),
next(Order, [Rest, Return]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_numobjs(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_numobjs(<<NumObjs:64/little, Rest/bitstring>>) ->
{ok, NumObjs, Rest}.
drr_numobjs(Order, Bitstring, Buf) ->
{ok, NumObjs, Rest} = drr_numobjs(Bitstring),
Return = maps:put(numobjs, NumObjs, Buf),
next(Order, [Rest, Return]).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_offset(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_offset(<<Offset:64/bitstring, Rest/bitstring>>) ->
{ok, Offset, Rest}.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec drr_length(bitstring()) -> {ok, bitstring(), bitstring()}.
drr_length(<<Length:64/bitstring, Rest/bitstring>>) ->
{ok, Length, Rest}.

63
src/zfs.hrl Normal file
View File

@@ -0,0 +1,63 @@
%%%-------------------------------------------------------------------
%%%
%%%-------------------------------------------------------------------
-define(MAXNAMELEN, 256).
-define(DRR_BEGIN, 0).
-define(DRR_OBJECT, 1).
-define(DRR_FREEOBJECTS, 2).
-define(DRR_WRITE, 3).
-define(DRR_FREE, 4).
-define(DRR_END, 5).
-define(DRR_WRITE_BYREF, 6).
-define(DRR_SPILL, 7).
-define(DRR_NUMTYPES, 8).
% drr_begin
% http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h#128
-define(DRR_MAGIC_SIZE, 64).
-define(DRR_MAGIC, ?DRR_MAGIC_SIZE/bitstring).
-define(DRR_VERSIONINFO_SIZE, 64).
-define(DRR_VERSIONINFO, ?DRR_VERSIONINFO_SIZE/bitstring).
-define(DRR_CREATIONTIME_SIZE, 64).
-define(DRR_CREATIONTIME, ?DRR_CREATIONTIME_SIZE/bitstring).
-define(DRR_TYPE_SIZE, 32).
-define(DRR_TYPE, ?DRR_TYPE_SIZE/bitstring).
-define(DRR_FLAGS_SIZE, 32).
-define(DRR_FLAGS, ?DRR_FLAGS_SIZE/bitstring).
-define(DRR_TOGUID_SIZE, 64).
-define(DRR_TOGUID, ?DRR_TOGUID_SIZE/bitstring).
-define(DRR_FROMGUID_SIZE, 64).
-define(DRR_FROMGUID, ?DRR_FROMGUID_SIZE/bitstring).
-define(DRR_TONAME_SIZE, ?MAXNAMELEN).
-define(DRR_TONAME, ?MAXNAMELEN/bitstring).
% drr_end
% http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h#139
-define(DRR_CHECKSUM_SIZE, 64).
% drr_object
% http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h#144
-define(DRR_OBJECT_SIZE, 64).
-define(DRR_BLKSZ_SIZE, 32).
-define(DRR_BONUSLEN_SIZE, 32).
-define(DRR_CHECKSUMTYPE_SIZE, 8).
-define(DRR_COMPRESS_SIZE, 8).
-define(DRR_PAD_SIZE, 8).
% drr_freeobjects
% http://bxr.su/FreeBSD/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_ioctl.h#157
% drr_write
% drr_free
% drr_write_byref

298
src/zfs_lib.erl Normal file
View File

@@ -0,0 +1,298 @@
%%%===================================================================
%%% @doc
%%% @end
%%% @see sys/cddl/contrib/opensolaris/common/zfs
%%%===================================================================
-module(zfs_lib).
-include("zfs.hrl").
-include_lib("eunit/include/eunit.hrl").
-compile(export_all).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec valid_char(integer()) -> integer().
valid_char(Char)
when (Char >= $0 andalso Char =< $9) orelse
(Char >= $a andalso Char =< $z) orelse
(Char >= $A andalso Char =< $Z) orelse
Char =:= $_ orelse Char =:= $- orelse
Char =:= $: orelse Char =:= $. orelse Char =:= $ ->
Char.
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec zfs_name(list() | bitstring())
-> {ok, atom(), bitstring()} |
{error, term()}.
zfs_name(List) when is_list(List) ->
zfs_name(erlang:list_to_bitstring(List));
zfs_name(Bitstring) ->
zfs_name(Bitstring, <<>>).
-spec zfs_name(bitstring(), bitstring(), atom())
-> {ok, atom(), bitstring()} |
{error, term()}.
zfs_name(Bitstring, Buf) ->
zfs_name(Bitstring, <<>>, pool).
% pool
zfs_name(<<>>, PoolName, pool) ->
{ok, pool, PoolName};
zfs_name(<<"/", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"@", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"_", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"c", Char:8, Rest/bitstring>>, <<>>, pool)
when Char >= $0 andalso Char =< $9 ->
{error, badname};
zfs_name(<<"log", Rest/bitstring>>, Buf, pool) ->
{error, badname};
zfs_name(<<"mirror", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"raidz", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"raidz1", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"raidz2", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"raidz3", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"spare", Rest/bitstring>>, <<>>, pool) ->
{error, badname};
zfs_name(<<"@", Rest/bitstring>>, Buf, pool) ->
{error, badname};
zfs_name(<<"%", Rest/bitstring>>, Buf, pool) ->
{error, badname};
zfs_name(<<"/", Rest/bitstring>>, Buf, pool) ->
zfs_name(Rest, <<Buf/bitstring, "/">>, dataset);
zfs_name(<<Char:8, Rest/bitstring>>, Buf, pool)
when (Char >= $0 andalso Char =< $9) orelse
(Char >= $a andalso Char =< $z) orelse
(Char >= $A andalso Char =< $Z) orelse
Char =:= $_ orelse Char =:= $- orelse
Char =:= $: orelse Char =:= $. ->
zfs_name(Rest, <<Buf/bitstring, Char>>, pool);
% dataset
zfs_name(<<>>, DatasetName, dataset) ->
{ok, dataset, DatasetName};
zfs_name(<<"%", Rest/bitstring>>, Buf, dataset) ->
{error, badname};
zfs_name(<<"@", Rest/bitstring>>, Buf, dataset) ->
zfs_name(Rest, <<Buf/bitstring, "@">>, snapshot);
zfs_name(<<Char:8, Rest/bitstring>>, Buf, dataset)
when (Char >= $0 andalso Char =< $9) orelse
(Char >= $a andalso Char =< $z) orelse
(Char >= $A andalso Char =< $Z) orelse
Char =:= $_ orelse Char =:= $- orelse
Char =:= $: orelse Char =:= $. orelse
Char =:= $/ ->
zfs_name(Rest, <<Buf/bitstring, Char>>, dataset);
% snapshot
zfs_name(<<>>, Buf, snapshot) ->
{ok, snapshot, Buf};
zfs_name(<<Char:8, Rest/bitstring>>, Buf, snapshot)
when (Char >= $0 andalso Char =< $9) orelse
(Char >= $a andalso Char =< $z) orelse
(Char >= $A andalso Char =< $Z) orelse
Char =:= $_ orelse Char =:= $- orelse
Char =:= $: orelse Char =:= $. orelse
Char =:= $/ ->
zfs_name(Rest, <<Buf/bitstring, Char>>, snapshot);
zfs_name(_, _, _) ->
{error, badname}.
zfs_name_test() ->
[ zfs_name_test(X) || X <- lists:seq(1,14) ].
zfs_name_test(1) ->
IN = <<"pool">>,
OUT = {ok, pool, IN},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(2) ->
IN = <<"pool/test/toto">>,
OUT = {ok, dataset, IN},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(3) ->
IN = <<"pool/test/toto@snap1">>,
OUT = {ok, snapshot, IN},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(4) ->
IN = <<"/pool/test">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(5) ->
IN = <<"@pool/test">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(6) ->
IN = <<"pool@test@1">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(7) ->
IN = <<"_pool/test">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(8) ->
IN = <<"%OFDATA">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(9) ->
IN = <<"ROOT">>,
OUT = {ok, pool, IN},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(10) ->
IN = <<"R_:0/level4-4">>,
OUT = {ok, dataset, IN},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(11) ->
IN = <<" test">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN));
zfs_name_test(12) ->
IN = [ <<"c012test">>, <<"c1">>, <<"c2">>
, <<"c3">>, <<"c4">>, <<"c5">>, <<"c6">>
, <<"c7">>, <<"c8">>, <<"c9">>
],
OUT = {error, badname},
[ ?assertEqual(OUT, zfs_name(X)) || X <- IN ];
zfs_name_test(13) ->
IN = [ <<"log">>, <<"mirror">>, <<"raidz">>
, <<"raidz1">>, <<"raidz2">>, <<"raidz3">>
, <<"spare">>
],
OUT = {error, badname},
[ ?assertEqual(OUT, zfs_name(X)) || X <- IN ];
zfs_name_test(14) ->
IN = <<"mypool/test%/1">>,
OUT = {error, badname},
?assertEqual(OUT, zfs_name(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec is_pool(bitstring()) -> true | false.
is_pool(Bitstring) ->
is_pool(Bitstring, <<>>).
-spec is_pool(bitstring(), bitstring()) -> true | false.
is_pool(<<>>, Buf)
when Buf =/= <<>> ->
true;
is_pool(<<Char:8, Rest/bitstring>>, Buf)
when Char >= $0 andalso Char =< $9 orelse
Char >= $A andalso Char =< $Z orelse
Char >= $a andalso Char =< $z ->
is_pool(Rest, <<Buf/bitstring, Char>>);
is_pool(_, _) ->
false.
is_pool_test() ->
[ is_pool_test(X) || X <- [1] ].
is_pool_test(1) ->
IN = <<"test">>,
OUT = true,
?assertEqual(OUT, is_pool(IN));
is_pool_test(2) ->
IN = <<"/test">>,
OUT = false,
?assertEqual(OUT, is_pool(IN));
is_pool_test(3) ->
IN = <<"@test">>,
OUT = false,
?assertEqual(OUT, is_pool(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec is_dataset(bitstring()) -> true | false.
is_dataset(Bitstring) ->
is_dataset(Bitstring, <<>>).
-spec is_dataset(bitstring(), bitstring()) -> true | false.
is_dataset(<<$/, _/bitstring>>, <<>>) ->
false;
is_dataset(<<>>, Buf)
when Buf =/= <<>> ->
true;
is_dataset(<<$/, $/, Rest/bitstring>>, _) ->
false;
is_dataset(<<Char:8, Rest/bitstring>>, Buf)
when Char >= $0 andalso Char =< $9 orelse
Char >= $A andalso Char =< $Z orelse
Char >= $a andalso Char =< $z orelse
Char =:= $/ ->
is_dataset(Rest, <<Buf/bitstring, Char>>);
is_dataset(_, _) ->
false.
is_dataset_test() ->
[ is_dataset_test(X) || X <- [1] ].
is_dataset_test(1) ->
IN = <<"test/test">>,
OUT = true,
?assertEqual(OUT, is_dataset(IN));
is_dataset_test(2) ->
IN = <<"/test/test">>,
OUT = false,
?assertEqual(OUT, is_dataset(IN));
is_dataset_test(3) ->
IN = <<"test//test">>,
OUT = false,
?assertEqual(OUT, is_dataset(IN)).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
-spec is_snapshot(bitstring()) -> true | false.
is_snapshot(Bitstring) ->
is_snapshot(Bitstring, <<>>).
-spec is_snapshot(bitstring(), bitstring()) -> true | false.
is_snapshot(<<>>, Buf)
when Buf =/= <<>> ->
true;
is_snapshot(<<Char:8, Rest/bitstring>>, Buf) ->
ok.
is_snapshot_test() ->
[ is_snapshot_test(X) || X <- [1] ].
is_snapshot_test(1) ->
?assertEqual(ok, ok).
%%--------------------------------------------------------------------
%% @doc
%% @end
%%--------------------------------------------------------------------
null_clean(Bitstring) ->
null_clean(Bitstring, <<>>).
null_clean(<<A, Rest/bitstring>>, Buf)
when A =/= 0 ->
null_clean(Rest, <<Buf/bitstring, A>>);
null_clean(_, Buf) ->
Buf.
null_clean_test() ->
[ null_clean_test(X) || X <- lists:seq(1,2) ].
null_clean_test(1) ->
IN = <<"thisisatest",0,0,0,0>>,
OUT = <<"thisisatest">>,
?assertEqual(OUT, null_clean(IN));
null_clean_test(2) ->
IN = <<"thisis",0,"atest">>,
OUT = <<"thisis">>,
?assertEqual(OUT, null_clean(IN)).

1
src/zpool.erl Normal file
View File

@@ -0,0 +1 @@
-module(zpool).

0
src/zpool.hrl Normal file
View File

1
src/zpool_test.erl Normal file
View File

@@ -0,0 +1 @@
-module(zpool_test).

1
test/README.md Normal file
View File

@@ -0,0 +1 @@
# Test

BIN
test/pool@test.zfs Normal file
View File

Binary file not shown.

2454
test/zstreamdump.return Normal file
View File

File diff suppressed because it is too large Load Diff