Windows Samba shadow copy & pyznap - can it be done?

[recklessnl]

Hi, I'm looking at pyznap and it looks solid. One thing that I would love to have as a Windows user though is to enable shadow copies with ZFS. Which basically allows any Samba share file and directory to restore to Previous Versions using a super simple GUI. This is all in Windows - and would be supported by the ZFS snapshot mechanism (and thus, pyznap).

See: https://blog.chaospixel.com/linux/2017/09/zfs-auto-snapshots-and-windows-shadow-copies-with-samba.html

Also this thread on the same issue with zfs-auto-snapshot: zfsonlinux/zfs-auto-snapshot#84

Is this possible with pyznap as well? If so, how would you do it? And if not, would you be able to add code in order to make this work? Would be really great.

[yboetz]

I'm not really familiar with samba and shadow copies. But would it not be possible to use regex to match the snapshot names? Something like

snapprefix = ^pyznap_
format = %Y-%m-%d_%H:%M:%S_\(frequent\)\{0,1\}\(hourly\)\{0,1\}\(daily\)\{0,1\}\(weekly\)\{0,1\}\(monthly\)\{0,1\}\(yearly\)\{0,1\}

as user @benpye suggested.

Rewriting pyznap to have custom labels will take a bit more work and I would like to avoid that :).

[yboetz]

Did this work? Or do you have any new info? Else I will close this.

[recklessnl]

I can't test it yet so I'm not sure. If you'd like to close it go ahead, I'll try to test this later when I can, I'm sure it should work as you described.

[DeViLRuNNeR-dev]

@yboetz

Can you reopen this issue please?
I was using ZnapZend before finding this awesome tool and there it was working.
I have tried changing the shadow:snapprefix and shadow:format serveral times but it does not work !

I believe windows shadow copies don't like that U are using ':' in the snapshot names.
below is what I believe is the correct RegEx for the shadow:format ;

shadow:format = pyznap_%Y-%m-%d_%H:%M:%S_([^_]*)$

Is there any way we can easily change the snapshot naming format ourself?
Perhaps somewhere in the .py files ?

[yboetz]

No, atm there is no way to change the snapshot naming. There is a PR (#39) to allow this, though atm I'm not comfortable to merge this, as there are some problems and no comprehensive tests. You could try to run that version locally.

[DeViLRuNNeR-dev]

@yboetz

Thank you for your quick response !
To bad this is not possible because your script is just perfect!
So much better then what I was using (ZnapZend), if only the shadow coppies could work on windows to without the ':' in snapshot names.

Could you guide me on how I can get the commit from @chazapis running on my system?
Is it just copying his files on top of the existing ones in the folder (/usr/local/lib/python3.7/dist-packages/pyznap/) ?

[yboetz]

Just copying over the files might break something, as there is a merge conflict between the most recent pyznap version and that PR. You can do the following:

$ git clone -b datetime-formatting [email protected]:chazapis/pyznap.git /tmp/pyznap
$ pip install /tmp/pyznap/

This will download his branch to your /tmp/pyznap/ folder and then you can install from there with pip.

[DeViLRuNNeR-dev]

I have done some testing myself and it turns out we can't use Windows Shadow Copies on Samba due to the naming of these snapshots. There is not option in SAMBA to catch the snapshottype (hourly, frequent) on the end of the snapshot name.

Does anybody know how we can change the code to put the snapshottype in front of the data/time?

[yboetz]

This here

snapprefix = ^pyznap_
format = %Y-%m-%d_%H:%M:%S_\(frequent\)\{0,1\}\(hourly\)\{0,1\}\(daily\)\{0,1\}\(weekly\)\{0,1\}\(monthly\)\{0,1\}\(yearly\)\{0,1\}

does not work?

Changing the code to put the type in front of date&time is a major change and might not be so easy. pyznap relies on being able to match the snapshots with pyznap_{date}_{time}_{type}.

[chazapis]

@DeViLRuNNeR-dev take a look at the options of the shadow copy Samba module ( https://www.samba.org/samba/docs/current/man-html/vfs_shadow_copy2.8.html).

You can change the format Samba expects, to what pyznap provides, with shadow:format = pyznap_%Y%m%d_%H%M%S_hourly, or something similar.

[DeViLRuNNeR-dev]

I have tried several things and whatever I do I cant' get it to work with a variable like _daily or _frequent appended @ the end.
If I append it after the prefix like so;
'pyznap_daily-2019-01-01-00-00-00' it works
but samba can't seem to handle these suffix in the end like so;
'pyznap_2019-01-01-00-00-00_daily' it does not works

Does anybody have it working with the snapshot type in the end?

[DeViLRuNNeR-dev]

Just a little update !
I have managed to get it working with the main source; meaning with snapshots in the format ;

pyznap_2019-01-01_01:01:01_frequent

What I have now in my samba.conf shadow: format = _%Y-%m-%d_%H:%M:%S shadow: snapprefix = ^pyznap shadow: delimiter = _ shadow:localtime = no
Thank you everybody for the help !

[recklessnl]

I couldn't get it to work either - I'm going to try your new code and see if that works. Did you have any other lines regarding shadow copy in your conf file as well?

[DeViLRuNNeR-dev]

My full smb.conf

`
[global]
## Browsing/Identification ###
# Change this to the workgroup/NT-domain name your Samba server will part of
workgroup = WORKGROUP
server string = PVE
netbios name = PVE
#disable netbios = yes
smb ports = 445
security = user
encrypt passwords = true
map to guest = bad user
guest account = nobody
vfs objects = shadow_copy2
vfs objects = acl_xattr
map acl inherit = yes
store dos attributes = yes
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
# Specify snapshot name: frequent, hourly, daily... as desired
shadow: format = %Y-%m-%d%H:%M:%S
shadow: snapprefix = ^pyznap
shadow: delimiter = _
shadow:localtime = no
#veto files = /.zfs/
usershare path =

[homes]
comment = Home Directories
browseable = no
writable = no
read only = yes

[DATA]
comment = DATA share (ZFS_Pool_RAID0_Z1-HDDs/DATA)
path = /ZFS_Pool_RAID0_Z1-HDDs/DATA
browseable = yes
writable = yes
read only = no
printable = no
guest ok = no
public = no
vfs objects = shadow_copy2
inherit permissions = yes
inherit acls = yes
`
And also changes to my ZFS settings;

apt-get install samba-vfs-modules zfs set acltype=posixacl ZFS_Pool_RAID0_Z1-HDDs/DATA zfs set aclinherit=passthrough ZFS_Pool_RAID0_Z1-HDDs/DATA zfs set xattr=sa ZFS_Pool_RAID0_Z1-HDDs/DATA

Hope is helps you getting it working !

[recklessnl]

@DeViLRuNNeR-dev not sure I fully understand - does this work with one specific snapshot name or all of them?

This still only works with ONLY frequent, or ONLY daily snapshots right? Or are you saying it works with all of them?

[DeViLRuNNeR-dev]

It works with all of them !

[recklessnl]

EDIT: You full smb.conf has an error, it should be this line for the prefix format:
shadow: format = _%Y-%m-%d_%H:%M:%S

Which is what you said earlier. I can confirm it works!

@DeViLRuNNeR-dev why do you not allow for local time? Wouldn't it be easier for users to see the actual local times of the files they are trying to restore?

[DeViLRuNNeR-dev]

@recklessnl Great you got it working to !
I wanted to use local times but for some reason when I set that to yes it doesn't work anymore.
Can you share if that works on your system?

Maybe it has something todo with my release of samba on debian ?
smbstatus --version = Version 4.9.5-Debian
Operating System: Debian GNU/Linux 10 (buster)
Kernel: Linux 5.0.21-2-pve
Architecture: x86-64

[florian-obradovic]

Just a little update !
I have managed to get it working with the main source; meaning with snapshots in the format ;

pyznap_2019-01-01_01:01:01_frequent

What I have now in my samba.conf shadow: format = _%Y-%m-%d_%H:%M:%S shadow: snapprefix = ^pyznap shadow: delimiter = _ shadow:localtime = no
Thank you everybody for the help !

Thx mate, that worked great (Ubuntu 20.04)
My snapshots look like: "tank/lxd/images@pyznap_2020-12-28_00:00:01_weekly"

smb.conf:

vfs objects = catia fruit streams_xattr shadow_copy2
shadow: snapdir = .zfs/snapshot
shadow: sort = desc
shadow: format = _%Y-%m-%d_%H:%M:%S
shadow: snapprefix = ^pyznap
shadow: delimiter = _
shadow:localtime = no