7MTT CIFS Question

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

7MTT CIFS Question

mrlizard23

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

basilberntsen
Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 


On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

Klise, Steve-2

If you use robocopy I used the following switches:

 

/MIR  /e /copyall /r:0 /w:0

 

/mir is a dangerous one; make sure you understand it.. it might make your day go bad.  Copyall copies perms PLUS ownership.  Worked like a charm (at least for me back in the day).

 

 

From: <[hidden email]> on behalf of Basil <[hidden email]>
Date: Tuesday, April 11, 2017 at 1:44 PM
To: Chris Hague <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

Tim McCarthy
robocopy  /E /COPYALL /MT:8 /R:1 /W:1 /MIR /LOG:c:\path /ZB X:\source y:\dest
 
/E : Copy Subfolders, including Empty Subfolders.​
/COPYALL : Copy ALL file info (equivalent to /COPY:DATSOU)
/MT:8 -> multi-threaded….how may cpu threads to use? 1-16
/R:1 -> how many retries on failure
/W:1 -> how long to wait between failures
/J : Copy using unbuffered I/O (recommended for large files).
/ZB : Use restartable mode; if access denied use Backup mode.
/LOG:c:\path -> where to log info to. Much faster than putting on screen. dump to file and check file when done.

--> if a subsequent run is done, use /LOG+:c:\path to append to original file.​

--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack




On Tue, Apr 11, 2017 at 4:51 PM, Klise, Steve <[hidden email]> wrote:

If you use robocopy I used the following switches:

 

/MIR  /e /copyall /r:0 /w:0

 

/mir is a dangerous one; make sure you understand it.. it might make your day go bad.  Copyall copies perms PLUS ownership.  Worked like a charm (at least for me back in the day).

 

 

From: <[hidden email]> on behalf of Basil <[hidden email]>
Date: Tuesday, April 11, 2017 at 1:44 PM
To: Chris Hague <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

Tim McCarthy
Hey Toasters, This tidbit of important info  has recently come to my attention:

Please be aware: There is an incompatibility between certain versions of robocopy and certain versions of clustered Data ONTAP that could result in files larger than 512MiB not being copied correctly. The incompatibility exists between versions of ONTAP starting with 8.3.1, and with the version of robocopy distributed with Windows 7 and Windows Server 2008R2 (internal version 5.1.10.1027). The robocopy distributed with later versions of Windows does not exhibit this problem.

So, if you are planning on using Robocopy, Use a windows version newer than windows 7 / server 2008R2.


--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack


443-228-TMAC (Google Voice)
214-279-3926 (eFAX)





NetApp - In partnership with Alpine Testing SolutionsNetApp Certified Data Administrator, ONTAPNetApp Certified Implementation Engineer - SAN Specialist, ONTAPNetApp Certified Storage Installation Engineer, ONTAPNetApp Certified Implementation Engineer - Data Protection Specialist


NetApp Candidate ID: NETAPP00041276
        FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M
FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH

           RHCE6 110-107-141 

On Tue, Apr 11, 2017 at 5:10 PM, tmac <[hidden email]> wrote:
robocopy  /E /COPYALL /MT:8 /R:1 /W:1 /MIR /LOG:c:\path /ZB X:\source y:\dest
 
/E : Copy Subfolders, including Empty Subfolders.​
/COPYALL : Copy ALL file info (equivalent to /COPY:DATSOU)
/MT:8 -> multi-threaded….how may cpu threads to use? 1-16
/R:1 -> how many retries on failure
/W:1 -> how long to wait between failures
/J : Copy using unbuffered I/O (recommended for large files).
/ZB : Use restartable mode; if access denied use Backup mode.
/LOG:c:\path -> where to log info to. Much faster than putting on screen. dump to file and check file when done.

--> if a subsequent run is done, use /LOG+:c:\path to append to original file.​

--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack




On Tue, Apr 11, 2017 at 4:51 PM, Klise, Steve <[hidden email]> wrote:

If you use robocopy I used the following switches:

 

/MIR  /e /copyall /r:0 /w:0

 

/mir is a dangerous one; make sure you understand it.. it might make your day go bad.  Copyall copies perms PLUS ownership.  Worked like a charm (at least for me back in the day).

 

 

From: <[hidden email]> on behalf of Basil <[hidden email]>
Date: Tuesday, April 11, 2017 at 1:44 PM
To: Chris Hague <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters




_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: 7MTT CIFS Question

mrlizard23
In reply to this post by basilberntsen

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [mailto:[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: 7MTT CIFS Question

mrlizard23

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: 7MTT CIFS Question

Alexander Griesser-2

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

Tim McCarthy
Something I did last year...

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.
Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

Then go back over with something like robocopy to apply permissions?
If I recall, robocopy will not copy the file, just the security features of the file.

Just a thought.

--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack




On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: 7MTT CIFS Question

Alexander Griesser-2

Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: tmac [mailto:[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:23
An: Alexander Griesser <[hidden email]>
Cc: Chris Hague <[hidden email]>; [hidden email]
Betreff: Re: 7MTT CIFS Question

 

Something I did last year...

 

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.

Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

 

Then go back over with something like robocopy to apply permissions?

If I recall, robocopy will not copy the file, just the security features of the file.

 

Just a thought.


--tmac

 

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack

 

 

 

On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: 7MTT CIFS Question

Parisi, Justin

XCP needs a license, but it’s free.

 

The license is for tracking and statistic purposes only.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Alexander Griesser
Sent: Wednesday, April 12, 2017 1:25 PM
To: NGC-tmacmd-gmail.com <[hidden email]>
Cc: [hidden email]
Subject: AW: 7MTT CIFS Question

 

Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: tmac [[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:23
An: Alexander Griesser <[hidden email]>
Cc: Chris Hague <[hidden email]>; [hidden email]
Betreff: Re: 7MTT CIFS Question

 

Something I did last year...

 

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.

Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

 

Then go back over with something like robocopy to apply permissions?

If I recall, robocopy will not copy the file, just the security features of the file.

 

Just a thought.


--tmac

 

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack

 

 

 

On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

Tim McCarthy
In reply to this post by Alexander Griesser-2
Just ask. The license is free. If it runs out, ask for another. They should just give it to you
They just do that to track who and how the software is being used.



From: Alexander Griesser <[hidden email]>
Sent: Wednesday, April 12, 2017 1:24:55 PM
To: tmac
Cc: Chris Hague; [hidden email]
Subject: AW: 7MTT CIFS Question
 

Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: tmac [mailto:[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:23
An: Alexander Griesser <[hidden email]>
Cc: Chris Hague <[hidden email]>; [hidden email]
Betreff: Re: 7MTT CIFS Question

 

Something I did last year...

 

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.

Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

 

Then go back over with something like robocopy to apply permissions?

If I recall, robocopy will not copy the file, just the security features of the file.

 

Just a thought.


--tmac

 

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack

 

 

 

On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: 7MTT CIFS Question

Alexander Griesser-2
In reply to this post by Parisi, Justin

Alright, great – will ask my sales rep to send me a bunch of those then J

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: Parisi, Justin [mailto:[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:26
An: Alexander Griesser <[hidden email]>; NGC-tmacmd-gmail.com <[hidden email]>
Cc: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

XCP needs a license, but it’s free.

 

The license is for tracking and statistic purposes only.

 

From: [hidden email] [[hidden email]] On Behalf Of Alexander Griesser
Sent: Wednesday, April 12, 2017 1:25 PM
To: NGC-tmacmd-gmail.com <[hidden email]>
Cc: [hidden email]
Subject: AW: 7MTT CIFS Question

 

Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: tmac [[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:23
An: Alexander Griesser <[hidden email]>
Cc: Chris Hague <[hidden email]>; [hidden email]
Betreff: Re: 7MTT CIFS Question

 

Something I did last year...

 

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.

Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

 

Then go back over with something like robocopy to apply permissions?

If I recall, robocopy will not copy the file, just the security features of the file.

 

Just a thought.


--tmac

 

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack

 

 

 

On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: 7MTT CIFS Question

Parisi, Justin
In reply to this post by Tim McCarthy

You don’t even need to ask. Go to xcp.netapp.com and request one there. Then download and enjoy.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim McCarthy
Sent: Wednesday, April 12, 2017 1:27 PM
To: Alexander Griesser <[hidden email]>
Cc: [hidden email]
Subject: Re: 7MTT CIFS Question

 

Just ask. The license is free. If it runs out, ask for another. They should just give it to you

They just do that to track who and how the software is being used.

 

 


From: Alexander Griesser <[hidden email]>
Sent: Wednesday, April 12, 2017 1:24:55 PM
To: tmac
Cc: Chris Hague; [hidden email]
Subject: AW: 7MTT CIFS Question

 

Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: tmac [[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:23
An: Alexander Griesser <[hidden email]>
Cc: Chris Hague <[hidden email]>; [hidden email]
Betreff: Re: 7MTT CIFS Question

 

Something I did last year...

 

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.

Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

 

Then go back over with something like robocopy to apply permissions?

If I recall, robocopy will not copy the file, just the security features of the file.

 

Just a thought.


--tmac

 

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack

 

 

 

On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AW: 7MTT CIFS Question

Michael Bergman
In reply to this post by Alexander Griesser-2
On 2017-04-12 19:24, Alexander Griesser wrote:
> Speaking of which: I tried that once but stopped early in the process due to
> XCP needing a license; anyone aware of a semi-automated way to get those or
> has this changed already?

As many have already pointed out, is quite easy to just get an XCP license
from a NetApp Web-portal. So, yes (I think) :-)  XCP is a very good and
impressive tool, its author certainly knows what he's doing (Peter Schay)

/M
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 7MTT CIFS Question

basilberntsen
In reply to this post by Parisi, Justin
At least on the version I use, xcp is nfs only. The secret sauce is that they don't use the OS for nfs, they use python. You can do anything they do yourself with python in a vm. 

Be careful of trying to use nfs on ntfs security style files- test first. If you want to use nfs to delete, you might be able to change the security style of the data to unix, which will drastically simplify your task. Once everything is unix security, being root on a server which isn't being rootsquashed should allow you to delete everything. 

On Wed, Apr 12, 2017 at 1:34 PM Parisi, Justin <[hidden email]> wrote:

You don’t even need to ask. Go to xcp.netapp.com and request one there. Then download and enjoy.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim McCarthy
Sent: Wednesday, April 12, 2017 1:27 PM
To: Alexander Griesser <[hidden email]>
Cc: [hidden email]


Subject: Re: 7MTT CIFS Question

 

Just ask. The license is free. If it runs out, ask for another. They should just give it to you

They just do that to track who and how the software is being used.

 

 


From: Alexander Griesser <[hidden email]>
Sent: Wednesday, April 12, 2017 1:24:55 PM
To: tmac
Cc: Chris Hague; [hidden email]
Subject: AW: 7MTT CIFS Question

 

Speaking of which: I tried that once but stopped early in the process due to XCP needing a license; anyone aware of a semi-automated way to get those or has this changed already?

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: tmac [[hidden email]]
Gesendet: Mittwoch, 12. April 2017 19:23
An: Alexander Griesser <[hidden email]>
Cc: Chris Hague <[hidden email]>; [hidden email]
Betreff: Re: 7MTT CIFS Question

 

Something I did last year...

 

Download XCP from netapp support. Run on a LINUX machine with a high speed connection.

Use XCP to copy via a backdoor NFS-like protocol, but extremely fast.

 

Then go back over with something like robocopy to apply permissions?

If I recall, robocopy will not copy the file, just the security features of the file.

 

Just a thought.


--tmac

 

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack

 

 

 

On Wed, Apr 12, 2017 at 1:14 PM, Alexander Griesser <[hidden email]> wrote:

In the past, I was having good experience with rsync, but this was not really tested on CIFS mounted shares.

Anyways, rsync is much faster in deleting a whole bunch of files and widespread folder structures than a recursive delete is, but that has only been tested on NFS mounted volumes so far.

The trick is to tell rsync to sync an empty directory over the full one and there are also parallel implementations of rsync which start off at different parts of the filesystem structure to further speed up the process – but again, not tested on Windows, but might be worth a try for just deleting this folder structure quickly.

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: [hidden email]

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Chris Hague
Gesendet: Mittwoch, 12. April 2017 17:52
An: [hidden email]
Betreff: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: 7MTT CIFS Question

mrlizard23
In reply to this post by mrlizard23

Just a quick update on this one for anyone interested;

 

Having left the script to take ownership and change permissions running in windows for over a week on one folder containing 22m files, it had not even gotten past the first 1 of 3160 root folders!!!!!!

 

So, with the aid of an ONTAP 9 simulator, we were able to rm –r –f from the systemshell.

 

You have to unlock the diag user, then “sudo rm –r –f /clust/svm/volume” but it works and works without requiring ownership or permission to the files. It’s fast too. (done nearly 140 root folders in an hour)

 

This isn’t ideal having to use the systemshell, but is the only way we have found to perform this task in a timely, reliable manor.

 

Kind Regards,

Chris.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 16:52
To: [hidden email]
Subject: RE: 7MTT CIFS Question

 

Ok, so fastcopy isn’t really much faster than windows delete. However it does delete the “path too long” files which windows fails on.

 

Now I have permissions issues!

 

So I  have put together a crude batchfile to take ownership with subinacl, set permissions with icacls, delete what it can with del and rmdir, then finish off the job with fastcopy.

 

set wrkfile=\\cifs_server\root_share\dead_share

subinacl.exe /file %wrkfile% /setowner=%username%

subinacl.exe /subdirectories %wrkfile%\*.*  /setowner=%username%

ICACLS %wrkfile% /grant %username%:F /T

del /f /s /q %wrkfile%

rmdir /q /s %wrkfile%

FastCopy.exe /cmd=delete /no_confirm_del /no_confirm_stop %wrkfile%

 

But I’m still getting errors trying to take ownership of subfolders deep down in userprofiles.

 

Surely there has got to be an easier way??

systemshell –node a

cd /folder

rm –rf

J

 

From: [hidden email] [[hidden email]] On Behalf Of Chris Hague
Sent: 12 April 2017 09:20
To: Basil; [hidden email]
Subject: RE: 7MTT CIFS Question

 

Thanks Basil,

 

I do prefer the clone and delete option because I can control users access to the data. When I wish to perform the cutover I can remove the 7-Mode share from DFS, update the baseline with 7MTT and then bring the shares online from CDOT before updating the DFS share paths.

 

Then indeed I can perform the clean up behind the scenes as I am sharing a subfolder from each of the clones.

 

I would just like a more reliable \ faster way of deleting the folders and files… I have read reviews that FastCopy https://ipmsg.org/tools/fastcopy.html.en may be able to do this, so I will perform a test operation using that.

 

Kind Regards,

Chris.

 

From: Basil [[hidden email]]
Sent: 11 April 2017 21:45
To: Chris Hague; [hidden email]
Subject: Re: 7MTT CIFS Question

 

Your clone and delete plan is likely your best option. It does indeed take forever but it's all behind the scenes assuming you are sharing the subfolder rather than the whole clone. 

 

 

On Tue, Apr 11, 2017 at 12:55 PM Chris Hague <[hidden email]> wrote:

Hi all,

 

I am soon to be transitioning a large CIFS share (5tb) from 7-Mode to CDOT and during the migration I would like to use the opportunity to split the share into several more manageable volumes.

 

My initial thoughts were to use robocopy, but there are potentially permissions issues.

 

Another method I’m exploring is to use 7MTT to mirror the volume, then once complete, create flexclones, 1 for each of the shares I would like to split out. Then delete the no longer required data from each respective share and vol split before deleting the master.

 

Firstly, am I missing some simpler way of doing this? systemshell maybe?

Secondly, if not, then is there a quicker way of deleting 20m files from each of my 6 shares? Maybe an on-filer way of destroying the folders? So far using command line tools (del, rmdir) still take far too long.

 

Many Thanks in advance,

Chris.

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Loading...