Backing up to tape?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Backing up to tape?

Ehrenwald, Ian
Hello
I’m curious what other people are using for software to back up volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes.  We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3.  AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.

After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc).  Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0.  The tape drives are offline, the media changer is available.

I’ve played with the various ‘storage tape’ commands but am making no headway.  I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it.  I understand we want SVM-scoped ndmp.  Does anyone have bare bones examples using ndmp or some other local utility?  If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.

Thanks!


--
Ian Ehrenwald
Senior Infrastructure Engineer
Hachette Book Group, Inc.
1.617.263.1948 / [hidden email]



This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

Jeffrey Mohler
Tape??


Crazy talk.   


#NeverTooMuchSarcasm


Sent from Yahoo Mail for iPhone

On Monday, May 15, 2017, 12:40, Ehrenwald, Ian <[hidden email]> wrote:

Hello
I’m curious what other people are using for software to back up volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes.  We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3.  AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.

After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc).  Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0.  The tape drives are offline, the media changer is available.

I’ve played with the various ‘storage tape’ commands but am making no headway.  I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it.  I understand we want SVM-scoped ndmp.  Does anyone have bare bones examples using ndmp or some other local utility?  If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.

Thanks!


--
Ian Ehrenwald
Senior Infrastructure Engineer
Hachette Book Group, Inc.
<a dir="ltr" href="tel:1.617.263.1948" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="1">1.617.263.1948 / [hidden email]



This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

_______________________________________________
Toasters mailing list

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

basilberntsen
In reply to this post by Ehrenwald, Ian
Without back up software, I think you can only control a drive, not a robot. Somebody can feel free to correct me if I'm wrong, I've never tried anything else. 

On Mon, May 15, 2017 at 3:36 PM Ehrenwald, Ian <[hidden email]> wrote:
Hello
I’m curious what other people are using for software to back up volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes.  We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3.  AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.

After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc).  Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0.  The tape drives are offline, the media changer is available.

I’ve played with the various ‘storage tape’ commands but am making no headway.  I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it.  I understand we want SVM-scoped ndmp.  Does anyone have bare bones examples using ndmp or some other local utility?  If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.

Thanks!


--
Ian Ehrenwald
Senior Infrastructure Engineer
Hachette Book Group, Inc.
1.617.263.1948 / [hidden email]



This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

Klise, Steve-2

We used NDMP and Symantec NetBackup.  Was pretty decent setup, but does put a load on your filer. 

 

From: <[hidden email]> on behalf of Basil <[hidden email]>
Date: Monday, May 15, 2017 at 2:24 PM
To: "Ehrenwald, Ian" <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: Backing up to tape?

 

Without back up software, I think you can only control a drive, not a robot. Somebody can feel free to correct me if I'm wrong, I've never tried anything else. 

 

On Mon, May 15, 2017 at 3:36 PM Ehrenwald, Ian <[hidden email]> wrote:

Hello
I’m curious what other people are using for software to back up volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes.  We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3.  AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.

After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc).  Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0.  The tape drives are offline, the media changer is available.

I’ve played with the various ‘storage tape’ commands but am making no headway.  I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it.  I understand we want SVM-scoped ndmp.  Does anyone have bare bones examples using ndmp or some other local utility?  If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.

Thanks!


--
Ian Ehrenwald
Senior Infrastructure Engineer
Hachette Book Group, Inc.
1.617.263.1948 / [hidden email]



This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

Tim McCarthy
Yepper. On volumes with lots of files, that ndmp history/catalog pass can be crazy!

I once tried to use ndmp backup on a volume with 500,000+ files. After 8 hours and a huge load (FAS6080) the ndmp gave out and quit. Eight hours of building the index and it didn't finish. 

Spelling errors courtesy of Swype for iOS...

Tim McCarthy, Consultant at Presidio
_____________________________
From: Klise, Steve <[hidden email]>
Sent: Monday, May 15, 2017 6:00 PM
Subject: Re: Backing up to tape?
To: <[hidden email]>, Basil <[hidden email]>, Ehrenwald, Ian <[hidden email]>


We used NDMP and Symantec NetBackup.  Was pretty decent setup, but does put a load on your filer. 

 

From: <<[hidden email]> on behalf of Basil <[hidden email]>
Date: Monday, May 15, 2017 at 2:24 PM
To: "Ehrenwald, Ian" <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: Backing up to tape?

 

Without back up software, I think you can only control a drive, not a robot. Somebody can feel free to correct me if I'm wrong, I've never tried anything else. 

 

On Mon, May 15, 2017 at 3:36 PM Ehrenwald, Ian <[hidden email]> wrote:

Hello
I’m curious what other people are using for software to back up volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes.  We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3.  AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.

After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc).  Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0.  The tape drives are offline, the media changer is available.

I’ve played with the various ‘storage tape’ commands but am making no headway.  I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it.  I understand we want SVM-scoped ndmp.  Does anyone have bare bones examples using ndmp or some other local utility?  If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.

Thanks!


--
Ian Ehrenwald
Senior Infrastructure Engineer
Hachette Book Group, Inc.
1.617.263.1948 / [hidden email]



This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters




_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

John Stoffel-5
In reply to this post by Ehrenwald, Ian

Ian> I’m curious what other people are using for software to back up
Ian> volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x
Ian> LTO7 drives and 24 tapes.  We have a four-node cluster running
Ian> 8.3.2Px and for now we’re directly connecting the TL4000’s A-side
Ian> FC to node 1 and the B-side to node 3.  AFAIK the device itself
Ian> is set up correctly and all the physical and virtual status LEDs
Ian> and are happy.

We're in the process of upgrading to 8.3.x from 8.2.3 cDOT just so we
can properly use Netbackup with NDMP at the VServer level.  Under 8.2,
it just doesnt' work reliably.  I'm hoping that this upgrade will make
it work properly.

In any case, you should make sure that each head on the cluster has
access to all the tape drives, otherwise you'll probably have a bunch
of your backups running across the Cluster Network to do your dumps.
And you *might* not be able to NDMP from all vservers.  

Ian> After installing the correct TCF file, ‘sysconfig –t’ run against
Ian> the directly connected nodes shows the drive as supported (IBM
Ian> LTO-7 ULT3580) with all sorts of device name permutations
Ian> (rewind, no rewind, unload, etc).  Running ‘storage tape show’
Ian> shows two tape drives, one per directly attached node, named st0
Ian> and a media changer on node 3 named mc0.  The tape drives are
Ian> offline, the media changer is available.

Ian> I’ve played with the various ‘storage tape’ commands but am
Ian> making no headway.  I’m sure the docs are out there, and I’ve
Ian> looked through the NetApp documentation as well as Googling for a
Ian> while, but don’t see a way to control the media changer to, for
Ian> example, tell tape drive 1 to load slot 6, and then another
Ian> command to dump a volume to it.  I understand we want SVM-scoped
Ian> ndmp.  Does anyone have bare bones examples using ndmp or some
Ian> other local utility?  If we need to purchase something like
Ian> Tivoli or Netbackup or CommVault etc then I guess we will, but
Ian> we’d prefer free.

There's no free NDMP (especially three way NDMP where it's initiated
by a server, dumped by a node to a local tape drive, and the index of
files and tape positions sent back to the server).

You're going to have to bite the bullet.


Personally, I'd go with EMC's Legato Networker (not that I've used it
in anger for years and years... but the CLI is/was excellent), then
probably Netbackup 7.7.x which is what we use now.

I've also run CommVault 7, 8 & now 9 for quite a number of years and I
dislike it alot.  It's a pain to use, indexes expire so damn quickly
it's not funny, and it's just not fun to use.

But... it's cheaper than the other options.

John

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

John Stoffel-5
In reply to this post by Tim McCarthy

Tim> Yepper. On volumes with lots of files, that ndmp history/catalog
Tim> pass can be crazy!

Hah!  We've got a volume with 50 million files and it sucks for NDMP
backups.  Fulls are actually not terrible... but the index pass just
takes forever.

Tim> I once tried to use ndmp backup on a volume with 500,000+
Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out
Tim> and quit. Eight hours of building the index and it didn't finish.

This is the probably with *any* file level backup.  Unfortunately,
there's no good solution in terms of price for backing up Netapps that
I'm aware of.

You could buy a cheaper pair of heads with lots and lots of cheap(ish)
SATA storage, but it's still god-awful expensive.  And the snapmirror
licenses aren't cheap either.

In my engineering environments, I've been trying to encourage them to
only backup what they need, and to work in scratch areas instead, but
it's hard to get people to change.

The Netapp is just so reliable that is really does keep data for years
and years without problems.  In more cases.

Heh.

But some $WORK orgs have off-site requirements, and they're not
willing to pay the price for a remote co-lo Netapp and storage and the
bandwidth to make backups work reliably and quickly enough.  So off to
tape it is.  Make's JSOX happy too...
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

NDMP Performance...

Jeffrey Mohler
Does anyone that is using NDMP for file-dense restores also notice a HUGE gap in time between starting the restore, and data to begin moving...with no observable load/activity during that odd bizzare Phase-0 period?

We can see -hours- of 0% CPU, 0KB to disk (short of CP label updates to them), which challenges us in other ways because it brings average dataset recover speed/times to low double-digit MB/sec numbers no matter how fast tape is.
 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Monday, May 15, 2017 5:58 PM, John Stoffel <[hidden email]> wrote:



Tim> Yepper. On volumes with lots of files, that ndmp history/catalog
Tim> pass can be crazy!

Hah!  We've got a volume with 50 million files and it sucks for NDMP
backups.  Fulls are actually not terrible... but the index pass just
takes forever.

Tim> I once tried to use ndmp backup on a volume with 500,000+
Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out
Tim> and quit. Eight hours of building the index and it didn't finish.

This is the probably with *any* file level backup.  Unfortunately,
there's no good solution in terms of price for backing up Netapps that
I'm aware of.

You could buy a cheaper pair of heads with lots and lots of cheap(ish)
SATA storage, but it's still god-awful expensive.  And the snapmirror
licenses aren't cheap either.

In my engineering environments, I've been trying to encourage them to
only backup what they need, and to work in scratch areas instead, but
it's hard to get people to change.

The Netapp is just so reliable that is really does keep data for years
and years without problems.  In more cases.

Heh.

But some $WORK orgs have off-site requirements, and they're not
willing to pay the price for a remote co-lo Netapp and storage and the
bandwidth to make backups work reliably and quickly enough.  So off to
tape it is.  Make's JSOX happy too...

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: NDMP Performance...

John Stoffel-5

Jeffrey> Does anyone that is using NDMP for file-dense restores also
Jeffrey> notice a HUGE gap in time between starting the restore, and
Jeffrey> data to begin moving...with no observable load/activity
Jeffrey> during that odd bizzare Phase-0 period?

I'm not too surprised actually... the NDMP is really just a wrapper
around the dump/restore built into OnTap, though I admit that maybe
this has changed with cDOT.

In older 7-mode, it was just a single file under /root/<vol>/ that
held the index of file(s) dumped to tape at which level.  So I doubt
it's been improved all that much since.

Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label
Jeffrey> updates to them), which challenges us in other ways because
Jeffrey> it brings average dataset recover speed/times to low
Jeffrey> double-digit MB/sec numbers no matter how fast tape is.

Is this before or after you select the file(s) to restore?  Or are you
doing a full volume restore?  I wonder if it's sorting the files by
tape block or something like that before starting the restore.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

Tim McCarthy
In reply to this post by John Stoffel-5
We killed those backups. After review, we determined if we had to, we could request and reconstitute the data from another source. We ended up just backing up our intellectual property which took minutes and only a few tapes....until we retired an older system. Then we made it the "backup" and snapmirrored the important stuff to it...got rid of tapes!

--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack




On Mon, May 15, 2017 at 8:51 PM, John Stoffel <[hidden email]> wrote:

Tim> Yepper. On volumes with lots of files, that ndmp history/catalog
Tim> pass can be crazy!

Hah!  We've got a volume with 50 million files and it sucks for NDMP
backups.  Fulls are actually not terrible... but the index pass just
takes forever.

Tim> I once tried to use ndmp backup on a volume with 500,000+
Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out
Tim> and quit. Eight hours of building the index and it didn't finish.

This is the probably with *any* file level backup.  Unfortunately,
there's no good solution in terms of price for backing up Netapps that
I'm aware of.

You could buy a cheaper pair of heads with lots and lots of cheap(ish)
SATA storage, but it's still god-awful expensive.  And the snapmirror
licenses aren't cheap either.

In my engineering environments, I've been trying to encourage them to
only backup what they need, and to work in scratch areas instead, but
it's hard to get people to change.

The Netapp is just so reliable that is really does keep data for years
and years without problems.  In more cases.

Heh.

But some $WORK orgs have off-site requirements, and they're not
willing to pay the price for a remote co-lo Netapp and storage and the
bandwidth to make backups work reliably and quickly enough.  So off to
tape it is.  Make's JSOX happy too...


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

John Stoffel-5

I wish we could do that... but we have offsite requirements that we
just can't get away from, along with data retention requirements.  We
just got asked for a restore of email from 2001... which ain't too
likely to actually happen.  LOL.

tmac> We killed those backups. After review, we determined if we had
tmac> to, we could request and reconstitute the data from another
tmac> source. We ended up just backing up our intellectual property
tmac> which took minutes and only a few tapes....until we retired an
tmac> older system. Then we made it the "backup" and snapmirrored the
tmac> important stuff to it...got rid of tapes!

tmac> --tmac

tmac> Tim McCarthy, Principal Consultant

tmac> Proud Member of the #NetAppATeam

tmac> I Blog at TMACsRack




tmac> On Mon, May 15, 2017 at 8:51 PM, John Stoffel <[hidden email]> wrote:


Tim> Yepper. On volumes with lots of files, that ndmp history/catalog
Tim> pass can be crazy!

tmac>     Hah!  We've got a volume with 50 million files and it sucks for NDMP
tmac>     backups.  Fulls are actually not terrible... but the index pass just
tmac>     takes forever.

Tim> I once tried to use ndmp backup on a volume with 500,000+
Tim> files. After 8 hours and a huge load (FAS6080) the ndmp gave out
Tim> and quit. Eight hours of building the index and it didn't finish.

tmac>     This is the probably with *any* file level backup.  Unfortunately,
tmac>     there's no good solution in terms of price for backing up Netapps that
tmac>     I'm aware of.

tmac>     You could buy a cheaper pair of heads with lots and lots of cheap(ish)
tmac>     SATA storage, but it's still god-awful expensive.  And the snapmirror
tmac>     licenses aren't cheap either.

tmac>     In my engineering environments, I've been trying to encourage them to
tmac>     only backup what they need, and to work in scratch areas instead, but
tmac>     it's hard to get people to change.

tmac>     The Netapp is just so reliable that is really does keep data for years
tmac>     and years without problems.  In more cases.

tmac>     Heh.

tmac>     But some $WORK orgs have off-site requirements, and they're not
tmac>     willing to pay the price for a remote co-lo Netapp and storage and the
tmac>     bandwidth to make backups work reliably and quickly enough.  So off to
tmac>     tape it is.  Make's JSOX happy too...



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: NDMP Performance...

Jeffrey Mohler
In reply to this post by John Stoffel-5
Full volume restore...a small rush of data comes off tape at the start, and then a walking-dead number of hours of zero indicated activity assembling the idea of what to do with it.

None of this is particularly a surprise, other than yes, it appears a forgotten 'who cares' section of ONTAP with still, a valuable amount of customer demand for it..and Im wondering if anyone else is applying any pressure to pull this out of the early 2000s.

Yes, there are other ways to manage data protection, but as long as this is still offered, it should receive responsible attention and adapt to what was 1 CPU systems, to now well over a dozen cores to get the job done.

Not doing this a different way than NDMP..is less of a reflection on where we are WRT to the year marked on the calendar, but certain scales make it harder to spend significantly (many multiples) more on a solution that in the end..is still a MAYBE you will use it restore proposition.

Change is in the air, and how/why/value/cost/support/attention to issues like this have become top of mind again.






 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Monday, May 15, 2017 6:15 PM, John Stoffel <[hidden email]> wrote:



Jeffrey> Does anyone that is using NDMP for file-dense restores also
Jeffrey> notice a HUGE gap in time between starting the restore, and
Jeffrey> data to begin moving...with no observable load/activity
Jeffrey> during that odd bizzare Phase-0 period?

I'm not too surprised actually... the NDMP is really just a wrapper
around the dump/restore built into OnTap, though I admit that maybe
this has changed with cDOT.

In older 7-mode, it was just a single file under /root/<vol>/ that
held the index of file(s) dumped to tape at which level.  So I doubt
it's been improved all that much since.

Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label
Jeffrey> updates to them), which challenges us in other ways because
Jeffrey> it brings average dataset recover speed/times to low
Jeffrey> double-digit MB/sec numbers no matter how fast tape is.

Is this before or after you select the file(s) to restore?  Or are you
doing a full volume restore?  I wonder if it's sorting the files by
tape block or something like that before starting the restore.





_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

RE: NDMP Performance...

Steiner, Jeffrey
In reply to this post by John Stoffel-5
Just a note:

NDMP is a backup control protocol. It does not define a data format. We run into a lot of confusion on that point. Since NDMP is a standard, you might think that an NDMP backup on one storage array could be restored via NDMP on a different array. That's not a valid conclusion. It's not impossible that you can do that, but it depends on the tape format.

When NDMP first came out, it used the "dump" format. It was similar to ufsdump, and you could even restore the data on a Solaris box pretty easily, although you'd lose all the permission data. That's probably still doable to some degree, but it's not supported.

The problem with dump is it's a logical backup. ONTAP has to walk the directory tree, and the more files you have, the longer it takes. That's why there's a second format option - NDMP with SMtape format. It's documented under "Understanding SMTape engine for FlexVol volumes" in the ONTAP documentation.

I've never met anyone using it personally, but it's been around for a while. I think TSM was the first to support it. It basically does a snapmirror operation to a tape drive, which means you preserve the efficiency. Since it's not walking the files, it was far faster than NDMP with the dump format.

I'm pretty sure that you still can't do a single file restore with SMtape format. You'd have to restore the whole thing. That may or may not be sufficient. If a customer just wants an offsite copy of everything as a quarterly archive or protection against disaster, that should be a good option.

I don't work with tape backups any longer, so I don't know anyone personally who's used SMtape, but if someone on Toasters has used it please let us know how it worked out.


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of John Stoffel
Sent: Tuesday, May 16, 2017 3:15 AM
To: Jeffrey Mohler <[hidden email]>
Cc: [hidden email]
Subject: Re: NDMP Performance...


Jeffrey> Does anyone that is using NDMP for file-dense restores also
Jeffrey> notice a HUGE gap in time between starting the restore, and
Jeffrey> data to begin moving...with no observable load/activity during
Jeffrey> that odd bizzare Phase-0 period?

I'm not too surprised actually... the NDMP is really just a wrapper around the dump/restore built into OnTap, though I admit that maybe this has changed with cDOT.

In older 7-mode, it was just a single file under /root/<vol>/ that held the index of file(s) dumped to tape at which level.  So I doubt it's been improved all that much since.

Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label
Jeffrey> updates to them), which challenges us in other ways because it
Jeffrey> brings average dataset recover speed/times to low double-digit
Jeffrey> MB/sec numbers no matter how fast tape is.

Is this before or after you select the file(s) to restore?  Or are you doing a full volume restore?  I wonder if it's sorting the files by tape block or something like that before starting the restore.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

Sebastian Goetze
In reply to this post by Ehrenwald, Ian
Hi Ian, 

I had a student in one of my courses, and she backs everything up to tape via scripted SMTape instead of NDMP. Fast, reliable, cheap. I remember being thoroughly impressed, in addition to female storage admins being a rarity...

Would that be an alternative? 
I'm not familiar with the details of her solution, though, or if she had a tape changer. But it was not a small environment (Hamburg/Germany waterworks).

Sebastian


On May 15, 2017 9:32 PM, "Ehrenwald, Ian" <[hidden email]> wrote:
Hello
I’m curious what other people are using for software to back up volumes to tape?  We’ve recently purchased a Dell TL4000 with 2 x LTO7 drives and 24 tapes.  We have a four-node cluster running 8.3.2Px and for now we’re directly connecting the TL4000’s A-side FC to node 1 and the B-side to node 3.  AFAIK the device itself is set up correctly and all the physical and virtual status LEDs and are happy.

After installing the correct TCF file, ‘sysconfig –t’ run against the directly connected nodes shows the drive as supported (IBM LTO-7 ULT3580) with all sorts of device name permutations (rewind, no rewind, unload, etc).  Running ‘storage tape show’ shows two tape drives, one per directly attached node, named st0 and a media changer on node 3 named mc0.  The tape drives are offline, the media changer is available.

I’ve played with the various ‘storage tape’ commands but am making no headway.  I’m sure the docs are out there, and I’ve looked through the NetApp documentation as well as Googling for a while, but don’t see a way to control the media changer to, for example, tell tape drive 1 to load slot 6, and then another command to dump a volume to it.  I understand we want SVM-scoped ndmp.  Does anyone have bare bones examples using ndmp or some other local utility?  If we need to purchase something like Tivoli or Netbackup or CommVault etc then I guess we will, but we’d prefer free.

Thanks!


--
Ian Ehrenwald
Senior Infrastructure Engineer
Hachette Book Group, Inc.
1.617.263.1948 / [hidden email]



This may contain confidential material. If you are not an intended recipient, please notify the sender, delete immediately, and understand that no disclosure or reliance on the information herein is permitted. Hachette Book Group may monitor email to and from our network.

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: Backing up to tape?

John Stoffel-5

Sebastian> I had a student in one of my courses, and she backs
Sebastian> everything up to tape via scripted SMTape instead of
Sebastian> NDMP. Fast, reliable, cheap. I remember being thoroughly
Sebastian> impressed, in addition to female storage admins being a
Sebastian> rarity...

Sebastian> Would that be an alternative?   I'm not familiar with the
Sebastian> details of her solution, though, or if she had a tape
Sebastian> changer. But it was not a small environment
Sebastian> (Hamburg/Germany waterworks).

I'm certainly going to look into this for my future backups.  The
problem of course is that to restore a single file you need to restore
it all, starting with the base full snapshot I would think.

But certainly tempting.


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|

Re: NDMP Performance...

Jeffrey Mohler
In reply to this post by Steiner, Jeffrey
We use SMTape, the only issue is the limited format support Netapp provides, IIRC two major versions...so it's not a long term backup solution because you have to renew/refresh it often.

It's very fast however compared to an end to end NDMP.

Snapdiff is looking interesting too.
 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Monday, May 15, 2017 11:42 PM, "Steiner, Jeffrey" <[hidden email]> wrote:


Just a note:

NDMP is a backup control protocol. It does not define a data format. We run into a lot of confusion on that point. Since NDMP is a standard, you might think that an NDMP backup on one storage array could be restored via NDMP on a different array. That's not a valid conclusion. It's not impossible that you can do that, but it depends on the tape format.

When NDMP first came out, it used the "dump" format. It was similar to ufsdump, and you could even restore the data on a Solaris box pretty easily, although you'd lose all the permission data. That's probably still doable to some degree, but it's not supported.

The problem with dump is it's a logical backup. ONTAP has to walk the directory tree, and the more files you have, the longer it takes. That's why there's a second format option - NDMP with SMtape format. It's documented under "Understanding SMTape engine for FlexVol volumes" in the ONTAP documentation.

I've never met anyone using it personally, but it's been around for a while. I think TSM was the first to support it. It basically does a snapmirror operation to a tape drive, which means you preserve the efficiency. Since it's not walking the files, it was far faster than NDMP with the dump format.

I'm pretty sure that you still can't do a single file restore with SMtape format. You'd have to restore the whole thing. That may or may not be sufficient. If a customer just wants an offsite copy of everything as a quarterly archive or protection against disaster, that should be a good option.

I don't work with tape backups any longer, so I don't know anyone personally who's used SMtape, but if someone on Toasters has used it please let us know how it worked out.


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of John Stoffel
Sent: Tuesday, May 16, 2017 3:15 AM
To: Jeffrey Mohler <[hidden email]>
Cc: [hidden email]
Subject: Re: NDMP Performance...


Jeffrey> Does anyone that is using NDMP for file-dense restores also
Jeffrey> notice a HUGE gap in time between starting the restore, and
Jeffrey> data to begin moving...with no observable load/activity during
Jeffrey> that odd bizzare Phase-0 period?

I'm not too surprised actually... the NDMP is really just a wrapper around the dump/restore built into OnTap, though I admit that maybe this has changed with cDOT.

In older 7-mode, it was just a single file under /root/<vol>/ that held the index of file(s) dumped to tape at which level.  So I doubt it's been improved all that much since.

Jeffrey> We can see -hours- of 0% CPU, 0KB to disk (short of CP label
Jeffrey> updates to them), which challenges us in other ways because it
Jeffrey> brings average dataset recover speed/times to low double-digit
Jeffrey> MB/sec numbers no matter how fast tape is.


Is this before or after you select the file(s) to restore?  Or are you doing a full volume restore?  I wonder if it's sorting the files by tape block or something like that before starting the restore.


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters