AFF and 4:1 guaranteed efficiency

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AFF and 4:1 guaranteed efficiency

jordan slingerland-2
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Ray Van Dolson-3
On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:

> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9.
> They say they grantee it.  I have specifically asked what
> stipulation.  In a VDI environment with linked clones, etc.  Sales
> guy tells me none and even specifically says they can dedup
> compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or
> what?
>
> Any comments welcome.
>
> --Jordan

We'll be POC'ing one, so will see.  Am skeptical, but don't think it's
like Pure Storage where they also factor in snapshots.

Most likely the guarantee is a carrot, and if they fall short, they'll
give you hardware to make up the difference.  Subsequent purchases,
however won't give you the same guarantee so you'll need to buy more
(again, this is also what Pure does).

Other thoughts?

Ray
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Mike Gossett
In reply to this post by jordan slingerland-2
my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Michael Bergman
In reply to this post by Ray Van Dolson-3
Well, just one (thought).  Life comes w/o guarantee.  :-)

Jokes aside, I agree w Ray van D; logically you can get free lunch once or
twice if there's something in it for the buyer (an opportunity of *some*
sort). But never will anyone buy you lunch for a whole year or more --
unless you did something that made them enter voluntary slavery

/M

On 2016-09-22 20:39, Ray Van Dolson wrote:

> We'll be POC'ing one, so will see.  Am skeptical, but don't think it's
> like Pure Storage where they also factor in snapshots.
>
> Most likely the guarantee is a carrot, and if they fall short, they'll
> give you hardware to make up the difference.  Subsequent purchases,
> however won't give you the same guarantee so you'll need to buy more
> (again, this is also what Pure does).
>
> Other thoughts?
>
> Ray
> _______________________________________________

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Ray Van Dolson-3
In reply to this post by Ray Van Dolson-3
On Thu, Sep 22, 2016 at 11:39:09AM -0700, Ray Van Dolson wrote:

> On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
> > Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9.
> > They say they grantee it.  I have specifically asked what
> > stipulation.  In a VDI environment with linked clones, etc.  Sales
> > guy tells me none and even specifically says they can dedup
> > compressed video or audio (mp3) 4:1.
> >
> > I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or
> > what?
> >
> > Any comments welcome.
> >
> > --Jordan
>
> We'll be POC'ing one, so will see.  Am skeptical, but don't think it's
> like Pure Storage where they also factor in snapshots.
>
> Most likely the guarantee is a carrot, and if they fall short, they'll
> give you hardware to make up the difference.  Subsequent purchases,
> however won't give you the same guarantee so you'll need to buy more
> (again, this is also what Pure does).
>
> Other thoughts?
>
> Ray

Correction: Looking back at notes I have, the guarantee relies on the
use of inline deduplication, inline compression, "compaction"[1], thin
provisioning as well as factoring in "regular" volume snapshots.

NetApp will help you realize the 4:1 savings given the techniques
described previously and then provide additional capacity if needed.

So, take out the snapshots and thin provisioning and will you see
significantly better savings than with other technologies?  Not sure --
I'd guess it'll be pretty similar.

Ray

[1] https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theregister.co.uk_2016_06_13_netapps-5F4kb-5Fblock-5Fwrites-5Fcan-5Fhold-5Fmore-5Fdata_&d=DQIBAg&c=n6-cguzQvX_tUIrZOS_4Og&r=LocN40EG5c3kHsbl3Fv0ZPr8zhg62ZqvBx3NsulACiM&m=dr4OWp4UEaiwuyOtPh5NiF0Ldhq9QU2CjkNd8uQ0XDk&s=oTpVQNNuRU2qfEzaxmW5wpM2n2nljWVDDfwAFO2N5RA&e= 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Michael Bergman
In reply to this post by Mike Gossett
On 2016-09-22 20:44, Mike Gossett wrote:
> my sales guy basically said the guarantee works as follows - they don't hit
> the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup
> shortfall.

*duh*

> The magic has to do with what they call "compaction" - writes <4KB (down to
> 512B/ea) are "compacted" into a single 4K block... this apparently adds to
> the std dedupe and compression they use.

Well, magic or not... the answer *must* be: it depends. Note the expressions
below "if", "can be" etc.

- - -
Currently Data ONTAP writes data to storage media in 4KB blocks
* An I/O or file has less than 4KB of data uses an entire 4KB

Adaptive compression can compress an 8KB I/O into a 4KB block on storage
* If the I/O is >50% compressible it is stored in a 4KB block – maximum
   storage efficiency ratio is 2:1

Storage space savings can be increased considerably if multiple small I/Os
or files can be stored together in a 4KB block

* Data ONTAP 9 does this with inline data compaction
* Turned on by default on AFF; can be turned on manually for hybrid systems



/M
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Klise, Steve-2
In reply to this post by Ray Van Dolson-3
I couldn’t find the document, but I believe NA has updated/upgraded dedupe/compression algorithms for CDOT9.  I think it would be “interesting” if you could compress already compressed images/videos.  They are typically not candidates for advanced storage features.  I am guessing at Insight (next week in US), they will show some of this.  I think the guarantee is more being in parity with other storage OEMs.

I think you would have a “discussion” with your Netapp/Sales team before you purchased anything.

http://www.netapp.com/us/forms/sales-inquiry/flash-3-4-5-promotion.aspx


On 9/22/16, 11:39 AM, "[hidden email] on behalf of Ray Van Dolson" <[hidden email] on behalf of [hidden email]> wrote:

    On Thu, Sep 22, 2016 at 02:32:30PM -0400, jordan slingerland wrote:
    > Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9.
    > They say they grantee it.  I have specifically asked what
    > stipulation.  In a VDI environment with linked clones, etc.  Sales
    > guy tells me none and even specifically says they can dedup
    > compressed video or audio (mp3) 4:1.
    >
    > I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or
    > what?
    >
    > Any comments welcome.
    >
    > --Jordan
   
    We'll be POC'ing one, so will see.  Am skeptical, but don't think it's
    like Pure Storage where they also factor in snapshots.
   
    Most likely the guarantee is a carrot, and if they fall short, they'll
    give you hardware to make up the difference.  Subsequent purchases,
    however won't give you the same guarantee so you'll need to buy more
    (again, this is also what Pure does).
   
    Other thoughts?
   
    Ray
    _______________________________________________
    Toasters mailing list
    [hidden email]
    http://www.teaparty.net/mailman/listinfo/toasters
   


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Michael Bergman
On 2016-09-22 20:54, Klise, Steve wrote:
> I think it would be
> “interesting” if you could compress already compressed images/videos.
> They are typically not candidates for advanced storage features.

You cannot really, since the Shannon Entropy of such files is pretty much
maxed out already.

<https://en.wikipedia.org/wiki/Entropy_(information_theory)>

But let's say you have a large no of smaller video files, sitting in an
ONTAP (WAFL) based system. This 4K block factor, the new "Compaction" will
give a positive effect.

Again: it depends

/M
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Brian Parent-2
In reply to this post by jordan slingerland-2
Those sales folks may be misunderstanding the details.
Ask to see the details of the guarantee, and read for yourself.

My experience with CDOT 8 on a AFF8020 was that video didn't compress/dedupe
hardly at all (12% savings from dedupe, 0% from compression).  Student home directories
are getting about 48% savings (close to 2:1).

I imagine certain data stores will have much better savings (like
lots of copies of VMs with identical OS images on them).

NetApp has a tool you can use to estimate savings of your own data,
though it caps the size of the analysis at 2TB (regardless of the size of the
volume being analyzed).  They call it SSET (Space Savings Estimation Tool).
I'm not sure whether that's specific to CDOT 8 or not.

Re:

> From: jordan slingerland <[hidden email]>
> Date: Thu, 22 Sep 2016 14:32:30 -0400
> Subject: AFF and 4:1 guaranteed efficiency
> To: [hidden email]
>
> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
> say they grantee it.  I have specifically asked what stipulation.  In a VDI
> environment with linked clones, etc.  Sales guy tells me none and even
> specifically says they can dedup compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what?
>
> Any comments welcome.
>
> --Jordan

> _______________________________________________
> Toasters mailing list
> [hidden email]
> http://www.teaparty.net/mailman/listinfo/toasters


--
Brian Parent
Information Technology Services Department
IT Infrastructure Operations Group
Workplace, Internal, Research, and Educational Platforms (WIRE) team
UC San Diego
(858) 534-6090
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

jordan slingerland-2
>>hose sales folks may be misunderstanding the details.
>>Ask to see the details of the guarantee, and read for yourself.

I agree.  Thanks for all the responses, much more inline with what I would think.  My data is not video or audio, but I did bring that up to the sales guys and very specifically they said they could get 4:1 on it.  In fact the only example dataset they had any hesitation on promising verbally was the output of /dev/urandom.  I call BS.  Don't get me wrong, ontap9 on AFF sounds like it contains a lot of great incremental improvement and I am  excited to try one out.  I just prefer realistic promises. 

--Jordan

On Thu, Sep 22, 2016 at 3:07 PM, Brian Parent <[hidden email]> wrote:
Those sales folks may be misunderstanding the details.
Ask to see the details of the guarantee, and read for yourself.

My experience with CDOT 8 on a AFF8020 was that video didn't compress/dedupe
hardly at all (12% savings from dedupe, 0% from compression).  Student home directories
are getting about 48% savings (close to 2:1).

I imagine certain data stores will have much better savings (like
lots of copies of VMs with identical OS images on them).

NetApp has a tool you can use to estimate savings of your own data,
though it caps the size of the analysis at 2TB (regardless of the size of the
volume being analyzed).  They call it SSET (Space Savings Estimation Tool).
I'm not sure whether that's specific to CDOT 8 or not.

Re:
> From: jordan slingerland <[hidden email]>
> Date: Thu, 22 Sep 2016 14:32:30 -0400
> Subject: AFF and 4:1 guaranteed efficiency
> To: [hidden email]
>
> Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They
> say they grantee it.  I have specifically asked what stipulation.  In a VDI
> environment with linked clones, etc.  Sales guy tells me none and even
> specifically says they can dedup compressed video or audio (mp3) 4:1.
>
> I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what?
>
> Any comments welcome.
>
> --Jordan

> _______________________________________________
> Toasters mailing list
> [hidden email]
> http://www.teaparty.net/mailman/listinfo/toasters


--
Brian Parent
Information Technology Services Department
IT Infrastructure Operations Group
Workplace, Internal, Research, and Educational Platforms (WIRE) team
UC San Diego
<a href="tel:%28858%29%20534-6090" value="+18585346090">(858) 534-6090


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Ray Van Dolson-3
On Thu, Sep 22, 2016 at 03:21:02PM -0400, jordan slingerland wrote:

> >>hose sales folks may be misunderstanding the details.
> >>Ask to see the details of the guarantee, and read for yourself.
>
> I agree.  Thanks for all the responses, much more inline with what I would
> think.  My data is not video or audio, but I did bring that up to the sales
> guys and very specifically they said they could get 4:1 on it.  In fact the
> only example dataset they had any hesitation on promising verbally was the
> output of /dev/urandom.  I call BS.  Don't get me wrong, ontap9 on AFF sounds
> like it contains a lot of great incremental improvement and I am  excited to
> try one out.  I just prefer realistic promises.
>
> --Jordan

As other's have mentioned, it's likely more about matching up with the
marketing of competitors.  Just about everyone has a data reduction
guarantee now with the same asterisks and strings attached.

Ray
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Arthur Alikulov
As it was mentioned before, you can use SSET to measure efficiency of NetApp technologies with your dataset. Last version of SSET support inline data compaction. You can download SSET from Utility Toolchest http://mysupport.netapp.com/tools/index.html

And here is some information about the guarantee:

"The total effective storage capacity ratio is calculated based on compression, deduplication, compaction, snapshots and clones on the dataset.

Although all volumes (new and migrated ones) need to be thinly provisioned, thin provisioning is not included in measuring the total effective storage capacity ratio."

"The guarantee is for all workloads running on All Flash FAS systems. The AFF Systems can be added to the new cluster provided they adhere to the NetApp Storage Efficiency technologies. However, already compressed data and encrypted data is excluded from the guarantee. "


Just ask sales to show you program Terms and Conditions.


On Thu, Sep 22, 2016 at 10:27 PM, Ray Van Dolson <[hidden email]> wrote:
On Thu, Sep 22, 2016 at 03:21:02PM -0400, jordan slingerland wrote:
> >>hose sales folks may be misunderstanding the details.
> >>Ask to see the details of the guarantee, and read for yourself.
>
> I agree.  Thanks for all the responses, much more inline with what I would
> think.  My data is not video or audio, but I did bring that up to the sales
> guys and very specifically they said they could get 4:1 on it.  In fact the
> only example dataset they had any hesitation on promising verbally was the
> output of /dev/urandom.  I call BS.  Don't get me wrong, ontap9 on AFF sounds
> like it contains a lot of great incremental improvement and I am  excited to
> try one out.  I just prefer realistic promises.
>
> --Jordan

As other's have mentioned, it's likely more about matching up with the
marketing of competitors.  Just about everyone has a data reduction
guarantee now with the same asterisks and strings attached.

Ray
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: AFF and 4:1 guaranteed efficiency

Steiner, Jeffrey
In reply to this post by jordan slingerland-2

I honestly don't know the terms of the guarantee at this time, but here's a few observations on this topic of efficiencies and guarantees:

 

1)      Always read the remedies for the guarantees. They seem mostly the same to me across the vendors.

2)      There's a reason for that - the efficiency rates are more or less the same for any of the various products on the market.

 

I've noted a few competitors who tend to cite 4:1, 5:1, or 6:1 without any qualification. On occasion, I've heard of one of them having to send lots of extra shelves of disks to make up for the shortfall, but most of the time the answer is "you had a nonstandard configuration" which is a different way of saying "too bad".

 

Most customers I know do see about 4:1 efficiencies when they properly implement an ONTAP 9 system, but obviously there will be variations. VDI is usually pretty easy because the VMDK's deduplicate extremely well. I've seen databases that were mostly empty and compressed and compacted 80:1. If someone is in the habit of storing loads of zeros in their databases, we'd blow that 4:1 out of the water, but that's a fringe case. It's due to compaction. All those 8K blocks are composed of a small header and trailer sandwiching some zeros. The zeros compress mostly out of existence and compaction will then store the residual header/trailer data into a tiny space. Fringe case, but not out of the question. On the flipside, I ran into a customer who was getting zero compression because their DBA's had enabled encryption across the board without telling the storage team. They were getting no savings at all. Still, 4:1 is about the typical level in my experience with ONTAP.

 

There's also more than just compression and efficiency. I know one customer who took a 60TB database and cloned it 40 times. That's 40:1 right there without compression or deduplication. You could do the same thing on some competing storage arrays and the clone savings would be done via deduplication, whereas we do it via direct cloning of a snapshot. It's quicker, but it's not classified as "efficiency" because it's not done via deduplication or cloning. It sometimes isn't getting the deserved credit as an efficiency option.

 

It's also worth mentioning the big picture. For one, the cost of SSD's keeps dropping, which erodes the value of efficiency. The cost savings are less and less all the time. Maybe some product out there does have 4:1 efficiency on a certain data set while ONTAP only had 3:1, but how much money does that really save, and what are the consequences? There are more features and needs beyond space savings. For example, some products use larger compression block sizes and get better efficiency. You could even force this with ONTAP via secondary compression which uses a larger block size and therefore delivers better efficiency levels.  Unfortunately, with data sets like VDI and databases that involve small block overwrites you will see a latency hit. So, better efficiency with worse performance.

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of jordan slingerland
Sent: Thursday, September 22, 2016 1:33 PM
To: [hidden email]
Subject: AFF and 4:1 guaranteed efficiency

 

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Jeffrey Mohler
In reply to this post by Mike Gossett
Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.

 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <[hidden email]> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/ mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Michael Bergman
On 2016-09-23 09:09, Jeffrey Mohler wrote:
> Compaction in our testing can be good, really good...an additional 20% in
> many of our test data sets on top of everything else. (SSET diagnosis)

I'm not too surprised (w.r.t. Yahoo). I expect much the same for us here

> However, there are limitations on how you get data into AFF to get
> compaction...IE, you cant snapmirror it.
>
> It must transfer, as far as we're told today, via a host/file based
> migration. Be thinking bout this when you consider an AFF migration. It must
> be done outside on ONTAP.
>
> We have pushed to get this [having to file migrate via protocol] fixed
> via  a scanner/etc that reads and relays out  the file based structure.

We = Yahoo? You've requested NetApp to implement a bg WAFL scanner which
will runt through the whole FlexVol (vol by vol) and compact it?
While this will prob be fine (-ish) for AFF I don't think that scanner will
be pleasant, even viable, to run on HyA based systems (spinning disk). If
they do this my hunch is it will be limited to run in AFF only...

/M

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Steiner, Jeffrey
In reply to this post by Jeffrey Mohler
A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 

On 23 Sep 2016, at 02:10, Jeffrey Mohler <[hidden email]> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.

 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <[hidden email]> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/ mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Francis Kim
Strange how compaction appears to work with vol move but not with snap mirror. 

.

On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <[hidden email]> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 

On 23 Sep 2016, at 02:10, Jeffrey Mohler <[hidden email]> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.

 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <[hidden email]> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/ mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

jordan slingerland-2
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged.  Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted?  If the vol move is within the same controller AFF the lack of backward compatibility is not a concern.  Just a though, I don't know. 

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <[hidden email]> wrote:
Strange how compaction appears to work with vol move but not with snap mirror. 

.

On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <[hidden email]> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 

On 23 Sep 2016, at 02:10, Jeffrey Mohler <[hidden email]> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.

 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <[hidden email]> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/ mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: AFF and 4:1 guaranteed efficiency

Steiner, Jeffrey

I would guess that snapmirror is currently looking at the physical blocks on disk, whereas a vol-move is a step up the chain and it can look at block that are actually a block within a block.

 

I'll check with engineering and report back...

 

From: jordan slingerland [mailto:[hidden email]]
Sent: Friday, September 23, 2016 10:26 AM
To: NGC-fkim-berkcom.com <[hidden email]>
Cc: Steiner, Jeffrey <[hidden email]>; [hidden email]
Subject: Re: AFF and 4:1 guaranteed efficiency

 

My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged.  Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted?  If the vol move is within the same controller AFF the lack of backward compatibility is not a concern.  Just a though, I don't know. 

 

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <[hidden email]> wrote:

Strange how compaction appears to work with vol move but not with snap mirror. 

 

.


On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <[hidden email]> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 

 

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 


On 23 Sep 2016, at 02:10, Jeffrey Mohler <[hidden email]> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

 

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.

 

 

_________________________________

[hidden email]

Tech Yahoo, Storage Architect, Principal

(831)454-6712
YPAC Gold Member

Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris

 

 

On Friday, September 23, 2016 1:53 AM, Mike Gossett <[hidden email]> wrote:

 

my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

 

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.

 

 

 

On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:

Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan


______________________________ _________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/ mailman/listinfo/toasters

 

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

 


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AFF and 4:1 guaranteed efficiency

Tim McCarthy
In reply to this post by jordan slingerland-2
I vaguely remember running a test on an ONTAP 9 simulator.

What I found was that if the destination aggr had compaction enabled, then snapmirror into that aggr would also have compacted data.
This DOES NOT work with XDP (snapvault & version-flexible snapmirror)



--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack


NetApp - In partnership with Alpine Testing SolutionsNetApp Certified Data Administrator, ONTAPNetApp Certified Implementation Engineer - SAN Specialist, ONTAPNetApp Certified Storage Installation Engineer, ONTAPNetApp Certified Implementation Engineer - Data Protection Specialist



NetApp Candidate ID: NETAPP00041276
        FlexPod Design: Oct 2015 - Jan 2018, S0N62WE1BMVEYF3M
FlexPod Implementation: Oct 2015 - Jan 2018, JH3QJT4KLEQ41HPH

           RHCE6 110-107-141 

On Fri, Sep 23, 2016 at 11:25 AM, jordan slingerland <[hidden email]> wrote:
My understanding is that both are block based operations so it makes sense to me that the blocks would be put down on disk unchanged.  Perhaps it has something to do with the fact that you may want to reverse the snapmirror back to the system that presumably does not have compaction enabled or support the feature, so the blocks are left uncompacted?  If the vol move is within the same controller AFF the lack of backward compatibility is not a concern.  Just a though, I don't know. 

On Fri, Sep 23, 2016 at 10:57 AM, Francis Kim <[hidden email]> wrote:
Strange how compaction appears to work with vol move but not with snap mirror. 

.

On Sep 23, 2016, at 4:39 AM, Steiner, Jeffrey <[hidden email]> wrote:

A vol-move operation will cause compaction to happen. I know it's not ideal but at least its internal. Obviously a scanner is preferred but functionally it would be a lot like a vol move. 

Likewise if you use FLI to import a LUN that will trigger all the efficiency features during the import. 

Sent from my mobile phone. 

On 23 Sep 2016, at 02:10, Jeffrey Mohler <[hidden email]> wrote:

Compaction in our testing can be good, really good...an additional 20% in many of our test data sets on top of everything else.  (SSET diagnosis)

However, there are limitations on how you get data into AFF to get compaction...IE, you cant snapmirror it.

It must transfer, as far as we're told today, via a host/file based migration.   Be thinking bout this when you consider an AFF migration.  It must be done outside on ONTAP.

We have pushed to get this fixed via a scanner/etc that reads and relays out the file based structure.

 
_________________________________
[hidden email]
Tech Yahoo, Storage Architect, Principal
Twitter: @PrincipalYahoo
CorpIM:  Hipchat & Iris



On Friday, September 23, 2016 1:53 AM, Mike Gossett <[hidden email]> wrote:


my sales guy basically said the guarantee works as follows - they don't hit the 4:1 target, they buy you whatever amount of shelf\ssd required to makeup shortfall.  

The magic has to do with what they call "compaction" - writes <4KB (down to 512B/ea) are "compacted" into a single 4K block... this apparently adds to the std dedupe and compression they use.



On Thu, Sep 22, 2016 at 1:32 PM, jordan slingerland <[hidden email]> wrote:
Sales guys are promising me 4:1 dedup ratios on an AFF with ontap9. They say they grantee it.  I have specifically asked what stipulation.  In a VDI environment with linked clones, etc.  Sales guy tells me none and even specifically says they can dedup compressed video or audio (mp3) 4:1.

I have A LOT of trouble believing that.  So, 4:1 dedup guaranteed or what? 

Any comments welcome.

--Jordan

______________________________ _________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/ mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters



_______________________________________________
Toasters mailing list
[hidden email]
http://www.teaparty.net/mailman/listinfo/toasters
12
Loading...