SOC limit (was Re: vxva vs. ODS (was Re: RSM 2000 list) )

[Matt Harris]

There is a warning of not to attach two SSA to one X1057A Soc Card with
two X595A daughter cards
for performance reasons. This was published in the 1.3 Version of SUN
VXVM. There are two Manuals
you recieve with your SSA, The Big user manual and the thin Hardware
manual. In the beginning of the
thin hardware manual, there is a series of diagrams showing various
connectivity options. One is the
above configuration with a note of not to attach two SSA's to one SOC
card for performance reasons.
I believe the  diagrams carried over to version 2.0, 2.1, 2.1.1 and 2.3
of the SUN SSA manuals.

Matt Harris

ft@angels.att.com wrote:

> On Sun, 1 Jun 1997, Doug Hughes wrote:
> > Note for Sun people: It would probably be a good idea to make a note
>
> > about not supporting non SSA disks for RAID-5 and RAID-0 to the web
> page
> > without a purchase of the full VxVM release.  He does have a
> legitimate
> > complaint about not being warned before hand (until the product was
> > in hand - which then he could have found out in Chapter1)
>
> I have a tangentially related issue regarding proper advertising,
> which is
> about two years old - but I wonder if anyone else was bitten by this.
>
> In the early days of our use of the SSA100 series, we had few enough
> (only
> about 60) that the distribution of arrays to hosts did not require
> "doubling up" arrays on SOCHAs (e.g. we only used one port on each
> host
> card).  When we moved to a much larger configuration, however, we
> became
> slot limited on the server side, which forced us to utilize both FC
> ports
> on the SOCHAs if we wanted to meet our throughput goals and maintain a
>
> reasonable processing to i/o ratio, dictated by our application needs.
>
> To this point, we had worked fairly hard to understand the performance
>
> characteristics of the arrays, and were able to sustain 18.5 to 20
> MB/s
> from each array for large sequential reads, which dominate this
> particular
> application.  Our primary goal was sustaining the highest possible
> sequential scan rate of our filesystems.
>
> Given that the SOCHA was touted as having "two 25 MB/s full duplex"
> fibre
> channel modules; we naively thought that, given our empirical ability
> to
> sustain roughly 19 MB/s with one SSA112; we would be able to sustain
> twice
> that with two arrays on a single SOCHA.  In fact, the "SPARCstorage
> Array
> Architecture" whitepaper from 1996 says:
>
> "Sun developed a Serial Optical Channel (SOC) ASIC that can service
> two
> full-duplex, SBus to Fibre Channel conversions at 50 MB/second."
>
> This is true in some respects, but is missing one vital detail which
> governs real life.  It was not long after we started testing that we
> discovered that the combined traffic of two SSA112's on a single SOCHA
> was
> limited to about 27 MB/s. When testing each port alone, we got our
> typical
> 19 MB/s; but when sustaining read traffic to both arrays, it dropped
> to
> about 13.5 MB/s per array (at least the balance was fair).
>
> Upon inquiry, we were told that this was indeed a limitation of the
> SOC
> ASIC, which was only able to queue 255 pending i/o requests for both
> ports.  Given the typical SSA svc_t under our conditions (for 128k
> reads),
> this translated to about 214.5 IOPS or 26.8 MB/s combined for both
> ports.
> I dredged all the product information I could find and NEVER saw a
> reference to this in any of the whitepapers or product literature.  Is
>
> anyone else out there aware of this limit being publicised anywhere
> that I
> should have seen it?
>
> Obviously, there was no way to overcome this limit (which effectively
> limited the filesystem throughput that the enterprise servers could
> sustain), and we had to live with the degraded rate.  It was not all
> that
> bad, but I must say that I was miffed that this was not in black and
> white
> before we made our purchase decision.  Our anticipation of being able
> to
> sustain about 40 MB/s to each SBus slot was one of the factors that
> led us
> to choose FC over USCSI; but we had been duped.  The lesson learned
> was
> indeed to test before deciding.
>
> I was very glad to see this very fact outlined clearly in Brian Wong's
>
> recent book.  In fact, he says: "In light of these realities, the
> second
> port on each FC host adapter is primarily useful for archival storage
> connectivity."  However, this was a bit late for us.
>
> In any case; I think I would warn Sun to be more forthcoming with
> limits
> such as this (or at least the technical data required to forecast
> them).
> Even our SE's were not aware of this practical limit until after we
> verified the results through testing, and they subsequently dug into
> the
> engineering groups for confirmation.  Had they published more
> technical
> details on the SOC ASIC (including the 255 command queue length) I
> could
> have worked this out myself.
>
> Live and learn.
>
> Fred
>
> --
> Fred True                               "My name is Ozymandias, King
> of Kings:
> AT&T World Class Database Development    Look on my works, ye Mighty,
> ft@angels.att.com                        and despair!"
>                                                 -P. B. Shelley



--
Matthew Harris
matt@microweb.com