[Veritas-bu] Backups slow to a crawl

Jeff McCombs jeffm at nicusa.com
Wed Mar 23 07:51:14 CST 2005


Gurus,

    NB 5.0 MP4, single combination media/master server, Solaris 9. Overland
Neo 2000 26-slot 2 drive DLT.

    I'm noticing that for some reason or another, all of my client backups
have slowed to a _crawl_. A _cumulative_ (!) backup of local disk on a Sun
V100 is taking somewhere on the order of 2 hours at this point, and with
over 40 systems, I'm blowing past my window  consistently.

    I'm not quite sure what's going on here, but as I sit and watch the
output from 'iostat', I'm noticing that rmt/1 (the 2nd drive in the Neo) is
fluxuating between 100% busy, with kw/s at close to zero, and busy @ 1-15%
and kw/s up into the 1000's.

    rmt/0 seems to be fine, kw/s sits consistently up in the 1.8-2K range,
while busy is anywhere from 2% - 30% on average. My other disks aren't
working hard, CPU isn't loaded and I've got plenty of memory.

    The policy I'm using allows for multiple datastreams, no limits on jobs,
and most schedules allow for an MPX of 2. I'm backing up ALL_LOCAL_DRIVES on
all clients, and I'm not using any NEW_STREAM directives. I'm not seeing any
errors on the media either.

    Can anyone shed some light on what might be happening here? Am I looking
at a drive that might be having some problems, or am I barking up the wrong
tree, and it's something else entirely?

    A small sample of iostat output covering the affected devices is below.

sample (extra disks removed from putput);
root at backup(pts/1):~# iostat -nx 1 100
                    extended device statistics
    r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
    0.0    4.1    0.0  252.2  0.0  0.0    0.0    5.9   0   2 rmt/0
    0.0    4.6    0.0  278.4  0.0  0.1    0.0   27.3   0  12 rmt/1

    0.0    4.1    0.0  252.3  0.0  0.0    0.0    5.9   0   2 rmt/0
    0.0    4.6    0.0  278.4  0.0  0.1    0.0   27.3   0  12 rmt/1

    0.0   33.0    0.0 2076.4  0.0  0.2    0.0    5.8   0  19 rmt/0
    0.0    2.0    0.0  125.8  0.0  1.0    0.0  490.0   0  98 rmt/1

    0.0   38.0    0.0 2394.0  0.0  0.2    0.0    5.4   0  21 rmt/0
    0.0    8.0    0.0  504.0  0.0  1.0    0.0  124.9   0 100 rmt/1

    0.0   27.0    0.0 1701.1  0.0  0.2    0.0    6.5   0  17 rmt/0
    0.0    2.0    0.0  126.0  0.0  1.0    0.0  499.9   0 100 rmt/1

    0.0   33.0    0.0 2078.9  0.0  0.2    0.0    5.3   0  18 rmt/0
    0.0    0.0    0.0    0.0  0.0  1.0    0.0    0.0   0 100 rmt/1

    0.0   16.0    0.0 1008.0  0.0  0.1    0.0    6.2   0  10 rmt/0
    0.0   13.0    0.0  819.0  0.0  0.6    0.0   48.4   0  63 rmt/1

    0.0   40.0    0.0 2520.1  0.0  0.2    0.0    5.9   0  24 rmt/0
    0.0    0.0    0.0    0.0  0.0  1.0    0.0    0.0   0 100 rmt/1

    0.0   33.0    0.0 2078.9  0.0  0.2    0.0    5.3   0  18 rmt/0
    0.0   10.0    0.0  630.0  0.0  1.0    0.0   99.9   0 100 rmt/1


-- 
Jeff McCombs                 |                                    NIC, Inc
Systems Administrator        |                       http://www.nicusa.com
jeffm at nicusa.com             |                                NASDAQ: EGOV
Phone: (703) 909-3277        |        "NIC - the People Behind eGovernment"
--
    "From space, I saw the fragile blue earth without borders."
        - Onizuka.





More information about the Veritas-bu mailing list