A presentation at LinuxCon + ContainerCon North America 2016 in in Toronto, ON, Canada by erik riedel
Scalable Storage with Ethernet Disk Drives – Kine7c on the Move Erik Riedel EMC erik.riedel@emc.com
from flickr/Blude, floppy disks for breakfast from flickr/oskay, Raw materials We Build Big Clouds From Li1le Pieces from flickr/erinhillaw, Math/Physics Bike Rack from flickr/jurvetson, SGI Detritus
Background • EMC Object Storage – Centera – since April 2002 – Atmos – since November 2008 – ECS – Elas7c Cloud Storage – since June 2014 • over $3b object storage sold across three products • over 3 EB exabytes of deployed capacity • all three are Linux-‐based ethernet object stores
since 2002
SCALE
srvs encs Atmos ECS disks TB drv TB rack gen1 16 16 240 1TB 240TB gen2 6 24 360 2TB 720TB gen3 8 8 480 4TB 1.9PB gen4 8 8 480 6TB 2.9PB gen5 8 8 480 8TB 3.8PB gen5+ 8 8 784 8, 10TB 6.3PB gen6 4 7 588 8, 12TB 7.0PB Gen1 (2008) 1TB mo Code 1 Front (tray pulled out) Gen3 (2012) 3,4,6TB Gen2 (2010) Gen4 (2014) 2,3TB 6TB Gen5 (2015) 8TB Gen5+ (2016) 8TB Gen6 (2017) 12TB
Mechanicals drawers modular drawers sleds back-‐to-‐back trays modular
Density 2012 Disks (raw) @ 3TB Updated from “Long-‐Term Storage”, presented at Library of Congress Workshop in September 2012 Disks (protected) Racks @ 480 disks 5 PB 1,700 disks 2,700 disks 6 racks 20 PB 6,700 disks 11,000 disks 23 racks 50 PB 17,000 disks 27,000 disks 56 racks 2014 5 PB Disks (raw) @ 6TB 830 disks Disks (protected) Racks @ 480 disks 1,300 disks 3 racks 20 PB 3,300 disks 5,300 disks 12 racks 50 PB 8,300 disks 13,000 disks 28 racks 2016 5 PB Disks (raw) @ 10TB 500 disks Disks (protected) Racks @ 780 disks 700 disks 1 rack 20 PB 2,000 disks 2,800 disks 4 racks 50 PB 5,000 disks 7,000 disks 9 racks
Scale Out RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Empty Rinjin 4 Blade Blank Blank Blank Blank Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 480 TB/4n60d U400 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 960 TB/4n120d U900 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Not Used 1.9 PB/4n240d U1900 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Not Used 3.8PB/8n480d U4000 17 PB petabytes 48 nodes 3,840 disks
Scale Out Further RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Empty Rinjin 4 Blade Blank Blank Blank Blank Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 320 TB U400 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 640 TB U400T RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Not Used 1.9 PB U2000 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Not Used 3.8PB U4000 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Pikes Peak 70 Disk Pikes Peak 70 Disk Pikes Peak 70 Disk Pikes Peak 70 Disk Pikes Peak 70 Disk Pikes Peak 70 Disk Pikes Peak 70 Disk Pikes Peak 70 Disk Not Used 4.5PB D4500 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Pikes Peak 98 Disk Pikes Peak 98 Disk Pikes Peak 98 Disk Pikes Peak 98 Disk Pikes Peak 98 Disk Pikes Peak 98 Disk Pikes Peak 98 Disk Pikes Peak 98 Disk Not Used 6.2PB D6200
FLEXIBILITY
Experiments in Flexibility RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Empty Rinjin 4 Blade Blank Blank Blank Blank Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 320 TB U400 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 1 Blade Rinjin 4 Blade Blank Blank Blank Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 400 TB U400E RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 2 Blade Rinjin 4 Blade Blank Blank Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 480 TB U480E RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Voyager 10 Disk Not Used 640 TB U400T RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Voyager 30 Disk Not Used 1.9 PB U2000 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 NILE DENSE GbE 10 GbE 10 GbE Rinjin 4 Blade Rinjin 4 Blade Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Voyager 60 Disk Not Used 3.8PB U4000 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 ECS FLEX 10 GbE 10 GbE GbE Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade Empty Empty Empty Empty Empty Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk 2.0 PB/24n F2000-‐24 5 4 3 2 1 10 GbE 24p 1 GbE 48p Rinjin 4 Blade Kine7c 12d 96 TB/4n F100 7 6 5 4 3 2 1 10 GbE 24p 10 GbE 24p 1 GbE 48p Rinjin 4 Blade Kine7c 12d Kine7c 12d 192 TB/4n F200 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 10 GbE 24p 10 GbE 24p 1 GbE 48p Rinjin 4 Blade Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d Kine7c 12d 960 TB/4n F1000
40 turtle 39 rabbit 38 leh right 37 lehi murray 36 layton 35 34 33 name hardware logan one orem ogden provo sandy eight 31 29 seven 27 25 six 23 21 Experiment – SAS Switching #drives drive type port A port B Voyager DAE 60 Muskie SATA 2TB leh 9 right 9 two Voyager DAE 60 Muskie SATA 2TB leh 10 right 10 three Voyager DAE 60 Muskie SATA 2TB leh 11 right 11 four Voyager DAE 60 Muskie SATA 2TB leh 12 right 12 five Voyager DAE 60 Muskie SATA 2TB leh 13 right 13 six Voyager DAE 60 Muskie SATA 2TB leh 14 right 14 seven Voyager DAE 60 Muskie SATA 2TB leh 15 right 15 eight Voyager DAE 60 Muskie SATA 2TB leh 16 right 16 five 19 17 four 15 13 three 11 09 two 07 05 one 03 01 EMPTY leh right 2012
Experiment – SAS Switching 1:120 Disk Node with HA SAS and 2 Service/Management Nodes SERVER SERVER Voy 1A EA PA LCC A PB SERVER SAS 16 Port SVC Node PA SERVER Voy 1B EA PA LCC A EB PB SVC Node Voy 2A EA PA LCC A EB PB Voy 2B EA PA LCC A EB PB Voy 3A EA PA LCC A EB PB Voy 3B EA PA LCC A EB PB Voy 4A EA PA LCC A EB PB Voy 4B EA LCC A EB PB EB SAS 16 Port Rack Config & SAS Wiring Diagram (A3-‐480/LightsOut Config) KEY Expansion/Secondary Primary 2012
Experiment – SAS Zoning Entry Point = 4 x 15 disk nodes = 60 disks 2013 2 x 15 disk nodes illustrated below 60 Disk JBOD, “VOYAGER” 4 Server “PHOENIX” Chassis Max = 24 x 15 disk nodes = 360 disks
Experiment – Kine7c Application File System DB POSIX File System Volume Manager Driver FC Storage Server RAID Battery Backed RAM CACHE SAS Devices SAS Interface SMR, Mapping Cylinder, Head, Sector Drive HDA From Seagate material “Kine7c Open Storage -‐ Enabling Break-‐through Economics in Scale-‐out Object Storage” 2014
Experiment – Kine7c PRODUCT SHEET • Newisys® EDA-4605 4U-60 3.5” Ultra Dense Ethernet Drive Storage Array Newisys EDA-‐4605 Enclosure – – – 60-‐disk dual 10 GbE controllers 4x 10 GbE uplinks Product Highlights Product Features • Seagate Kinetic Ethernet drive – 4TB in October 2014 (2x 1 GbE net) – 8TB in September 2015 (2x 2.5 GbE net) 2014
PRESENT DAY
Storage w/ Kine7c -‐ Prototype 12n504d 240 cores 4,032 TB raw 160 Gbps 4.0PB/12n F4000-‐12 RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 ECS FLEX 10 GbE 10 GbE GbE Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk Titan/Kine7c 84 Disk RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 ECS FLEX 10 GbE 10 GbE GbE Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk RU 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 ECS FLEX 10 GbE 10 GbE GbE Rinjin 4 Blade Rinjin 4 Blade Rinjin 4 Blade • Parts List – – – Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk • Titan/Kine7c 14 Disk Titan/Kine7c 14 Disk 2016 – – – – – – – 9x Rinjin servers (36 nodes) 2x 10 GbE per node SFP+ 6x 10 GbE data switches (Arista 64-‐port SFP+) 3x 1 GbE mgmt switches (Arista 48-‐port Cat6) 18x Titan enclosures (dual controller, 4x 10 GbE uplinks) 3x loose Titan enclosures 84 * 6 + 14 * 15 = 714 714x Kine7c/8TB drives SFP+ twinax cables (data) Cat6 cables (mgmt) Adding – – 6x 40 GbE data switches (Arista 32-‐port QSFP+) upda7ng nodes to 2x 40GbE
PERFORMANCE
Enclosure + Server Bandwidth drives total bandwidth enclosure physical enclosure bandwidth enclosure server #drives gen4 voyager phoenix 60 8,400 MB/s 4x 3G SATA 1 980 MB/s gen5 voyager rinjin 60 8,400 MB/s 8x 6G SAS 1 4,100 MB/s gen5+ pikes peak rinjin 98 13,720 MB/s 8x 12G SAS 1 5,800 MB/s gen6 7tan rinjin 84 10,080 MB/s 4x 10G ethernet 4 4,000 MB/s #nodes *all bandwidths are micro-‐benchmark “OS-‐level” measurements, end-‐to-‐end • ethernet-‐based enclosure aggregate performance is comparable to SAS-‐based enclosure • using 2.5 GbE drive links and 10 GbE enclosure uplinks
MANAGEABILITY
from flickr/purplemarish, Broken hard drive?
Topology Map – getrackinfo provo-vanilla:~ # getrackinfo -a Node private Node Ip Address Id Status =============== ====== ====== 192.168.219.1 1 MA 192.168.219.2 2 SA 192.168.219.3 3 SA 192.168.219.4 4 SA 192.168.219.5 N/A noLink 192.168.219.6 N/A noLink 192.168.219.7 N/A noLink 192.168.219.8 N/A noLink 192.168.219.9 9 O 192.168.219.10 10 O 192.168.219.11 11 O 192.168.219.12 12 O 192.168.219.13 13 O 192.168.219.14 14 O 192.168.219.15 15 O 192.168.219.16 16 O 192.168.219.17 N/A noLink 192.168.219.18 N/A noLink 192.168.219.19 N/A noLink 192.168.219.20 N/A noLink 192.168.219.21 N/A noLink 192.168.219.22 N/A noLink 192.168.219.23 N/A noLink 192.168.219.24 N/A noLink Status: M - Master, S - Slave E – Epoxy I - Initializing, U - Updating, A P - On, O - Off Public Mac ================= 00:1e:67:9f:01:96 00:1e:67:9f:01:a2 00:1e:67:9e:ff:9e 00:1e:67:9e:ff:7f N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Active Ip Address ================= 10.249.248.111 10.249.248.112 10.249.248.113 10.249.248.114 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A RMM Mac ================= 00:1e:67:69:29:8f 00:1e:67:69:28:72 00:1e:67:69:29:99 00:1e:67:69:29:5f N/A N/A N/A N/A 00:1e:67:69:28:4f 00:1e:67:69:29:6d 00:1e:67:69:29:3f 00:1e:67:69:27:2c 00:1e:67:69:28:8c 00:1e:67:69:28:3f 00:1e:67:69:29:8a 00:1e:67:69:29:8c N/A N/A N/A N/A N/A N/A N/A N/A Ip Address ================= 10.249.248.101 10.249.248.102 10.249.248.103 10.249.248.104 N/A N/A N/A N/A 10.249.248.109 10.249.248.110 10.249.248.111 10.249.248.112 10.249.248.113 10.249.248.114 10.249.248.115 10.249.248.116 N/A N/A N/A N/A N/A N/A N/A N/A Node Name ========= provo-vanilla sandy-vanilla orem-vanilla ogden-vanilla N/A N/A N/A N/A boston-vanilla chicago-vanilla houston-vanilla phoenix-vanilla dallas-vanilla detroit-vanilla columbus-vanilla austin-vanilla N/A N/A N/A N/A N/A N/A N/A N/A
Topology Map – getrackinfo – details provo-vanilla:~ # getrackinfo -a Node private Node Ip Address Id Status =============== ====== ====== 192.168.219.1 1 MA 192.168.219.2 2 SA 192.168.219.3 3 SA 192.168.219.4 4 SA 192.168.219.5 N/A noLink 192.168.219.6 N/A noLink 192.168.219.7 N/A noLink 192.168.219.8 N/A noLink 192.168.219.9 9 O 192.168.219.10 10 O 192.168.219.11 11 O 192.168.219.12 12 O 192.168.219.13 13 O 192.168.219.14 14 O 192.168.219.15 15 O 192.168.219.16 16 O 192.168.219.17 N/A noLink 192.168.219.18 N/A noLink 192.168.219.19 N/A noLink 192.168.219.20 N/A noLink 192.168.219.21 N/A noLink 192.168.219.22 N/A noLink 192.168.219.23 N/A noLink 192.168.219.24 N/A noLink Status: M - Master, S - Slave E – Epoxy I - Initializing, U - Updating, A P - On, O - Off Public Mac ================= 00:1e:67:9f:01:96 00:1e:67:9f:01:a2 00:1e:67:9e:ff:9e 00:1e:67:9e:ff:7f N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Active RMM Ip Address Mac Ip Address Node Name ================= ================= ================= ========= 10.249.248.111 00:1e:67:69:29:8f 10.249.248.101 provo-vanilla 10.249.248.112 00:1e:67:69:28:72 10.249.248.102 sandy-vanilla 10.249.248.113 00:1e:67:69:29:99 10.249.248.103 orem-vanilla 10.249.248.114 00:1e:67:69:29:5f 10.249.248.104 ogden-vanilla N/A N/A N/A N/A provo-vanilla:~ # getrackinfo -v N/A N/A N/A N/A ===================================================== N/A N/A N/A N/A NodeName : provo-vanilla N/A N/A N/A N/A Node Id : 1 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A & IP) N/A N/A Interfaces(MAC N/A N/A N/A N/A —————————-N/A N/A N/A N/A public : 00:1e:67:9f:01:96 10.249.248.111/21 N/A N/A N/A N/A private : 00:1e:67:69:29:8b 192.168.219.1/24 N/A N/A N/A N/A private ipmiN/A : 00:1e:67:69:29:8d 192.168.219.101/24 N/A N/A N/A private.4(NAN) : 00:1e:67:69:29:8b 169.254.186.1/16 N/A N/A N/A N/A remote ipmi N/A : 00:1e:67:69:29:8f 10.249.248.101/21 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Network Services N/A N/A N/A N/A ———————-N/A N/A N/A N/A NTP Configuration: N/A N/A N/A N/A server: 10.254.140.21 10.254.140.22 N/A N/A N/A N/A DNS Configuration: domain: search: server: sea.lab.emc.com corp.emc.com emc.com 192.168.219.254 10.6.149.11
ECS Hardware Abstraction Layer = provo-silver Disks(s): SCSI Device Block Device —————- —————-/dev/sg2 /dev/sdc /dev/sg3 /dev/sdd /dev/sg4 /dev/sde /dev/sg5 /dev/sdf /dev/sg6 /dev/sdg /dev/sg7 /dev/sdh /dev/sg8 /dev/sdi /dev/sg9 /dev/sdj /dev/sg10 /dev/sdk /dev/sg11 /dev/sdl /dev/sg12 /dev/sdm /dev/sg13 /dev/sdn /dev/sg14 /dev/sdo /dev/sg15 /dev/sdp /dev/sg16 /dev/sdq /dev/sg17 /dev/sdr … total: 60 slot numbers (physical) Enclosure —————/dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 /dev/sg18 Partition Name —————————————————ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA ECS:object:cGSk6IN2REyhmawe8r9xXA Slot —-A00 A01 A02 A03 A04 A05 A06 A07 A08 A09 A10 A11 B00 B01 B02 B03 physical serial numbers (as shown on the drive label) resilient to /dev/sd* and /dev/sg* changes Serial Number SMART —————————- ———NAHJ3TDY GOOD NAH08A3X GOOD NAHJWW9X GOOD NAHKNU2Y GOOD NCG555KS GOOD NAHKNXSY GOOD NAHJEJUY GOOD NCG4VL8S SUSPECT NAHKDVBY GOOD NCG335PS GOOD NAHH72KX GOOD NAHKP0AY GOOD NCG4ZWGS GOOD NAHHU5RY GOOD NAHHMTXY GOOD SMART inquiries for NCG4PEGS GOOD all drives (GOOD, BAD, SUSPECT)
ECS Basic Health Checks = lehi-silver Disks(s): SCSI Device Block Device —————- —————-/dev/sg3 /dev/sdd /dev/sg4 /dev/sde /dev/sg5 /dev/sdf /dev/sg6 /dev/sdg /dev/sg7 /dev/sdh /dev/sg8 /dev/sdi /dev/sg9 /dev/sdj /dev/sg10 /dev/sdk /dev/sg11 /dev/sdl /dev/sg12 /dev/sdm /dev/sg13 /dev/sdn /dev/sg14 /dev/sdo /dev/sg15 /dev/sdp /dev/sg16 /dev/sdq /dev/sg17 /dev/sdr /dev/sg18 /dev/sds … total: 60 resilient to /dev/sd* and /dev/sg* changes Enclosure —————/dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 /dev/sg19 Partition Name —————————————————ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ ECS:object:IrHy+LjnQ+mXJK7ekqKwPQ Slot —-A00 A01 A02 A03 A04 A05 A06 A07 A08 A09 A10 A11 B00 B01 B02 B03 Serial Number —————————NAHWHUMY NCGBN24S NAHWKLSY NAHWHK9Y NAHW9ELY NAHWHN5Y NAHWGZPY NAHVXAMY NAHSURUY NAHWW20X NAHWUYYX NAHW8BUY NAHWHEXY NAHWH54Y NAHV9X9X NAHWLL7X SMART ———GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD SUSPECT GOOD GOOD stable slot numbers (physical)
emc@provo-chartreuse:~> sudo -i cs_hal list disks Disks(s): SCSI Device Block Device Enclosure Partition Name —————- —————— —————- —————————————————/dev/sg11 /dev/sdl /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg12 /dev/sdm /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg13 /dev/sdn /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg14 /dev/sdo /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg15 /dev/sdp /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg16 /dev/sdq /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg17 /dev/sdr /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg18 /dev/sds /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg19 /dev/sdt /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg20 /dev/sdu /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg21 /dev/sdv /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg22 /dev/sdw /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg23 /dev/sdx /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg24 /dev/sdy /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg25 /dev/sdz /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg26 /dev/sdaa /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg27 /dev/sdab /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg28 /dev/sdac /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg29 /dev/sdad /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg30 /dev/sdae /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg31 /dev/sdaf /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg32 /dev/sdag /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg33 /dev/sdah /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg34 /dev/sdai /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg35 /dev/sdaj /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg36 /dev/sdak /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg37 /dev/sdal /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg38 /dev/sdam /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg39 /dev/sdan /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg40 /dev/sdao /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg41 /dev/sdap /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA /dev/sg42 /dev/sdaq /dev/sg44 ECS:object:qL4GBbuLTw+FkT+Sfz+FyA Slot —-B05 B06 B07 C02 C03 C04 C05 C06 C07 D02 D03 D04 D05 D06 D07 E02 E03 E04 E05 E06 E07 F02 F03 F04 F05 F06 F07 G02 G03 G04 G05 G06 Serial Number —————————2EG7604R 2EG74KUR 2EG76ZVR 2EG75Z7R 2EG74N3R 2EG7788R 2EG73YLR 2EG74S3R 2EG75N2R 2EG6GJZR 2EG6M2VR 2EG6M65R 2EG6MHAR 2EG6GKMR 2EG6M1BR 2EG6YBUR 2EG73U4R 2EG7PGLR 2EG76VYR 2EG7RA1R 2EG7647R 2EG6Z67R 2EG6YHGR 2EG6YM3R 2EG6YZ2R 2EG6Y2VR 2EG712ZR 2EG75XNR 2EG73ZER 2EG77A1R 2EG7119R 2EG6ZGPR SMART DiskSet ———- —————-GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD
SCSI Device —————/dev/sg43 /dev/sg2 /dev/sg3 /dev/sg4 /dev/sg5 /dev/sg6 /dev/sg7 /dev/sg8 /dev/sg9 /dev/sg10 /dev/sg55 /dev/sg56 /dev/sg57 /dev/sg58 /dev/sg59 /dev/sg60 /dev/sg61 /dev/sg62 /dev/sg63 /dev/sg64 /dev/sg65 /dev/sg66 /dev/sg67 /dev/sg68 /dev/sg69 /dev/sg70 /dev/sg71 /dev/sg72 /dev/sg73 /dev/sg74 /dev/sg75 /dev/sg76 /dev/sg77 /dev/sg78 Block Device —————-/dev/sdar /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdbc /dev/sdbd /dev/sdbe /dev/sdbf /dev/sdbg /dev/sdbh /dev/sdbi /dev/sdbj /dev/sdbk /dev/sdbl /dev/sdbm /dev/sdbn /dev/sdbo /dev/sdbp /dev/sdbq /dev/sdbr /dev/sdbs /dev/sdbt /dev/sdbu /dev/sdbv /dev/sdbw /dev/sdbx /dev/sdby /dev/sdbz Enclosure —————/dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 Partition Name —————————————————ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA Slot —-G07 A02 A03 A04 A05 A06 A07 B02 B03 B04 B12 B11 B10 B09 B08 B01 C14 C13 C12 C11 C10 C09 C08 C01 D14 D13 D12 D11 D10 D09 D08 D01 E14 E13 Serial Number —————————2EG73Y2R 2EG6YWZR 2EG6VEPR 2EG6VJ0R 2EG6Z50R 2EG6YTWR 2EG6YT2R 2EG771PR 2EG73NXR 2EG76Z5R 2EG7R9DR 2EG7MXBR 2EG6V5VR 2EG825VR 2EG5YUBR 2EG74VWR 2EG742ZR 2EG73RZR 2EG5RD8R 2EG74S4R 2EG75Z3R 2EG76P7R 2EG74TKR 2EG73S2R 2EG6LPTR 2EG42JPR 2EG6GGBR 2EG6EN9R 2EG6B32R 2EG6B1WR 2EG5KYPR 2EG6GS5R 2EG76XZR 2EG75YZR SMART DiskSet ———- —————-GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD
SCSI Device —————/dev/sg79 /dev/sg80 /dev/sg81 /dev/sg82 /dev/sg83 /dev/sg84 /dev/sg85 /dev/sg86 /dev/sg87 /dev/sg88 /dev/sg89 /dev/sg90 /dev/sg91 /dev/sg92 /dev/sg93 /dev/sg94 /dev/sg95 /dev/sg96 /dev/sg97 /dev/sg98 /dev/sg99 /dev/sg100 /dev/sg45 /dev/sg46 /dev/sg47 /dev/sg48 /dev/sg49 /dev/sg50 /dev/sg51 /dev/sg52 /dev/sg53 /dev/sg54 total: 98 Block Device —————-/dev/sdca /dev/sdcb /dev/sdcc /dev/sdcd /dev/sdce /dev/sdcf /dev/sdcg /dev/sdch /dev/sdci /dev/sdcj /dev/sdck /dev/sdcl /dev/sdcm /dev/sdcn /dev/sdco /dev/sdcp /dev/sdcq /dev/sdcr /dev/sdcs /dev/sdct /dev/sdcu /dev/sdcv /dev/sdas /dev/sdat /dev/sdau /dev/sdav /dev/sdaw /dev/sdax /dev/sday /dev/sdaz /dev/sdba /dev/sdbb Enclosure —————/dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 /dev/sg44 Partition Name —————————————————ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA ECS:object:qL4GBbuLTw+FkT+Sfz+FyA Slot —-E12 E11 E10 E09 E08 E01 F14 F13 F12 F11 F10 F09 F08 F01 G14 G13 G12 G11 G10 G09 G08 G01 A14 A13 A12 A11 A10 A09 A08 A01 B14 B13 Serial Number —————————2EG710RR 2EG73XLR 2EG73WAR 2EG6ZD9R 2EG7102R 2EG76TXR 2EG6YNRR 2EG6YXZR 2EG6Z77R 2EG6YHTR 2EG6YKAR 2EG5B24R 2EG7103R 2EG743HR 2EG73W8R 2EG5M2WR 2EG76RER 2EG5JAUR 2EG599ER 2EG70XGR 2EG75YRR 2EG775SR 2EG6GB8R 2EG6BMDR 2EG5P5PR 2EG65VPR 2EG6ZM7R 2EG7138R 2EG6VHBR 2EG6GJDR 2EG6Y11R 2EG6ZTER SMART DiskSet ———- —————-GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD GOOD
dino-black:~ % cs_hal list fs Volume(s): SCSI Device Block Device FS UUID —————- —————— ——————————————————/dev/sg2 /dev/sda 0ddb9635-ff27-4cd3-8c2f-58a6f5226d30 /dev/sg2 /dev/sda 2192b3ef-2a44-4450-9b04-327c00215454 /dev/sg2 /dev/sda ffa9607a-4b6f-4218-9266-c083fb1989a1 /dev/sg2 /dev/sda 746b09d4-f07a-49dc-8b40-86220dfc7edc /dev/sg2 /dev/sda f7c37c92-4bc5-4abf-95a5-efa51c46f6bc /dev/sg3 /dev/sdb 90a52650-e0f3-49e4-810b-a505cdcadb51 /dev/sg4 /dev/sdc 173aef8b-80e9-4be2-a510-3b88d3343f8a /dev/sg5 /dev/sdd bcfb1897-152b-482b-bde6-de9665ad7c51 /dev/sg6 /dev/sde bc6946ae-770f-4621-9ea5-f2d1e5ec0f28 /dev/sg7 /dev/sdf 52446742-a566-4036-8b0c-5cd7901474f0 /dev/sg8 /dev/sdg c9ee0971-d8dc-4621-8958-d79890d0f590 /dev/sg9 /dev/sdh 294bcd25-ab19-40ee-8c03-cd71e94e9e06 /dev/sg10 /dev/sdi cb5cac6c-1cdf-49ec-8754-a475db3d4afd /dev/sg11 /dev/sdj 91739495-2a46-47d2-8676-d8b4b3f8fd76 /dev/sg12 /dev/sdk 9f2a0ae1-d97b-4fb1-873e-6a9bfb2c3254 /dev/sg13 /dev/sdl 404a8c5a-19c0-4949-bd33-edd83ca4ee8f /dev/sg14 /dev/sdm da36046f-41f7-46d4-bcaa-af183002b792 dino-black:~ % cs_hala71b6937-8ae5-4a37-96d0-78feeb0e62c4 list fs /dev/sg15 /dev/sdn /dev/sg16 /dev/sdo 34d6f5c5-1f5d-4cea-af5a-af157324aee8 Volume(s): /dev/sg17 /dev/sdp 9cc59415-cab5-4456-881f-a0c533e1823d SCSI Device Block Device FS UUID —————/dev/sg2 /dev/sg2 /dev/sg2 /dev/sg2 /dev/sg2 —————-/dev/sda /dev/sda /dev/sda /dev/sda /dev/sda Type ————ext3 xfs xfs xfs swap v1 xfs xfs xfs xfs xfs xfs xfs xfs xfs xfs xfs xfs xfs xfs xfs Slot Label SMART Mount Point ——- ——- ————————BOOT GOOD /boot GOOD /root2 GOOD /var GOOD / GOOD 0 GOOD /data-disks/ss-90a52650-e0f3-49e4-810b-a505cdcadb51 1 GOOD /data-disks/ss-173aef8b-80e9-4be2-a510-3b88d3343f8a 2 GOOD /data-disks/ss-bcfb1897-152b-482b-bde6-de9665ad7c51 3 SUSPECT /data-disks/ss-bc6946ae-770f-4621-9ea5-f2d1e5ec0f28 4 GOOD /data-disks/ss-52446742-a566-4036-8b0c-5cd7901474f0 5 GOOD /data-disks/ss-c9ee0971-d8dc-4621-8958-d79890d0f590 6 GOOD /meta/294bcd25-ab19-40ee-8c03-cd71e94e9e06 7 GOOD /data-disks/ss-cb5cac6c-1cdf-49ec-8754-a475db3d4afd 8 GOOD /data-disks/ss-91739495-2a46-47d2-8676-d8b4b3f8fd76 9 GOOD /data-disks/ss-9f2a0ae1-d97b-4fb1-873e-6a9bfb2c3254 10 GOOD /meta/404a8c5a-19c0-4949-bd33-edd83ca4ee8f 11 GOOD /data-disks/ss-da36046f-41f7-46d4-bcaa-af183002b792 12 GOOD /data-disks/ss-a71b6937-8ae5-4a37-96d0-78feeb0e62c4 13 GOOD /meta/34d6f5c5-1f5d-4cea-af5a-af157324aee8 14 /data-disks/ss-9cc59415-cab5-4456-881f-a0c533e1823d TypeGOOD Slot Label SMART Mount —————————————————-0ddb9635-ff27-4cd3-8c2f-58a6f5226d30 2192b3ef-2a44-4450-9b04-327c00215454 ffa9607a-4b6f-4218-9266-c083fb1989a1 746b09d4-f07a-49dc-8b40-86220dfc7edc f7c37c92-4bc5-4abf-95a5-efa51c46f6bc volumes (file systems) ———- —— ——ext3 BOOT xfs xfs xfs swap v1 dino-black:~ % cs_hal list disks Disks(s): SCSI Device Block Device Enclosure —————- —————— —————n/a /dev/sda RAID vol /dev/sg0 n/a RAID array /dev/sg1 n/a RAID array /dev/sg3 /dev/sdb /dev/sg18 —————GOOD /boot GOOD /root2 GOOD /var GOOD / GOOD - Slot —-n/a 0 1 0 Serial Number ————————-not supported 9QE801ME 9QE834TG 9WM0R49P SMART ——n/a GOOD GOOD GOOD
COMMUNITY
Kine7c Open Storage Project, hosted @ Linux Founda7on htps://github.com/Kine7c
Plugfests
–
Par7cipa7ng
Companies
• • • • • • • • • • •
Seagate
Toshiba
Western
Digital/HGST
Sanmina/Newisys
EMC
Exablox
Red
Hat
Swihstack
Huawei
NetApp
AOL
•
Plugfest
Ac7vity
– – –
•
End-‐to-‐end
opera7on
– –
•
All
drives
present
successfully
operated
in
all
enclosures
present
Ceph,
Exablox,
Swistack soware
deployed
to
successfully
read/write
objects
using
all
drives
present
Events
– – –
•
Mul]ple
Kine7c
drives
Mul]ple
Kine7c
chassis
Mul]ple
Kine7c
client
sohware
December
2015
in
Cambridge,
MA
April
2016
at
Vault
in
Raleigh,
NC
20
September
2016
at
the
SNIA
Storage
Developer
Conference
in
Santa
Clara,
CA
Join
us!
SUMMARY
Sohware-‐Defined Storage • Scale-‐out storage is all about density (PB/rack) and cost ($/TB) – achieved by simplicity – less components, less cables – less code, less layers • Many deployments need flexibility – start small, grow large – adjustable compute/storage ra7os – purchase-‐7me choice good; dynamic choice be1er • Ethernet drives offer this flexibility & scalability
Learn More • Join the next Kine7c Plugfest & Open House – Tuesday, 20 September 2016 – SNIA Storage Developer Conference – Santa Clara, CA • 28 February 2017 at FAST in Santa Clara, CA • 22 March 2017 at Vault in Cambridge, MA
QUESTIONS?
This talk will outline use of Kinetic ethernet disk drive technology to build a petabyte-scale object storage system. The Kinetic Open Storage Project has incubated a set of hardware products and software interfaces to allow individual disk drives to be directly connected to general-purpose ethernet networking (instead of being hidden behind SAS/SATA or FC busses) and to be access via a key-value API (instead of traditional LBA access). The talk will overview experiences with a cluster of over 500 ethernet drives interconnected into a single cluster and operating as a single large-scale storage system. We will discuss the physical design, logical software structures, API benefits and limitations, manageability, and performance of this solution in comparison to traditional systems.
for free. You
can too.