{"id":615,"date":"2016-08-20T00:49:59","date_gmt":"2016-08-19T23:49:59","guid":{"rendered":"http:\/\/www.balajibandi.com\/?p=615"},"modified":"2018-02-23T23:59:29","modified_gmt":"2018-02-23T23:59:29","slug":"cisco-nexus-asic-basics","status":"publish","type":"post","link":"https:\/\/www.balajibandi.com\/?p=615","title":{"rendered":"Cisco Nexus ASIC Basics"},"content":{"rendered":"

Cisco Nexus ASIC Basics<\/strong><\/span><\/h1>\n

 <\/p>\n

The ASIC is the powerhouse of the Nexus switch. The ASIC, or Application Specific Integrated Circuit, is a special purpose sillicon\u00a0chip. The ASIC achieves high forwarding speeds as packets pass through the\u00a0forwarding plane.<\/p>\n

An ASIC in a general sense is not unique to networking. Many electronic devices use them for some specific purpose. In contrast, general processors, such as the CPU, perform many different tasks.<\/p>\n

 <\/p>\n

Different routers and switches will use ASICs in different ways. Even within the Nexus family, different ASICs achieve different goals. The Nexus 5000 series, for example, uses\u00a0two different ASIC’s.<\/p>\n

The\u00a0Unified Port Controller<\/strong>\u00a0(UPC) is the first N5K ASIC. Each UPC chip controls a small group of 4 or 8 ports, depending on whether the Nexus is a\u00a0 5500 series or Above. It handles processing packets upon ingress or egress for the port it manages. It also handles\u00a0forwarding decisions between ports.<\/p>\n

\n

The second ASIC type is the\u00a0Unified Crossbar Fabric<\/strong>\u00a0(UCF). This\u00a0is a\u00a0bridge between many UPC’s, and manages packet switching and scheduling. This couples a UPC for the ingress port to a UPC for the egress\u00a0port.<\/p>\n

The Nexus switches use ASICs in\u00a0cut-through switching\u00a0mode. This achieves\u00a0low latency and line-rate speeds. The 5548 has a latency of\u00a02 microseconds.\u00a0The Nexus switches transparently forward packets through the ASICs. This achieves high-performance in the data-plane.<\/p>\n

The ASICs have a special place in forwarding packets. This means that querying them\u00a0for packet drops is beneficial during troubleshooting.<\/p>\n<\/div>\n

ASIC Separation<\/strong><\/h3>\n
\n

It is good practice to spread\u00a0etherchannel or LAG\u00a0ports across different ASIC chips.\u00a0For example, a four-port etherchannel may use two ports connected to UPC #1, and two on\u00a0UPC #2.<\/p>\n

This is to protect against ASIC failure. In the rare occasions when a port fails, there’s a good chance that it’s actually the ASIC that has failed. This may\u00a0mean that\u00a0a group of ports has or soon will fail. If all ports in an etherchannel are on\u00a0a single failed ASIC, then the etherchannel will also fail.<\/p>\n

Another reason to spread links across ASIC’s is because of oversubscription. Four 10G ports may have a theoretical capacity of 40G throughput. However,\u00a0the ASIC that manages them\u00a0may only\u00a0have a throughput of 30G or 35G. This type of oversubscription works on the theory that not all ports will reach full capacity at once.<\/p>\n

 <\/p>\n

There’s good news for N5K owners though, as\u00a0each 10G port is mapped at 12G to the ASIC. This means that the N5K is\u00a0undersubscribed.\u00a0<\/em>This is\u00a0to get line rate (one of the reasons for the high Nexus price tag). If you’re expecting ports to reach capacity, understand the internal architecture of your switch.<\/p>\n

The\u00a0show hardware internal carmel<\/strong>\u00a0command lists the ASICs in a Nexus 5500. Different switches have different ASIC names, so the commands vary between models. For example,\u00a0the\u00a0Nexus 5000 series ASIC is ‘Gatos’ and the Nexus 5500 series is ‘Carmel’.<\/p>\n<\/div>\n

<\/div>\n
# show hardware internal carmel<\/strong><\/div>\n
<\/div>\n
Carmel ASIC Info:<\/div>\n
idx|glb|sl|as|pts|state |kaddr\u00a0 \u00a0|uaddr\u00a0 \u00a0|bus:fn|irq(hex)|\u00a0 MMmode[0:7]\u00a0 |diag<\/div>\n
—+—+–+–+—+——+——–+——–+——+——–+—————+—-<\/div>\n
0\u00a0 |0\u00a0 |0 |0 |8\u00a0 |ff\u00a0 \u00a0 |fa800000|674fe000|f:0\u00a0 \u00a0|d2\u00a0 \u00a0 \u00a0 |x:x:x:x:x:x:x:x|pass<\/div>\n
1\u00a0 |1\u00a0 |0 |1 |8\u00a0 |ff\u00a0 \u00a0 |fa000000|66cfe000|11:0\u00a0 |d1\u00a0 \u00a0 \u00a0 |x:x:x:x:x:x:x:x|pass<\/div>\n
2\u00a0 |2\u00a0 |0 |2 |8\u00a0 |ff\u00a0 \u00a0 |f9800000|664fe000|13:0\u00a0 |d0\u00a0 \u00a0 \u00a0 |x:x:x:x:x:x:g:g|pass<\/div>\n
3\u00a0 |3\u00a0 |0 |3 |8\u00a0 |ff\u00a0 \u00a0 |f8000000|65cfe000|19:0\u00a0 |cf\u00a0 \u00a0 \u00a0 |x:x:x:x:x:x:x:x|pass<\/div>\n
4\u00a0 |4\u00a0 |0 |4 |2\u00a0 |ff\u00a0 \u00a0 |f7800000|654fe000|1b:0\u00a0 |ce\u00a0 \u00a0 \u00a0 |-:-:-:-:s:s:-:-|pass<\/div>\n
5\u00a0 |16 |1 |0 |8\u00a0 |ff\u00a0 \u00a0 |f8800000|64ad5000|17:0\u00a0 |cd\u00a0 \u00a0 \u00a0 |x:x:x:x:x:x:x:x|pass<\/div>\n
6\u00a0 |17 |1 |1 |8\u00a0 |ff\u00a0 \u00a0 |f9000000|642d5000|15:0\u00a0 |cc\u00a0 \u00a0 \u00a0 |x:x:x:x:x:x:x:x|pass<\/div>\n
<\/div>\n
<\/div>\n
\n

The\u00a0show hardware internal carmel all-ports<\/strong>\u00a0command shows which port maps to each ASIC. This\u00a0is useful when designing ASIC separation.<\/p>\n

The\u00a0name<\/strong>\u00a0column is the internal port name, and the\u00a0car<\/strong>\u00a0column is the\u00a0Carmel Number<\/em>\u00a0(the ASIC) that the port maps to.\u00a0Mac<\/strong>\u00a0is the port number in the ASIC.<\/p>\n<\/div>\n

\n
# show hardware internal carmel all-ports<\/strong><\/div>\n
<\/div>\n
Carmel Port Info:<\/div>\n
name\u00a0 \u00a0|log|car|mac|flag|adm|opr|m:s:l|ipt|fab|xcar|xpt|if_index|diag|ucVer<\/div>\n
——-+—+—+—+—-+—+—+—–+—+—+—-+—+——–+—-+—–<\/div>\n
xgb1\/1 |0\u00a0 |0\u00a0 |0 p|b7\u00a0 |en |up |0:0:f|0\u00a0 |92 |0\u00a0 \u00a0|0\u00a0 |1a000000|pass| 0.00<\/div>\n
xgb1\/2 |1\u00a0 |0\u00a0 |1 p|b7\u00a0 |en |up |1:1:f|1\u00a0 |88 |0\u00a0 \u00a0|0\u00a0 |1a001000|pass| 0.00<\/div>\n
xgb1\/3 |2\u00a0 |0\u00a0 |2 p|b7\u00a0 |en |up |2:2:f|2\u00a0 |93 |0\u00a0 \u00a0|0\u00a0 |1a002000|pass| 0.00<\/div>\n
xgb1\/4 |3\u00a0 |0\u00a0 |3 p|b7\u00a0 |en |up |3:3:f|3\u00a0 |89 |0\u00a0 \u00a0|0\u00a0 |1a003000|pass| 0.00<\/div>\n
xgb1\/5 |4\u00a0 |0\u00a0 |4 p|b7\u00a0 |en |up |4:4:f|4\u00a0 |90 |0\u00a0 \u00a0|0\u00a0 |1a004000|pass| 0.00<\/div>\n
xgb1\/6 |5\u00a0 |0\u00a0 |5 p|b7\u00a0 |en |up |5:5:f|5\u00a0 |94 |0\u00a0 \u00a0|0\u00a0 |1a005000|pass| 0.00<\/div>\n
xgb1\/7 |6\u00a0 |0\u00a0 |6 p|b7\u00a0 |en |up |6:6:f|6\u00a0 |95 |0\u00a0 \u00a0|0\u00a0 |1a006000|pass| 0.00<\/div>\n
xgb1\/8 |7\u00a0 |0\u00a0 |7 p|b7\u00a0 |en |up |7:7:f|7\u00a0 |91 |0\u00a0 \u00a0|0\u00a0 |1a007000|pass| 0.00<\/div>\n
xgb1\/9 |8\u00a0 |1\u00a0 |0 p|b7\u00a0 |en |up |0:0:f|0\u00a0 |80 |0\u00a0 \u00a0|0\u00a0 |1a008000|pass| 0.00<\/div>\n
xgb1\/10|9\u00a0 |1\u00a0 |1 p|b7\u00a0 |en |up |1:1:f|1\u00a0 |87 |0\u00a0 \u00a0|0\u00a0 |1a009000|pass| 0.00<\/div>\n
xgb1\/11|10 |1\u00a0 |2 p|b7\u00a0 |en |up |2:2:f|2\u00a0 |81 |0\u00a0 \u00a0|0\u00a0 |1a00a000|pass| 0.00<\/div>\n
xgb1\/12|11 |1\u00a0 |3 p|b7\u00a0 |en |up |3:3:f|3\u00a0 |86 |0\u00a0 \u00a0|0\u00a0 |1a00b000|pass| 0.00<\/div>\n
xgb1\/13|12 |1\u00a0 |4 p|b7\u00a0 |en |up |4:4:f|4\u00a0 |82 |0\u00a0 \u00a0|0\u00a0 |1a00c000|pass| 0.00<\/div>\n
xgb1\/14|13 |1\u00a0 |5 p|b7\u00a0 |en |up |5:5:f|5\u00a0 |85 |0\u00a0 \u00a0|0\u00a0 |1a00d000|pass| 0.00<\/div>\n
xgb1\/15|14 |1\u00a0 |6 p|b7\u00a0 |en |up |6:6:f|6\u00a0 |83 |0\u00a0 \u00a0|0\u00a0 |1a00e000|pass| 0.00<\/div>\n
xgb1\/16|15 |1\u00a0 |7 p|b7\u00a0 |en |up |7:7:f|7\u00a0 |84 |0\u00a0 \u00a0|0\u00a0 |1a00f000|pass| 0.00<\/div>\n
xgb1\/17|16 |2\u00a0 |0 -|b7\u00a0 |dis|dn |0:0:f|0\u00a0 |75 |0\u00a0 \u00a0|0\u00a0 |1a010000|pass| 0.00<\/div>\n
xgb1\/18|17 |2\u00a0 |1 -|b7\u00a0 |dis|dn |1:1:f|1\u00a0 |76 |0\u00a0 \u00a0|0\u00a0 |1a011000|pass| 0.00<\/div>\n
xgb1\/19|18 |2\u00a0 |2 -|b7\u00a0 |dis|dn |2:2:f|2\u00a0 |74 |0\u00a0 \u00a0|0\u00a0 |1a012000|pass| 0.00<\/div>\n
xgb1\/20|19 |2\u00a0 |3 -|b7\u00a0 |dis|dn |3:3:f|3\u00a0 |77 |0\u00a0 \u00a0|0\u00a0 |1a013000|pass| 0.00<\/div>\n
xgb1\/21|20 |2\u00a0 |4 -|b7\u00a0 |dis|dn |4:4:f|4\u00a0 |78 |0\u00a0 \u00a0|0\u00a0 |1a014000|pass| 0.00<\/div>\n
xgb1\/22|21 |2\u00a0 |5 -|b7\u00a0 |dis|dn |5:5:f|5\u00a0 |73 |0\u00a0 \u00a0|0\u00a0 |1a015000|pass| 0.00<\/div>\n
1gb1\/23|22 |2\u00a0 |6 -|b7\u00a0 |dis|dn |6:6:0|6\u00a0 |72 |0\u00a0 \u00a0|0\u00a0 |1a016000|pass| 0.00<\/div>\n
1gb1\/24|23 |2\u00a0 |7 -|b7\u00a0 |dis|dn |7:7:0|7\u00a0 |79 |0\u00a0 \u00a0|0\u00a0 |1a017000|pass| 0.00<\/div>\n
xgb1\/25|24 |3\u00a0 |0 -|b7\u00a0 |dis|dn |0:0:f|0\u00a0 |71 |0\u00a0 \u00a0|0\u00a0 |1a018000|pass| 0.00<\/div>\n
xgb1\/26|25 |3\u00a0 |1 -|b7\u00a0 |dis|dn |1:1:f|1\u00a0 |64 |0\u00a0 \u00a0|0\u00a0 |1a019000|pass| 0.00<\/div>\n
xgb1\/27|26 |3\u00a0 |2 -|b7\u00a0 |dis|dn |2:2:f|2\u00a0 |65 |0\u00a0 \u00a0|0\u00a0 |1a01a000|pass| 0.00<\/div>\n
xgb1\/28|27 |3\u00a0 |3 -|b7\u00a0 |dis|dn |3:3:f|3\u00a0 |70 |0\u00a0 \u00a0|0\u00a0 |1a01b000|pass| 0.00<\/div>\n
xgb1\/29|28 |3\u00a0 |4 -|b7\u00a0 |dis|dn |4:4:f|4\u00a0 |66 |0\u00a0 \u00a0|0\u00a0 |1a01c000|pass| 0.00<\/div>\n
xgb1\/30|29 |3\u00a0 |5 -|b7\u00a0 |dis|dn |5:5:f|5\u00a0 |69 |0\u00a0 \u00a0|0\u00a0 |1a01d000|pass| 0.00<\/div>\n
xgb1\/31|30 |3\u00a0 |6 -|b7\u00a0 |dis|dn |6:6:f|6\u00a0 |67 |0\u00a0 \u00a0|0\u00a0 |1a01e000|pass| 0.00<\/div>\n
xgb1\/32|31 |3\u00a0 |7 -|b7\u00a0 |dis|dn |7:7:f|7\u00a0 |68 |0\u00a0 \u00a0|0\u00a0 |1a01f000|pass| 0.00<\/div>\n
sup0\u00a0 \u00a0|32 |4\u00a0 |4 -|b7\u00a0 |en |dn |4:4:0|4\u00a0 |62 |0\u00a0 \u00a0|0\u00a0 |15020000|pass| 0.00<\/div>\n
sup1\u00a0 \u00a0|33 |4\u00a0 |5 -|b7\u00a0 |en |dn |5:5:1|5\u00a0 |59 |0\u00a0 \u00a0|0\u00a0 |15010000|pass| 0.00<\/div>\n
xgb2\/1 |0\u00a0 |0\u00a0 |0 -|b7\u00a0 |dis|dn |0:0:f|0\u00a0 |61 |0\u00a0 \u00a0|0\u00a0 |1a080000|pass| 0.00<\/div>\n
xgb2\/2 |1\u00a0 |0\u00a0 |1 -|b7\u00a0 |dis|dn |1:1:f|1\u00a0 |58 |0\u00a0 \u00a0|0\u00a0 |1a081000|pass| 0.00<\/div>\n
xgb2\/3 |2\u00a0 |0\u00a0 |2 -|b7\u00a0 |dis|dn |2:2:f|2\u00a0 |55 |0\u00a0 \u00a0|0\u00a0 |1a082000|pass| 0.00<\/div>\n
xgb2\/4 |3\u00a0 |0\u00a0 |3 -|b7\u00a0 |dis|dn |3:3:f|3\u00a0 |57 |0\u00a0 \u00a0|0\u00a0 |1a083000|pass| 0.00<\/div>\n
xgb2\/5 |4\u00a0 |0\u00a0 |4 -|b7\u00a0 |dis|dn |4:4:f|4\u00a0 |53 |0\u00a0 \u00a0|0\u00a0 |1a084000|pass| 0.00<\/div>\n
xgb2\/6 |5\u00a0 |0\u00a0 |5 -|b7\u00a0 |dis|dn |5:5:f|5\u00a0 |56 |0\u00a0 \u00a0|0\u00a0 |1a085000|pass| 0.00<\/div>\n
xgb2\/7 |6\u00a0 |0\u00a0 |6 -|b7\u00a0 |dis|dn |6:6:f|6\u00a0 |49 |0\u00a0 \u00a0|0\u00a0 |1a086000|pass| 0.00<\/div>\n
xgb2\/8 |7\u00a0 |0\u00a0 |7 -|b7\u00a0 |dis|dn |7:7:f|7\u00a0 |51 |0\u00a0 \u00a0|0\u00a0 |1a087000|pass| 0.00<\/div>\n
xgb2\/9 |8\u00a0 |1\u00a0 |0 -|b7\u00a0 |dis|dn |0:0:f|0\u00a0 |48 |0\u00a0 \u00a0|0\u00a0 |1a088000|pass| 0.00<\/div>\n
xgb2\/10|9\u00a0 |1\u00a0 |1 -|b7\u00a0 |dis|dn |1:1:f|1\u00a0 |50 |0\u00a0 \u00a0|0\u00a0 |1a089000|pass| 0.00<\/div>\n
xgb2\/11|10 |1\u00a0 |2 -|b7\u00a0 |dis|dn |2:2:f|2\u00a0 |47 |0\u00a0 \u00a0|0\u00a0 |1a08a000|pass| 0.00<\/div>\n
xgb2\/12|11 |1\u00a0 |3 -|b7\u00a0 |dis|dn |3:3:f|3\u00a0 |52 |0\u00a0 \u00a0|0\u00a0 |1a08b000|pass| 0.00<\/div>\n
xgb2\/13|12 |1\u00a0 |4 -|b7\u00a0 |dis|dn |4:4:f|4\u00a0 |46 |0\u00a0 \u00a0|0\u00a0 |1a08c000|pass| 0.00<\/div>\n
xgb2\/14|13 |1\u00a0 |5 -|b7\u00a0 |dis|dn |5:5:f|5\u00a0 |54 |0\u00a0 \u00a0|0\u00a0 |1a08d000|pass| 0.00<\/div>\n
xgb2\/15|14 |1\u00a0 |6 -|b7\u00a0 |dis|dn |6:6:f|6\u00a0 |38 |0\u00a0 \u00a0|0\u00a0 |1a08e000|pass| 0.00<\/div>\n
xgb2\/16|15 |1\u00a0 |7 -|b7\u00a0 |dis|dn |7:7:f|7\u00a0 |45 |0\u00a0 \u00a0|0\u00a0 |1a08f000|pass| 0.00<\/div>\n<\/div>\n
<\/div>\n
\n

Separation\u00a0Necessary?<\/strong><\/h3>\n
\n

Is it\u00a0really\u00a0<\/em>necessary to go to all the effort of separating ASICs? It depends on where you are in the design lifecycle, and the level of risk you are willing to accept.<\/p>\n

If you’re still in the design phase, it may be worth spending\u00a0time thinking about ASIC separation. It won’t take a lot longer, as it’s just a matter of selecting appropriate ports for the etherchannel. If your switches are already in production, then it may not be worth going back and changing the design.<\/p>\n

It’s a rare occurrence to lose an ASIC, so the risk is quite low. If you’re in an industry with no room for\u00a0error, it may be worth your time to consider ASIC separation. This eliminates\u00a0another potential failure point.<\/p>\n

Another way to mitigate this risk is to use virtual port-channels. vPC’s spread links across two different switches, and thus different ASICs. Keep in mind that the peer-link and keep-alive would still need ASIC separation.<\/p>\n<\/div>\n<\/div>\n

Good information to share.<\/div>\n
<\/div>\n
happy Labbbing !!!!<\/div>\n","protected":false},"excerpt":{"rendered":"

Cisco Nexus ASIC Basics   The ASIC is the powerhouse of the Nexus switch. The ASIC, or Application Specific Integrated Circuit, is a special purpose sillicon\u00a0chip. The ASIC achieves high forwarding speeds as packets pass through the\u00a0forwarding plane. An ASIC in a general sense is not unique to networking. Many electronic devices use them for […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,2],"tags":[],"class_list":["post-615","post","type-post","status-publish","format-standard","hentry","category-ccie-dc","category-cisco"],"_links":{"self":[{"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=\/wp\/v2\/posts\/615","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=615"}],"version-history":[{"count":1,"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=\/wp\/v2\/posts\/615\/revisions"}],"predecessor-version":[{"id":616,"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=\/wp\/v2\/posts\/615\/revisions\/616"}],"wp:attachment":[{"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=615"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=615"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.balajibandi.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}