You are on page 1of 466

Operating Cisco

Application Centric
Infrastructure

Alejandra Sanchez, Andres Vega, Arvind Chari, Carly Stoughton,
Christopher Stokes, Gabriel Fontenot, Jonathan Cornell, Ken Fee,
Kevin Corbin, Lauren Malhoit, Loy Evans, Lucien Avramov,
Paul Lesiak, Steven Lym, Rafael Muller, Robert Burns

Operating Cisco
Application Centric
Infrastructure

Copyright
Op​
er​
at​
ing Cisco Ap​
pli​
ca​
tion Cen​
tric In​
fra​
struc​
ture
Ale​
jan​
dra Sanchez, An​
dres Vega, Arvind Chari, Carly Stoughton, Christo​
pher Stokes,
Gabriel Fontenot, Jonathan Cor​
nell, Ken Fee, Kevin Corbin, Lau​
ren Mal​
hoit, Loy Evans,
Lu​
cien Avramov, Paul Lesiak, Rafael Muller, Robert Burns. Steven Lym
Copy​
right © 2015 Cisco Sys​
tems, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or
agreed to in writing, software distributed under the License is distributed on an "AS IS"
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing permissions and
limitations under the License.
Cisco Press logo is a trade​
mark of Cisco Sys​
tems, Inc.
Pri​
vately pub​
lished by Cisco Sys​
tems, Inc.

Warning and Disclaimer
This book is de​
signed to pro​
vide in​
for​
ma​
tion about Cisco ACI. Every ef​
fort has been
made to make this book as com​
plete and as ac​
cu​
rate as pos​
si​
ble, but no war​
ranty or
fit​
ness is im​
plied.
The in​
for​
ma​
tion is pro​
vided on an “as is” basis. The au​
thors, and Cisco Sys​
tems, Inc.
shall have nei​
ther li​
ab
​il​
ity nor re​
spon​
si​
bil​
ity to any per​
son or en​
tity with re​
spect to any
loss or dam​
ages aris​
ing from the in​
for​
ma​
tion con​
tained in this doc​
um
​ent.
The opin​
ions ex​
pressed in this book be​
long to the au​
thor and are not nec​
es​
sar​
ily those
of Cisco Sys​
tems, Inc.

Feed​
back In​
for​
ma​
tion
Read​
ers’ feed​
back is a nat​
ural con​
tin​
ua
​t​
ion of this process. If you have any com​
ments
re​
gard​
ing how we could im​
prove the qual​
ity of this book, or oth​
er​
wise alter it to bet​
ter
suit your needs, you can contact
​ us through email at ops-booksprint@​
cisco.​
com . ​

Contents

. . . . . Prologue
........................................................................

1

. . . . . Abstract
...................................................................

3

. . . . . Authors
...................................................................

5

. . . . . Book
. . . . . Writing
. . . . . . . .Methodology
......................................................

7

. . . . . Hardware
. . . . . . . . . .and
. . . .Software
. . . . . . . . .Included
. . . . . . . . in
. . the
. . . .Book
..............................

9

. . . . . Introduction
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
. . . . . The
. . . . Story
. . . . . .of
. . ACME
. . . . . . Inc.
.................................................

13

. . . . . The
. . . . Why,
. . . . . .Who,
. . . . .What,
. . . . . .When
. . . . . .and
. . . .How
....................................

15

. . . . . Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
. . . . . APIC
. . . . . Overview
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
. . . . . Configuring
. . . . . . . . . . . .Management
. . . . . . . . . . . . Protocols
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
. . . . . Role-Based
. . . . . . . . . . . Access
. . . . . . .Control
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
. . . . . Import
. . . . . . . and
. . . . Export
. . . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

. . . . . Upgrading
. . . . . . . . . . .and
. . . Downgrading
. . . . . . . . . . . . . Firmware
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
. . . . . Section
. . . . . . . .Content
...........................................................

51

. . . . . Firmware
. . . . . . . . . .Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
. . . . . Upgrading
. . . . . . . . . . .and
. . . Downgrading
. . . . . . . . . . . . . Considerations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
. . . . . Upgrading
. . . . . . . . . . .the
. . . Fabric
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

. . . . . Fabric
. . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
. . . . . Section
. . . . . . . .Content
...........................................................

71

. . . . . Understanding
. . . . . . . . . . . . . . .Fabric
. . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
. . . . . Adding
. . . . . . . New
. . . . .Devices
. . . . . . . .to
. .the
. . . .Fabric
.........................................

81

. . . . . Server
. . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
. . . . . Virtual
. . . . . . . Machine
. . . . . . . . Networking
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
. . . . . Deploying
. . . . . . . . . . the
. . . .Application
. . . . . . . . . . .Virtual
. . . . . . Switch
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
. . . . . External
. . . . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
. . . . . Application
. . . . . . . . . . . Migration
. . . . . . . . . .Use
. . . .Case
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

. . . . . Tenants
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
. . . . . ACI
. . . . Tenancy
. . . . . . . . .Models
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
. . . . . Application
. . . . . . . . . . . Profile
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
. . . . . Endpoint
. . . . . . . . . Group
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
. . . . . Endpoint
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

. . . . . Private
. . . . . . . Network
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
. . . . . Bridge
. . . . . . .Domain
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
. . . . . Tenant
. . . . . . . Networking
. . . . . . . . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

. . . . . Working
. . . . . . . . .with
. . . . Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
. . . . . Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
. . . . . Filters
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
. . . . . Taboo
. . . . . . .Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
. . . . . Inter-Tenant
. . . . . . . . . . . . .Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
. . . . . Contracts
. . . . . . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

. . . . . Layer
. . . . . .4. .to
. .Layer
. . . . . .7. Services
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .249
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
. . . . . Understanding
. . . . . . . . . . . . . . .Layer
. . . . . 4. .to
. . Layer
. . . . . .7. Integration
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
. . . . . Services
. . . . . . . . .Deployment
. . . . . . . . . . . Guide
. . . . . . Reference
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
. . . . . Service
. . . . . . . .Graph
. . . . . .Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
. . . . . ASAv
. . . . . Sample
. . . . . . . .Configuration
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

. . . . . Health
. . . . . . .Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .269
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
. . . . . Understanding
. . . . . . . . . . . . . . .Health
. . . . . . Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
. . . . . Understanding
. . . . . . . . . . . . . . .Faults
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
. . . . . How
. . . . . Health
. . . . . . .Scores
. . . . . . Are
. . . . Calculated
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
. . . . . Health
. . . . . . .Score
. . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

. . . . . Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . -. .Tenant
. . . . . . .and
. . . .Fabric
. . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . -. .Infrastructure
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . Use
. . . . Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . Tools
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . Use
. . . . Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

. . . . . Scripting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .349
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
. . . . . Leveraging
. . . . . . . . . . .Network
. . . . . . . . Programmability
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
. . . . . ACI
. . . . and
. . . . Scripting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
. . . . . API
. . . .Inspector
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
. . . . . Development
. . . . . . . . . . . . . Techniques
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

. . . . . STman
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
. . . . . Cobra
. . . . . . SDK
. . . . .and
. . . .Arya
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
. . . . . ACI
. . . . Toolkit
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
. . . . . GitHub
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

. . . . . Hardware
. . . . . . . . . .Expansion
. . . . . . . . . .and
. . . .Replacement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .389
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
. . . . . Expanding
. . . . . . . . . . .and
. . . Shrinking
. . . . . . . . . .the
. . . Fabric
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
. . . . . Hardware
. . . . . . . . . .Diagnostics
. . . . . . . . . . . and
. . . .Replacement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

. . . . . Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .413
. . . . . Classes
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
. . . . . Package
. . . . . . . . Decoder
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
. . . . . Acronyms
. . . . . . . . . . and
. . . .Definitions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
. . . . . Reference
. . . . . . . . . . Material
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
....

1 Prologue .

.

and processes that can be em​ ployed to sup​ port day 1+ op​ er​ a​ tions within an ACI-based fab​ ric. but being able to ef​ fec​ tively op​ er​ ate the in​ fra​ struc​ ture be​ yond the Day Zero build ac​ tiv​ i​ ties is cru​ cial to long term ef​ fec​ tive​ ness and ef​ fi​ ciency. Hav​ ing the tools to change how in​ fra​ struc​ ture is built is one thing.Prologue 3 Abstract Cisco's Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) pro​ vides pow​ er​ ful new ways to dy​ nam​ i​ cally man​ age in​ fra​ struc​ ture in the mod​ ern world of IT au​ toma​ tion and De​ vOps. This book ex​ am​ ines some of the com​ mon op​ er​ a​ tional ac​ tiv​ i​ ties that IT teams use to pro​ vide con​ tin​ ued in​ fra​ struc​ ture op​ er​ a​ tions and gives the reader ex​ po​ sure to the tools. method​ olo​ gies. To ef​ fec​ tively har​ ness the power of ACI. or​ ga​ ni​ za​ tions will need to un​ der​ stand how to in​ cor​ po​ rate ACI into their daily op​ er​ a​ tions. .

.

So​ lu​ tions Ar​ chi​ tect. Busi​ ness Tech​ nol​ ogy Ar​ chi​ tects Kevin Corbin .Net​ work Con​ sult​ ing En​ gi​ neer. with par​ tic​ ip ​a​ tion of a num​ ber of Cisco func​ tional or​ ga​ ni​ za​ tions in​ clud​ ing Cisco Ad​ vanced Ser​ vices. Sales Lu​ cien Avramov . So​ lu​ tion Val​ id ​a​ tion Ser​ vices Robert Burns . Tech​ ni​ cal Ser​ vices An​ dres Vega .Tech​ ni​ cal Writer. IT. Sales Ken Fee .Tech​ ni​ cal So​ lu​ tions Ar​ chi​ tect. INSBU Christo​ pher Stokes . INSBU Paul Lesiak . Ad​ vanced Ser​ vices Carly Stoughton .Tech​ ni​ cal Mar​ ket​ ing En​ gi​ neer. Authors Ale​ jan​ dra Sanchez . INSBU .Con​ sult​ ing Ar​ chi​ tect.Tech​ ni​ cal Mar​ ket​ ing En​ gi​ neer.Prologue 5 Authors This book rep​ re​ sents a joint in​ tense col​ lab​ or​ a​ tive ef​ fort over the course of a week. Sys​ tems En​ gi​ neer​ ing Lau​ ren Mal​ hoit . Tech​ ni​ cal Ser​ vices Steven Lym .Tech​ ni​ cal Leader. Prod​ uct Mar​ ket​ ing. Ad​ vanced Ser​ vices Jonathan Cor​ nell .Sys​ tems En​ gi​ neer.Cus​ tomer Sup​ port En​ gi​ neer.Cisco IT Net​ work Con​ sult​ ing En​ gi​ neer Gabriel Fontenot .Con​ sult​ ing Sys​ tem En​ gi​ neer. Tech​ ni​ cal Ser​ vices.Cus​ tomer Sup​ port En​ gi​ neer. INSBU Loy Evans . So​ lu​ tion Val​ id ​a​ tion Test​ ing.Mar​ ket​ ing Con​ sul​ tant.So​ lu​ tions Ar​ chi​ tect. Tech​ ni​ cal Ser​ vices Arvind Chari . Ad​ vanced Ser​ vices Rafael Muller .Tech​ ni​ cal Leader. and Sales.

6 Prologue Book Sprint Facilitation Bar​ bara Rühling Faith Bosworth Illustrations Hen​ rik van Leuwen Book Production Julien Taquet Clean Up Raewyn Whyte .

​ net) method​ ol​ ogy was used for writ​ ing this book. The Book Sprint method​ ol​ ogy is an in​ no​ v​ at​ ive new style of co​ op​ er​ at​ ive and col​ lab​ o​ ra​ tive book pro​ duc​ tion. but in​ volved hun​ dreds of au​ thor​ ing per​ son-hours. By lever​ ag​ ing the input of many ex​ perts. . and in​ cluded thou​ sands of ex​ pe​ ri​ enced en​ gi​ neer​ ing hours. Book Sprints are strongly fa​ cil​ it​ ated and lever​ age team-ori​ ented in​ spi​ ra​ tion and mo​ ti​ va​ tion to rapidly de​ liver large amounts of well-au​ thored and re​ viewed con​ tent.Prologue 7 Book Writing Methodology The Book Sprint (booksprints. al​ low​ ing for ex​ tremely high qual​ ity in a very short pro​ duc​ tion time pe​ riod. and in​ cor​ po​ rate it into a com​ plete nar​ ra​ tive in a short amount of time. the com​ plete book was writ​ ten in only five days.

.

1 soft​ ware re​ lease. and 93128TX • Cisco Application Virtual Switch (AVS) • Cisco UCS B and C series servers • Several models of switches and routers.Prologue 9 Hardware and Software Included in the Book The Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) com​ bines hard​ ware. the fol​ low​ ing hard​ ware de​ vices were used: • Cisco Application Policy Infrastructure Controller (APIC) • ACI Spine switches. and VMware • IXIA IxVM product family traffic generators vSphere This book was writ​ ten based on the Cisco ACI 1. 9396TX. . and vir​ tu​ al​ iza​ tion. soft​ ware. including Cisco Nexus 9508 and 9336PQ • ACI Leaf switches. ap​ pli​ ca​ tion. including KVM. se​ cu​ rity. and ASIC in​ no​ va​ tions into an in​ te​ grated sys​ tems ap​ proach and pro​ vides a com​ mon man​ age​ ment frame​ work for net​ work. Microsoft Hyper-V. including Cisco Nexus 5000 switches and Cisco Integrated Services Routers (ISR) • A variety of hypervisors. including Cisco Nexus 9396PX. For the pur​ pose of writ​ ing this book.

.

11 Introduction .

.

In an ef​ fort to be more com​ pet​ it​ ive. can be tested along​ side the ac​ tual ap​ pli​ ca​ tion. jetpro​ pelled uni​ cy​ cles. is a multi​ na​ tional cor​ po​ ra​ tion that spe​ cial​ izes in man​ uf​ ac​ tur​ ing. can be checked into ver​ sion con​ trol. and taken longer to ac​ com​ plish than the busi​ ness lead​ ers would have liked. ACME has un​ der​ taken a pro​ ject to build a mo​ bile ap​ pli​ ca​ tion plat​ form to sup​ port or​ der​ ing and lo​ gis​ tics for prod​ uct de​ liv​ ery to their cus​ tomers for their en​ tire port​ fo​ lio. ACME busi​ ness units have lever​ aged third party soft​ ware com​ pa​ nies and com​ mer​ cially avail​ able soft​ ware to meet their IT de​ mands. These prod​ uct groups op​ er​ ate as sep​ ar​ ate busi​ ness groups within the com​ pany. They have largely fo​ cused on re​ tail routes to mar​ ket. and there​ fore ACME is look​ ing for a new ap​ proach to both ap​ pli​ ca​ tion and in​ fra​ struc​ ture life​ cy​ cle man​ age​ ment. but have re​ cently de​ cided to pur​ sue a more di​ rect-to-con​ sumer busi​ ness model due to in​ tense pres​ sure from new com​ peti​ tors who have dom​ in ​ated the on​ line sales chan​ nels. they have lever​ aged a tra​ di​ tional in​ fra​ struc​ ture and soft​ ware model that does not allow them to keep up with the chang​ ing re​ quire​ ments. For this rea​ son. The ap​ pli​ ca​ tion de​ vel​ op​ ers have been look​ ing at new ap​ pli​ ca​ tion de​ vel​ op​ ment trends such as Con​ tin​ uo ​us De​ liv​ ery and Con​ tin​ uo ​us In​ te​ gra​ tion. . ACME Inc. and can con​ tin​ ua ​lly im​ prove. and have pre​ vi​ ously main​ tained sep​ ar​ ate in​ fra​ struc​ ture and ap​ pli​ ca​ tions. and dis​ tri​ bu ​t​ ion of a di​ verse prod​ uct port​ fo​ lio. the in​ fra​ struc​ ture com​ po​ nents need to be ca​ pa​ ble of map​ ping to these new par​ a-​ digms in a way that is not pos​ si​ ble using tra​ di​ tional con​ cepts. in​ clud​ ing rocket-pow​ ered roller skates. caused cus​ tomer-im​ pact​ ing out​ ages. One of the largest chal​ lenges ACME has his​ tor​ ic ​ally faced is that op​ er​ at​ ions and in​ fra​ struc​ ture has been an af​ ter​ thought to prod​ uct de​ vel​ op​ ment. sales. ACME Inc. Where they have used cus​ tom soft​ ware in the past. This has led to sev​ eral sit​ u​ at​ ions where ap​ pli​ ca​ tion de​ ploy​ ments have meant long week​ end hours for all of the teams. and var​ io ​us ex​ plo​ sive ma​ te​ ri​ als.Introduction 13 The Story of ACME Inc. Tra​ di​ tion​ ally. and the new ap​ pli​ ca​ tion plat​ form is to be de​ vel​ oped in this man​ ner. To sup​ port this. has de​ cided to change by cre​ at​ ing an en​ vi​ ron​ ment where in​ fra​ struc​ ture ar​ ti​ facts are treated as part of the ap​ pli​ ca​ tion. while in​ cor​ po​ rat​ ing an on​ go​ ing im​ prove​ ment cycle so they can react to chang​ ing mar​ ket dy​ nam​ ics in a more nim​ ble fash​ ion. but would like to cre​ ate a more in​ ti​ mate re​ la​ tion​ ship with their con​ sumers and be able to take feed​ back on the plat​ form di​ rectly from those users.

ACME Inc. is highly de​ sir​ able. The abil​ ity to ab​ stract all phys​ ic ​al and vir​ tual in​ fra​ struc​ ture con​ fig​ ur​ a​ tion into a sin​ gle con​ fig​ ur​ a​ tion that is con​ sis​ tent across dev. "Why fix what is al​ ready work​ ing?" .14 Introduction While ACME is in​ tensely fo​ cused on de​ liv​ er​ ing the new ap​ pli​ ca​ tion plat​ form in a timely man​ ner. told ACME: "The world is chang​ ing. Un​ less com​ pa​ nies make in​ vest​ ments in tech​ nolo​ gies that allow for con​ sump​ tion of au​ to​ mated com​ po​ nents. this runs con​ trary to how most op​ er​ at​ ions groups exist in the re​ la​ tion​ ship be​ tween busi​ ness and tech​ nol​ ogy. such as Ama​ zon Web Ser​ vices (AWS) and Open​ stack. and prod en​ vi​ ron​ ments. John Cham​ bers. thus al​ low​ ing IT to op​ er​ ate and man​ age at the speed of busi​ ness. is a fic​ ti​ tious com​ pany. While ACME Inc. the only other way to scale is by break​ ing the human level com​ po​ nent." As ev​ id ​enced by the suc​ cess of cloud plat​ forms. con​ sump​ tion mod​ els of tech​ nol​ ogy de​ liv​ ery have the abil​ ity to adapt tech​ nol​ ogy more quickly to rapid busi​ ness re​ quire​ ments changes. Con​ trol of op​ er​ at​ ions is what op​ er​ at​ ions groups are fo​ cused on. test.'s busi​ ness own​ ers need. with a change of this na​ ture comes fear. After an​ al​ yz​ ing cur​ rent of​ fers from var​ io ​us tech​ nol​ ogy ven​ dors. At an ex​ ec​ ut​ ive brief​ ing. un​ cer​ tainty. se​ lected Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI). but con​ trol can be a bar​ rier to a pure con​ sump​ tion model. How​ ever. Every com​ pany is a tech​ nol​ ogy com​ pany. How​ ever. and if you don't adapt. In​ no​ va​ tion at this level will pro​ vide more op​ por​ tu​ ni​ ties for ex​ pand​ ing the tools with which users in​ ter​ act. the CEO of Cisco Sys​ tems. The thought is. this is the true story of every com​ pany. as well as portable across the var​ io ​us data cen​ ter lo​ ca​ tions cur​ rently main​ tained by ACME. and doubt. This is the type of con​ sump​ tion that ACME Inc. This book will at​ tempt to bring some level of com​ fort and fa​ mil​ iar​ ity with op​ er​ at​ ions ac​ tiv​ i-​ ties within an ACI fab​ ric. and few peo​ ple would re​ ally choose to work for that type of com​ pany. Work​ ers in the IT in​ dus​ try need to adapt to keep up with the rapid change of the busi​ ness. you'll get left be​ hind. This is where the ful​ crum will tilt in the favor of IT and in​ fra​ struc​ ture being more dy​ namic. ACME is also in​ ter​ ested in cre​ at​ ing a foun​ da​ tion on which it can grow to de​ liver a com​ mon pool of in​ fra​ struc​ ture that is shared across all busi​ ness groups and op​ er​ ated in a multi-ten​ ant fash​ ion to in​ crease ef​ fi​ ciency. ACI has been built from the ground up to change the sub​ struc​ ture used to build net​ work de​ vices and pro​ to​ cols. Most IT op​ er​ at​ ions groups in​ vest a lot of time in the tools needed to de​ liver ser​ vices today and there is an or​ ganic re​ sis​ tance to re-in​ vest​ ing. and just im​ por​ tant this is the story of the em​ ploy​ ees of those com​ pa​ nies.

Ini​ tially. and this in​ fra​ struc​ ture are new to ACME Inc. ACME must ad​ dress fun​ da​ men​ tal ques​ tions in​ clud​ ing Who man​ ages What. Who As with most or​ ga​ ni​ za​ tions. Look​ ing at why an au​ to​ mated fab​ ric is ben​ e​ fi​ cial to an or​ ga​ ni​ za​ tion is im​ por​ tant for set​ ting ex​ pec​ ta​ tions of re​ turn on in​ vest​ ment. When dif​ fer​ ent groups per​ form reg​ ul​ ar op​ er​ at​ ions. and Where they go to per​ form these op​ er​ at​ ions. but more likely the bound​ aries are blurred to some de​ gree. and How they go about their tasks. but more tac​ ti​ cal and pointin-time-rel​ e​ vant. Also. as it could be looked at as a threat to their own job. can di​ vide the work​ load. a high de​ gree of au​ toma​ tion is re​ quired. What. tra​ di​ tion​ ally had dif​ fer​ ent types of stake​ hold​ ers in​ volved in mak​ ing any IT ini​ tia​ tive suc​ cess​ ful. In the case of ACME Inc. but rec​ og​ nizes that this ini​ tia​ tive. are also con​ sid​ er​ at​ ions. ACME Inc. the key suc​ cess cri​ te​ ria is to stream​ line processes and pro​ ce​ dures re​ lated to the de​ ploy​ ment of in​ fra​ struc​ ture re​ quired to sup​ port the ap​ pli​ ca​ tion ini​ tia​ tives. When and How Within ACI. At the macro level. au​ toma​ tion can be a scary propo​ si​ tion for some of the key stake​ hold​ ers. au​ toma​ tion is about mak​ ing work more en​ joy​ able for all team mem​ bers. al​ low​ ing them the free​ dom to in​ no​ vate and add value.Introduction 15 The Why. repet​ it​ ive tasks. Listed below are some char​ ac​ ter​ is​ tics of these groups. these groups can have dis​ tinct or​ ga​ ni​ za​ tional bound​ aries. Quite the op​ po​ site. this ap​ pli​ ca​ tion. look​ ing at why an op​ er​ at​ ional prac​ tice is done a spe​ cific way can help with fram​ ing the tools and processes that are em​ ployed. Why "Why" is the most im​ por​ tant as​ pect of what should be con​ sid​ ered in op​ er​ at​ ional​ iz​ ing an ACI fab​ ric. the fact that these dif​ fer​ ent or​ ga​ ni​ za​ tions exist should . but keep in mind that some of these char​ ac​ ter​ is​ tics might be com​ bined. and each has a spe​ cific el​ e​ ment of the in​ fra​ struc​ ture in which they have spe​ cific ex​ per​ tise and about which they care most. Au​ toma​ tion adds speed to repet​ it​ ive tasks and elim​ in ​ates er​ rors or missed steps. Who. In any IT or​ ga​ ni​ za​ tion. while re​ mov​ ing mun​ dane. To achieve the de​ sired re​ sult. This sec​ tion dis​ cusses the rel​ e​ vant as​ pects of these monikers as it re​ lates to ACI fab​ ric op​ er​ at​ ions and how a com​ pany such as ACME Inc. ACME is look​ ing to sim​ plify how it op​ er​ ates in​ fra​ struc​ ture.

and avail​ abil​ ity of ap​ pli​ ca​ tions for the end users. . His​ tor​ ic ​ally. the team is the most in​ ter​ ested in de​ cou​ pling over​ loaded net​ work con​ structs and re​ turn​ ing to the spe​ cific net​ work prob​ lems that the team was in​ tended to solve. The team is also in​ ter​ ested in al​ low​ ing more trans​ parency in the per​ for​ mance of the net​ work for​ ward​ ing. and the team is mak​ ing key met​ rics avail​ able on de​ mand in a self-ser​ vice ca​ pac​ ity. ACME's Stor​ age Team is pri​ mar​ ily fo​ cused on de​ liv​ ery of data stor​ age re​ sources to the or​ ga​ ni​ za​ tion.16 Introduction not be ev​ id ​ent to the end-user. The ca​ pa​ bil​ it​ ies pro​ vided by the ACI fab​ ric allow them to con​ fi​ dently de​ ploy newer IP-based stor​ age and clus​ ter​ ing tech​ nolo​ gies. The De​ vel​ op​ ment part of the team will be writ​ ing the mo​ bile ap​ pli​ ca​ tion soft​ ware plat​ form. per​ for​ mance. Both parts of this team will need to work closely with the other teams in this sec​ tion to en​ able the best de​ sign. These ca​ pa​ bil​ it​ ies are pri​ mar​ ily dis​ cussed in the mon​ it​ or​ ing sec​ tions. What the team needs to know is how to con​ fig​ ure the net​ work​ ing con​ structs. The team is also very in​ ter​ ested in being able to see how the stor​ age ac​ cess is per​ form​ ing and would like to be no​ ti​ fied in the event of con​ tention. all while main​ tain​ ing high lev​ els of avail​ abil​ ity. With ACI. ACME's De​ vel​ op​ ment and Ap​ pli​ ca​ tion Team is fo​ cused on the soft​ ware and ap​ pli​ ca​ tions the com​ pany uses in​ ter​ nally and the soft​ ware that it de​ liv​ ers to its cus​ tomers. and so on. The team is chal​ lenged with jug​ gling ap​ pli​ ca​ tion re​ quire​ ments. ACME's Net​ work Team is pri​ mar​ ily fo​ cused on cre​ at​ ing and man​ ag​ ing net​ work​ ing con​ structs to for​ ward pack​ ets at Layer 2 (MAC/switch​ ing) and Layer 3 (IP rout​ ing). The team typ​ ic ​ally has some spe​ cific re​ quire​ ments around QoS. while al​ low​ ing other groups to lever​ age their spe​ cific ex​ per​ tise to ma​ nip​ ul​ ate se​ cu​ rity and ap​ pli​ ca​ tion level poli​ cies. The Ap​ pli​ ca​ tion part of the team con​ tains ap​ pli​ ca​ tion own​ ers and sub​ ject mat​ ter ex​ perts that en​ sure other busi​ ness units are able to do their jobs by uti​ liz​ ing the busi​ ness ap​ pli​ ca​ tions avail​ able. the en​ tire or​ ga​ ni​ za​ tion should be seen as one team with a com​ mon goal of de​ liv​ er​ ing value to their or​ ga​ ni​ za​ tion. In​ stead. ACI will pro​ vide the stor​ age team with the vis​ ib ​il​ ity they will re​ quire. multi-pathing. how to ver​ ify for​ ward​ ing. The stor​ age team has been very suc​ cess​ ful in main​ tain​ ing very tight SLAs and has tra​ di​ tion​ ally man​ aged sep​ ar​ ate in​ fra​ struc​ ture for stor​ age ac​ cess. and how to trou​ bleshoot net​ work for​ ward​ ing as​ pects in the fab​ ric. man​ ag​ ing SLA. and as​ sist​ ing in the en​ force​ ment of in​ for​ ma​ tion se​ cu​ rity. how to tie Layer 2 to Layer 3. The stor​ age team is con​ cerned with pro​ tect​ ing the data in terms of avail​ abil​ ity. the team had to worry about de​ liv​ er​ ing a stor​ age fab​ ric in ad​ di​ tion to man​ ag​ ing stor​ age de​ vices them​ selves. as well as mak​ ing sure that sen​ si​ tive data is se​ cure.

How a tool is se​ lected in ACI will often be a prod​ uct of what is being done and the as​ pects of how the tool is used. What The as​ pect of "what" can be looked at in many dif​ fer​ ent ways. and has been re​ spon​ si​ ble for per​ form​ ing vul​ ner​ ab ​il​ ity as​ sess​ ment and data clas​ si​ fi​ ca​ tion ef​ forts. or some com​ bi​ na​ tion of these. but the fab​ ric man​ age​ ment is rooted in an ab​ stracted ob​ ject model that pro​ vides a more flex​ ib ​le base. Ad​ di​ tion​ ally. script​ ing. Due to the sen​ si​ tiv​ ity of this in​ for​ ma​ tion and the se​ cu​ rity as​ pects of the ACI fab​ ric. the ap​ pli​ ca​ tion de​ vel​ op​ ers are in​ creas​ ingly in​ ter​ ested in lever​ ag​ ing Linux con​ tainer tech​ nolo​ gies to allow for even greater ap​ pli​ ca​ tion porta​ bil​ ity. the op​ er​ at​ or of the fab​ ric can choose from mul​ ti​ ple modes of man​ age​ ment. pro​ gram​ ming. has tra​ di​ tion​ ally been en​ gaged late in an ap​ pli​ ca​ tion de​ ploy​ ment process. This is timely as the ap​ pli​ ca​ tion roll​ out will have both vir​ tu​ al​ ized and non-vir​ tu​ al​ ized work​ loads. the new ap​ pli​ ca​ tion will be stor​ ing sen​ si​ tive cus​ tomer in​ for​ ma​ tion. The In​ for​ ma​ tion Se​ cu​ rity Team at ACME Inc. CLI. In ACI there are some el​ e​ ments of the tra​ di​ tional tools.Introduction 17 The Com​ pute and Vir​ tu​ al​ iza​ tion Team at ACME Inc. but the main con​ cept in the con​ text of this book is what tools are used to man​ age op​ er​ at​ ions of an ACI fab​ ric. For ex​ am​ ple. The team also re​ cently em​ ployed new con​ fig​ ur​ a​ tion man​ age​ ment tools to ac​ count for new work​ loads that fell out​ side of the vir​ tu​ al​ iza​ tion ef​ fort to get sim​ il​ ar agility for bare metal servers that the team gained from its vir​ tu​ al​ iza​ tion ef​ forts. the In​ for​ ma​ tion Se​ cu​ rity Team is able to pro​ vide input ear​ lier in the process and avoid re-do​ ing work be​ cause of se​ cu​ rity or com​ pli​ ance is​ sues. These ca​ pa​ bil​ it​ ies are dis​ cussed fur​ ther in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter. API in​ te​ gra​ tion. if an op​ er​ at​ ions staff is try​ ing to gather a bunch of in​ for​ ma​ tion across a num​ ber of in​ ter​ faces and switches or is man​ ag​ ing the con​ fig​ ur​ a​ - . and these tools in​ te​ grate into man​ age​ ment plat​ forms and con​ fig​ ur​ a​ tion and man​ age​ ment processes. you have some tra​ di​ tional tools. Role Based Ac​ cess Con​ trol (RBAC). to man​ age net​ work op​ er​ at​ ions. in​ clud​ ing credit card num​ bers. mon​ it​ or​ ing. With the cur​ rent pro​ ject. The In​ for​ ma​ tion Se​ cu​ rity Team is in​ ter​ ested in the op​ er​ at​ ional as​ pects of the ACI se​ cu​ rity model as it re​ lates to the fol​ low​ ing ca​ pa​ bil​ it​ ies: ten​ ancy. The Com​ pute and Vir​ tu​ al​ iza​ tion teams are in​ ter​ ested in ACI for its abil​ ity to pro​ vide com​ mon ac​ cess to phys​ ic ​al and vir​ tual servers. such as CLI and SNMP. is wrap​ ping up a major ini​ tia​ tive to vir​ tu​ al​ ize the server farms that it is re​ spon​ si​ ble for main​ tain​ ing. With this base. In a tra​ di​ tional net​ work. such as GUI. and Layer 4 to Layer 7 ser​ vices. al​ low​ ing the team to pub​ lish end​ point groups to vir​ tu​ al​ iza​ tion clus​ ters from a cen​ tral​ ized place across mul​ ti​ ple hy​ per​ vi​ sors.

Introduction 18 tion of many dif​ fer​ ent ob​ jects at once. There are mech​ an ​isms for backup and re​ store that will be dis​ cussed in fol​ low-on chap​ ters. When "When" refers to when the teams listed above are in​ volved in the plan​ ning. It is a good idea to in​ volve the dif​ fer​ ent teams early when build​ ing poli​ cies and processes for how the fab​ ric is im​ ple​ mented and then man​ aged. An eval​ ua ​t​ ion of cur​ rent change con​ trol and con​ tin​ uo ​us in​ te​ gra​ tion/de​ liv​ ery strate​ gies is war​ ranted as op​ er​ at​ ional pro​ ce​ dures evolve. How "How" an​ swers the fol​ low​ ing basic ques​ tions: • How does a networking person go about configuring the network forwarding? • How does the compute team get information from the infrastructure to make • How do the application team track performance and usage metrics? • How does a storage team track the access to storage subsystems and ensure optimal workload placement decisions? that it is performant? When "how" in​ volves mak​ ing a change to the con​ fig​ ur​ a​ tion of an en​ vi​ ron​ ment. The ACI pol​ icy model has been de​ signed to re​ duce the over​ all size of a fault do​ main and pro​ vide a mech​ an ​ism for in​ cre​ men​ tal change. Through​ out this book we will high​ light the meth​ ods and pro​ ce​ dures to proac​ tively and re​ ac​ tively man​ age the fab​ ric. This is a key dif​ fer​ ence be​ tween ACI and tra​ di​ tional processes that were very se​ ri​ al in na​ ture. The col​ lab​ o​ ra​ tive na​ ture of ACI al​ lows for a high de​ gree of par​ al​ leliza​ tion of work flow. re​ sult​ ing in a longer de​ ploy​ ment time for ap​ pli​ ca​ tions and a higher mean-time to res​ ol​ u​ tion when is​ sues arise. FCAPS. We will also dis​ cuss the model and which ob​ jects af​ fect the ten​ ants and the fab​ ric as a whole. As a base​ line. most or​ ga​ ni​ za​ tions are im​ ple​ ment​ ing some kind of struc​ tured changecon​ trol method​ ol​ ogy to mit​ ig ​ate busi​ ness risk and en​ hance sys​ tem avail​ abil​ ity. whereas sim​ ple dash​ board mon​ it​ or​ ing might be more suited to a GUI. There are a num​ ber of change/IT man​ age​ ment prin​ ci​ ples (Cisco Life​ cy​ cle ser​ vices. Change con​ trol is a fact of life in the mis​ sioncrit​ ic ​al en​ vi​ ron​ ments that ACI has been de​ signed to sup​ port. script​ ing might be more ef​ fi​ cient. an im​ por​ tant con​ sid​ er​ at​ ion is change con​ trol. .

in​ fra​ struc​ ture) and are per​ formed to re​ store ser​ vice to af​ fected sys​ tems. and then (as​ sum​ ing it is au​ tho​ rized) planned and im​ ple​ mented. Ul​ ti​ mately each change must be eval​ ua ​ted pri​ mar​ ily in terms of both its risk and value to the busi​ ness. and en​ sur​ ing that de​ liv​ ery teams are work​ ing on the most valu​ able thing they could be at any given time. soft​ ware. The multi-ten​ ant and role-based ac​ cess con​ trol fea​ tures in​ her​ ent to the ACI so​ lu​ tion allow the iso​ la​ tion or draw​ ing of a very clean box around the scope and im​ pact of the changes that can be made. Nor​ mal changes are those that go through the reg​ ul​ ar change man​ age​ ment process. there are three fun​ da​ men​ tal kinds of changes: • Emergency changes • Normal • Standard Emer​ gency changes are by de​ f​ i​ n​ i​ tion a re​ sponse to some kind of tech​ ni​ cal out​ age (hard​ ware. Ap​ ply​ ing change man​ age​ ment prin​ ci​ ples based on tech​ nol​ ogy from five years ago would not en​ able the rapid de​ ploy​ ment of tech​ nol​ ogy that ACI can de​ liver. based on feed​ back from users. and then ei​ ther au​ tho​ rized or re​ jected. Train​ ing op​ er​ a​ tions teams on norms (a stated goal of this book) is also key. A com​ mon sense ap​ proach to change man​ age​ ment and con​ tin​ uo ​us in​ te​ gra​ tion should be a premise that is dis​ cussed early in the de​ sign and im​ ple​ men​ ta​ tion cycle be​ fore hand​ ing the fab​ ric to the op​ er​ a-​ tions teams for day-to-day main​ te​ nance. Con​ tin​ uo ​us de​ liv​ ery does ex​ actly this by en​ sur​ ing that re​ leases are per​ formed reg​ ul​ arly from early on in the de​ liv​ ery process. In the In​ for​ ma​ tion Man​ age​ ment Sys​ tems world. as​ sessed.Introduction 19 and ITIL) that are good guides from which to start. which starts with the cre​ ation of a re​ quest for change which is then re​ viewed. and pro​ vi​ sion​ ing. see the RBAC sec​ tion of this book. For more de​ tails. A way to en​ able a low-over​ head change man​ age​ ment process is to re​ duce the risk of each change and in​ crease its value. mon​ it​ or​ ing. In an ACI en​ vi​ ron​ ment a nor​ mal change could apply to any​ thing within the fol​ low​ ing com​ po​ nents: • Fabric Policies (fabric internal and access will be discussed in detail later) • Configuration objects in the Common tenant that are shared with all other tenants (things that affect the entire fabric) Private Networks .

. This en​ ables the change to be re​ verted quickly. and the process for man​ ag​ ing them. but are rep​ re​ sen​ ta​ tive of com​ mon tasks per​ formed day-to-day and week-to-week. The abil​ ity to audit changes that are hap​ pen​ ing to the en​ vi​ ron​ ment is a re​ quire​ ment for ACME Inc. As with nor​ mal changes. This is a key trou​ bleshoot​ ing tool for "when some​ thing mag​ ic ​ally stops work​ ing". cor​ re​ lat​ ing this to any faults that re​ sult from said change. Im​ me​ di​ ate ac​ tion should be to check the audit log as it will tell who made what change and when.Introduction 20 Bridge Domains Subnets • Virtual Machine Manager (VMM) integrations • Layer 4 to Layer 7 devices Device packages Creation of logical devices Creation of concrete devices • Layer 2 or Layer 3 external configuration • Attachable Entity Profile (AEP) creation • Server or external network attachment • Changes to currently deployed contracts and filters that would materially change the way traffic flows Stan​ dard changes are low-risk changes that are pre-au​ tho​ rized. APIC main​ tains an audit log for all con​ fig​ ur​ a​ tion changes to the sys​ tem. they must still be recorded and ap​ proved. who is al​ lowed to ap​ prove them. the cri​ te​ ria for a change to be con​ sid​ ered "stan​ dard". Each or​ ga​ ni​ za​ tion will de​ cide the kind of stan​ dard changes that they allow. In the ACI en​ vi​ ron​ ment some ex​ am​ ples of "stan​ dard" changes could be: • Tenant creation • Application profile creation • Endpoint group (EPG) creation • Contracts scoped at a tenant level • Layer 4 to Layer 7 service graphs • Domain associations for EPGs The items men​ tioned above are not in​ tended to be all-in​ clu​ sive.

to com​ plete these tasks in par​ al​ lel with the var​ i​ ous stake​ hold​ ers who are high​ lighted through​ out. ACI addresses IT requirements from across the organization . The re​ main​ der of this book an​ swers these ques​ tions. to pro​ vide the great​ est value to the cus​ tomer.Introduction 21 A more in-depth dis​ cus​ sion of con​ tin​ u​ ous de​ liv​ ery in the con​ text of in​ fra​ struc​ ture man​ age​ ment is out​ side of the scope of this book. there is a more in-depth sec​ tion at the end that ex​ plains how to use the ACI API to au​ to​ mate most op​ er​ a​ tional tasks. the most im​ por​ tant thing is for these groups to work in​ sieme (to​ gether). user. and ul​ ti​ mately the busi​ ness. ACI en​ ables ACME Inc. The book is laid out in a spe​ cific order. While some script​ ing op​ por​ tu​ ni​ ties are called out through​ out the book. pro​ vid​ ing you with a frame​ work of how to take the con​ cepts and pro​ ce​ dures and apply them to sim​ i​ lar ini​ tia​ tives within your or​ ga​ ni​ za​ tions. How​ ever. While or​ ga​ ni​ za​ tional struc​ tures might be siloed into these teams. and this book il​ lus​ trates how the stake​ hold​ ers can work to​ gether in a more col​ lab​ o​ ra​ tive man​ ner than they have in the past.

.

23 Management .

.

Management Section Content • APIC Overview • Configuring Management Protocols Cisco Discovery Protocol Link Layer Discovery Protocol Time Synchronization and NTP Out-of-Band Management NTP In-Band Management NTP Verify NTP Operation Verifying that the NTP Policy Deployed to Each Node Domain Name Services (DNS) Verifying DNS Operation • Role-Based Access Control Multiple Tenant Support User Roles Security Domains Creation of a Security Domain Adding Users Remote Authentication • Import and Export Policies Configuration Export (Backup) Add a Remote Location (SCP) Create a One Time Export Policy Verify Export Policy was Successful Extract and View Configuration Files Configuration Import (Restore/Merge) 25 .

.

with no risk of in​ con​ sis​ tency be​ tween the var​ io ​us data in​ ter​ faces. and de​ cou​ ple the switch​ ing hard​ ware from the de​ sired con​ fig​ ur​ a​ tion in​ tent. These changes in​ clude: • Single point of management controller-based architecture • Stateless hardware • Desired state-driven eventual consistency model The sin​ gle point of man​ age​ ment within the ACI ar​ chi​ tec​ ture is known as the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). Each node re​ ceives a unique node iden​ ti​ fier. and net​ work man​ age​ ment is sim​ pli​ fied. and ap​ pli​ ca​ tion pro​ gram​ ming in​ ter​ face (API). mean​ ing that hard​ ware swaps can be faster. Hav​ ing a cen​ tral​ ized con​ troller with an ap​ pli​ ca​ tion pro​ gram​ ming in​ ter​ face (API) means that all func​ tions con​ fig​ ured or ac​ cessed through the fab​ ric can be uni​ formly ap​ proached through a graph​ ic ​al user in​ ter​ face (GUI). which mod​ if​ ies the con​ tents of a syn​ chro​ nized data​ base that is repli​ cated across APICs in a clus​ ter and pro​ vides an ab​ strac​ tion layer be​ tween all of the in​ ter​ faces. topol​ ogy changes are less im​ pact​ ful. mon​ it​ or​ ing. and not from the se​ ri​ al num​ ber of the chas​ sis. re​ duce the num​ ber of touch​ points. and health func​ tions. This con​ troller pro​ vides ac​ cess to all con​ fig​ ur​ a​ tion. The de​ vice can also be sub​ sti​ tuted in a state​ less fash​ ion. com​ mand line in​ ter​ face (CLI). This con​ troller-based ar​ chi​ tec​ ture also makes pos​ si​ ble a state​ less con​ fig​ ur​ a​ tion model that de​ cou​ ples the hard​ ware from the con​ fig​ ur​ a​ tion run​ ning on it. The de​ sired state model for con​ fig​ ur​ a​ tion fur​ ther com​ ple​ ments these con​ cepts of con​ troller-based man​ age​ ment and state​ less​ ness by tak​ ing ad​ van​ tage of a con​ cept known as de​ clar​ at​ ive con​ trol-based man​ age​ ment. man​ age​ ment. which al​ lows for the de​ vice to down​ load the cor​ rect con​ fig​ ur​ a​ tion at​ trib​ utes from the con​ troller. nor from a con​ fig​ ur​ a​ tion file re​ sid​ ing on the de​ vices. based on a con​ cept known as the promise . This trans​ lates to an APIC clus​ ter that man​ ages in​ di​ vid​ ual fab​ ric nodes of leaf and spine switches that de​ rive their iden​ tity from what the con​ troller de​ fines as being the de​ sired in​ tent. This re​ sults in a clean and pre​ dictable tran​ si​ tion be​ tween the in​ ter​ faces. These dif​ fer​ ences serve to sim​ plify the man​ age​ ment greatly. The un​ der​ ly​ ing in​ ter​ face for all ac​ cess meth​ ods is pro​ vided through a REST-based API.Management 27 APIC Overview There are a num​ ber of fun​ da​ men​ tal dif​ fer​ ences be​ tween the op​ er​ at​ ions of tra​ di​ tional net​ work​ ing hard​ ware and an ACI fab​ ric.

28 Management the​ ory. De​ clar​ at​ ive con​ trol dic​ tates that each ob​ ject is asked to achieve a de​ sired state and makes a "promise" to reach this state with​ out being told pre​ cisely how to do so. This stands in con​ trast with the tra​ di​ tional model of im​ per​ at​ ive con​ trol. . A sys​ tem based on de​ clar​ at​ ive con​ trol is able to scale much more ef​ fi​ ciently than an im​ per​ at​ ive-based sys​ tem. where each man​ aged el​ e​ ment must be told pre​ cisely what to do. dic​ tated by the man​ ag​ ing con​ troller. and take into ac​ count the spe​ cific sit​ ua ​t​ ional as​ pects that will im​ pact its abil​ ity to get from its cur​ rent state to the con​ fig​ ured state. be told how to do it. since each en​ tity within the do​ main is re​ spon​ si​ ble for know​ ing its cur​ rent state and the steps re​ quired to get to the de​ sired state.

providing name resolution to fabric components consistent with the application and server teams. ACME has existing NTP servers and will leverage them for maintaining time in their ACI fabric deployment.These policies are primarily consumed by the network team. With ACI. such as when to cre​ ate switch pro​ files to con​ nect to Cisco UCS B Se​ ries servers. there may be times when you must pro​ vi​ sion a par​ tic​ ul​ ar host-fac​ ing port man​ ua ​lly with a CDP on or off pol​ icy. • Link Layer Discovery Protocol (LLDP) .Management 29 Configuring Management Protocols For the op​ ti​ miza​ tion of ACME's Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. • Network Time Protocol (NTP) . maintaining accurate time is of increased importance as accurate time is a prerequisite for features such as atomic counters. Layer 4 to Layer 7 services devices. It is also im​ por​ tant to note that CDP might be re​ quired for cer​ tain con​ fig​ ur​ a​ tions.Discovery protocols are no longer limited to just network devices. the ACI fab​ ric pro​ vides a de​ fault pol​ icy where CDP is dis​ abled. ACME is deploying LLDP on servers. the net​ work team cre​ ated the fol​ low​ ing man​ age​ ment pro​ to​ col poli​ cies: • Cisco Discovery Protocol (CDP) . Cisco Discovery Protocol Cisco Dis​ cov​ ery Pro​ to​ col (CDP) is a valu​ able source of in​ for​ ma​ tion in trou​ bleshoot​ ing phys​ ic ​al and data-link layer is​ sues. • Domain Name Services (DNS) . . man​ ua ​lly cre​ ate a “CDP-EN​ ABLED" pol​ icy with CDP en​ abled. Therefore. these policies will be available for consumption by all of the teams.Maintaining accurate time across the application logs and infrastructure components is extremely important for being able to correlate events. the important ACI components are also registered with ACME's DNS server so teams can easily access them in their management tasks. and storage arrays. As a rec​ om​ mended prac​ tice. In the course of ACI op​ er​ at​ ions. By de​ fault. as well as a “CDP-DIS​ ABLED” pol​ icy with CDP dis​ abled that can be ref​ er​ enced through​ out all in​ ter​ face pol​ icy con​ fig​ ur​ a​ tions. LLDP is a standards-based protocol for discovering topology relationships. as CDP is the standard for their existing network equipment. In addition to be able to resolve names within the fabric.

by de​ fault. 3 In the Work pane. 5 6 In the Work pane. Link Layer Discovery Protocol Link Layer Dis​ cov​ ery Pro​ to​ col (LLDP) is a valu​ able source of in​ for​ ma​ tion for trou​ bleshoot​ ing phys​ ic ​al and data-link layer is​ sues. choose Actions > Create CDP Interface Policy. c. 4 In the Create LLDP Interface Policy dialog box.Management 30 To cre​ ate CDP poli​ cies: 1 On the menu bar. b. choose Interface Policies > Policies > LLDP Interface. In the course of ACI op​ er​ at​ ions. In the Name field enter the name of the policy. Click Submit. 3 In the Work pane. when con​ nect​ ing to Cisco Uni​ fied Com​ put​ ing Sys​ tem (UCS) Blade servers. Your CDP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric in​ ter​ faces. choose Fabric > Access Policies. There might be times in the op​ er​ at​ ion of the ACI fab​ ric in which you must man​ ua ​lly ad​ just the LLDP pro​ to​ col to con​ form with in​ ter​ op​ er​ abil​ ity re​ quire​ ments of end-host de​ vices. 2 In the Navigation pane. you must dis​ able LLDP. 2 In the Navigation pane. such as "CDP-ENABLED". perform the following actions: a. In the Name field enter the name of the policy. click Disabled. 4 In the Create CDP Interface Policy dialog box. the LLDP fea​ ture is en​ abled on the fab​ ric. To cre​ ate an LLDP pol​ icy: 1 On the menu bar. click Enabled. choose Actions > Create CDP Interface Policy. For the Admin State radio buttons. perform the following actions: a. For the Admin State radio buttons. c. For ex​ am​ ple. choose Actions > Create LLDP Interface Policy. choose Interface Policies > Policies > CDP Interface. b. Cisco rec​ om​ mends that you pre-pro​ vi​ sion LLDP en​ able and dis​ able pro​ to​ col poli​ cies to make fu​ ture in​ ter​ face pol​ icy de​ ploy​ ment de​ ci​ sions stream​ lined and eas​ ily con​ fig​ ured. such as "CDP-DISABLED". choose Fabric > Access Policies. perform the following actions: . In the Create CDP Interface Policy dialog box. Click Submit.

9 In the Create LLDP Interface Policy dialog box. Click Submit. d. click Disabled. Nonex​ is​ tent or im​ proper con​ fig​ ur​ a​ tion of time syn​ chro​ niza​ tion does not nec​ es​ sar​ ily trig​ ger a fault or a low health score. Clock syn​ chro​ niza​ tion is im​ por​ tant for proper analy​ sis of traf​ fic flows as well as for cor​ re​ lat​ ing debug and fault time​ stamps across mul​ ti​ ple fab​ ric nodes. For the Transmit State radio buttons. choose Actions > Create LLDP Interface Policy. enter "LLDP-TX-ON-RX-OFF". enter "LLDP-TX-OFF-RX-OFF". Click Submit. Your LLDP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric in​ ter​ faces. For the Transmit State radio buttons. click Disabled. clock syn​ chro​ niza​ tion al​ lows for the full uti​ liza​ tion of the atomic counter ca​ pa​ bil​ ity that is built into the ACI upon which the ap​ pli​ ca​ tion health scores de​ pend. perform the following actions: a. For the Receive State radio buttons. In the Name field. For the Receive State radio buttons.Management 31 a. In ad​ di​ tion. perform the following actions: a. b. Click Submit. For the Receive State radio buttons. In the Name field. Time Synchronization and NTP Within the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. b. click Enabled. and trou​ bleshoot​ ing tasks de​ pend. enter "LLDP-TX-ON-RX-ON". enter "LLDP-TX-OFF-RX-ON". click Enabled. click Disabled. For the Receive State radio buttons. An off​ set pre​ sent on one or more de​ vices can ham​ per the abil​ ity to prop​ erly di​ ag​ nose and re​ solve many com​ mon op​ er​ at​ ional is​ sues. For the Transmit State radio buttons. time syn​ chro​ niza​ tion is a cru​ cial ca​ pa​ bil​ ity upon which many of the mon​ it​ or​ ing. c. d. c. In the Create LLDP Interface Policy dialog box. In the Name field. perform the following actions: a. In the Create LLDP Interface Policy dialog box. d. b. click Disabled. Click Submit. 5 6 In the Work pane. click Enabled. c. b. click Enabled. You . In the Name field. op​ er​ at​ ional. For the Transmit State radio buttons. choose Actions > Create LLDP Interface Policy. d. c. 7 8 In the Work pane.

The most widely adapted method for syn​ chro​ niz​ ing a de​ vice clock is to use Net​ work Time Pro​ to​ col (NTP). such as "Production-NTP". 4 In the Create Date and Time Policy dialog box. only one pod per ACI fab​ ric is al​ lowed.Management 32 should con​ fig​ ure time syn​ chro​ niza​ tion be​ fore de​ ploy​ ing a full fab​ ric or ap​ pli​ ca​ tions so as to en​ able proper usage of these fea​ tures. con​ sider what man​ age​ ment IP ad​ dress scheme is in place within the ACI fab​ ric. 3 In the Work pane. and Minimum Polling Intervals. Prior to con​ fig​ ur​ ing NTP. and Maximum Polling Intervals. Date and Time poli​ cies are con​ fined to a sin​ gle pod and must be de​ ployed across all pods pro​ vi​ sioned in the ACI fab​ ric. a Date and Time pol​ icy must be cre​ ated that ref​ er​ ences an out-ofband man​ age​ ment end​ point group. choose Fabric > Fabric Policies. choose Actions > Create Date and Time Policy. enter all relevant information. in-band man​ age​ ment and/or out-of-band man​ age​ ment. c. Provide a name for the policy to easily distinguish between your environment's different NTP configurations. As of the pub​ lish​ ing of this doc​ um ​ent. 1 On the menu bar. This IP reach​ ab ​il​ ity will be lever​ aged so that each node can in​ di​ vid​ u​ ally query the same NTP server as a con​ sis​ tent clock source. . is man​ aged from out​ side the ACI fab​ ric. Description. con​ fig​ ur​ a​ tion of NTP will vary. perform the following actions: a. d. b. Out-of-Band Management NTP When an ACI fab​ ric is de​ ployed with out-of-band man​ age​ ment. choose Pod Policies > Policies. see the Re​ ac​ tive Mon​ it​ or​ ing Tools chap​ ter. Click Next. leaves and all mem​ bers of the APIC clus​ ter. To con​ fig​ ure NTP. Click the + sign to specify the NTP server information (provider) to be used. including the following fields: Name. In the Create Providers dialog box. The re​ li​ ab ​il​ ity of the source must be care​ fully con​ sid​ ered when de​ ter​ min​ ing if you will use a pri​ vate in​ ter​ nal clock or an ex​ ter​ nal pub​ lic clock. An​ other con​ sid​ er​ at​ ion in de​ ploy​ ing time syn​ chro​ niza​ tion where the time source is lo​ cated. 2 In the Navigation pane. in​ clu​ sive of spines. each node of the fab​ ric. De​ pend​ ing on which man​ age​ ment op​ tion was cho​ sen for the fab​ ric. There are two op​ tions for con​ fig​ ur​ ing man​ age​ ment of all ACI nodes and Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ trollers (APICs). For more in​ for​ ma​ tion on atomic coun​ ters.

by de​ f​ in ​i​ t​ ion. see the "In-Band Management NTP" section. Repeat steps c and d for each provider that you want to create. Pol​ icy con​ fig​ ur​ a​ tion can be ver​ if​ ied by these steps: 1 On the menu bar. choose Pod Policies > Policies > Date and Time > ntp_policy > server_name. e. To lever​ age an NTP server ex​ ter​ nal to the fab​ ric with in-band man​ age​ ment. 3 In the Work pane. In-Band Management NTP When an ACI Fab​ ric is de​ ployed with in-band man​ age​ ment. not reach​ able from any​ where out​ side the fab​ ric. you must con​ struct a pol​ icy to en​ able this com​ mu​ ni​ ca​ tion. Click OK. ii. iii. verify the details of the server. If you are creating multiple providers. In the Management EPG drop-down list. 2 In the Navigation pane.Management 33 i. choose Out-ofBand. click the Preferred check box for the most reliable NTP source. Your NTP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric nodes. If you have deployed in-band management. In-band IP ad​ dress​ ing used within the ACI fab​ ric is. if the NTP server is reachable by all nodes on the fabric through out-of-band management. Verifying NTP Operation Full ver​ if​ i​ ca​ tion of NTP func​ tion​ al​ ity is best ac​ com​ plished by lever​ ag​ ing both the ACI CLI and the APIC GUI. Start ver​ if​ y​ ing time syn​ chro​ niza​ tion con​ fig​ ur​ a​ tion by en​ sur​ ing that all poli​ cies are con​ fig​ ured prop​ erly in​ side of the APIC GUI. you can cre​ ate the NTP pol​ icy as es​ tab​ lished in the "Out-of-Band Man​ age​ ment NTP" sec​ tion. The steps used to con​ fig​ ure in-band man​ age​ ment poli​ cies are iden​ ti​ cal to those used to es​ tab​ lish an out-of-band man​ age​ ment pol​ icy: the dis​ tinc​ tion is around how to allow the fab​ ric to con​ nect to the NTP server. con​ sider the reach​ ab ​il​ ity of the NTP server from within the ACI in-band man​ age​ ment net​ work. . choose Fabric > Fabric Policies. The ntp_policy is the previously created policy. Once you have es​ tab​ lished a pol​ icy to allow com​ mu​ ni​ ca​ tion.

you should use the NX-OS Vir​ tual Shell that is pre​ sent in the APIC. open an SSH ses​ sion to the out-of-band man​ age​ ment IP in​ ter​ face of any of the APIC nodes. To ac​ cess the NX-OS Vir​ tual Shell. 1 SSH to an APIC in the fabric. in the Management EPG drop-down list. 3 In the Work pane. choose the appropriate management EPG. 1 On the menu bar.Management 34 Verifying that the NTP Policy Deployed to Each Node To ver​ ify that the pol​ icy has been suc​ cess​ fully de​ ployed to each node. 2 In the Navigation pane. 2 Press the Tab key twice after the entering the attach command to list all of the available node names: admin@apic1:~> attach <Tab> <Tab> 3 Log in to one of the nodes using the same password that you used to access the APIC. This is use​ ful when in​ te​ grat​ ing VMM do​ mains or other Layer 4 to Layer 7 de​ vices and the host​ name is ref​ er​ enced. ex​ e​ cute the "ver​ sion" com​ mand to ob​ tain a con​ sol​ id ​ated list of node host​ names. admin@apic1:~> attach node_name 4 View the NTP peer status: leaf-1# show ntp peer-status A reach​ able NTP server has its IP ad​ dress pre​ fixed by an as​ ter​ isk (*). choose Fabric > Fabric Policies. From this prompt. . Domain Name Services (DNS) Set​ ting up a DNS server al​ lows the APIC to re​ solve var​ io ​us host​ names to IP ad​ dresses. choose Global Policies > DNS Profiles > default. 5 Repeat steps 3 and 4 for each node on the fabric. and the delay is a non-zero value.

7 Repeat step 6 for each additional DNS Domain suffix. 6 Click + next to DNS Domains to add a DNS domain. Click Update.com nameserver 171. c. such as "cisco.Management 35 Note: The default is default (Out-of-Band). In the Address field.183 nameserver 173.36.70.131.com". Verifying DNS Operation 1 Check the resolv. click the check box to make this domain the default domain. 8 Click Submit. 5 Repeat step 4 for each additional DNS provider.conf file from the APIC CLI: admin@apic1:~> cat /etc/resolv. b. Click Update. In the Preferred field. a. Note: You can have only one preferred provider. b. 4 Click + next to DNS Providers to add a DNS provider. a.conf # Generated by IFC search cisco. In the Default field . enter the provider address.10 . click the check box if you want to have this address as the preferred provider. In the Name field. enter the domain name. Note: You can have only one domain name as the default.168. c.

163.161): icmp_seq=2 ttl=241 time=42. admin@apic1:~> ping cisco.903/0.4.4.163.com (72.cisco.7 ms 64 bytes from www1.4 ms 64 bytes from www1.161): icmp_seq=1 ttl=241 time=42.Management 36 2 Ping a host by DNS name that will be reachable from the APIC out-of-band management.cisco.163.com ping statistics --3 packets transmitted.163.161) 56(84) bytes of data. 64 bytes from www1.161): icmp_seq=3 ttl=241 time=43.4.com (72.485/43.com PING cisco.cisco.9 ms ^C --.038/43.4.619 ms admin@apic1:~> . 0% packet loss. time 2102ms rtt min/avg/max/mdev = 42.com (72.cisco.com (72. 3 received.

whereas a ten​ ant-ad​ min can only have ac​ cess to the com​ po​ nents within the ten​ ant to which they're as​ so​ ci​ ated. In an ACI fab​ ric. What. the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) grants ac​ cess to all ob​ jects based on a user's es​ tab​ lished role in the con​ fig​ ur​ a​ tion. you can im​ ple​ ment very gran​ ul​ ar ac​ cess con​ trol in the sys​ tem. and Where. and is un​ able to man​ age other user roles at a fab​ ric wide level. privileges and privilege types (which can be "no access". a priv​ il​ ege type. Each user can be as​ signed to a set of roles. When com​ bin​ ing these ob​ jects. For ex​ am​ ple. The APIC pro​ vides ac​ cess ac​ cord​ ing to a user’s role through RBAC. ACME's IT or​ ga​ ni​ za​ tion fits per​ fectly into this ac​ cess con​ trol model.Management 37 Role-Based Access Control Role-based ac​ cess con​ trol (RBAC) and ten​ ancy are im​ por​ tant con​ cepts to mas​ ter when op​ er​ at​ ing a Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. An ACI fab​ ric user is as​ so​ ci​ ated with the fol​ low​ ing: • A set of roles • For each role. Role-based ac​ cess and the con​ cept of ten​ ancy are two core foun​ da​ tions upon which the ACI man​ age​ ment and pol​ icy mod​ els are built. a fab​ ad​ min can have ac​ cess to the en​ tire fab​ ric as well as to as​ sign​ ing other user roles and as​ so​ ci​ ate them to se​ cu​ rity do​ mains. and the se​ cu​ rity do​ main de​ fines Where the user can per​ form these ac​ tions. The role is the Who (log​ ic ​al col​ lec​ tion of priv​ il​ eges). and a se​ cu​ rity do​ main. or "read-write") information tree (MIT) that a user can access . Think of this in terms of Who. "read- • One or more security domain tags that identify the portions of the management only". priv​ il​ eges de​ fine What func​ tions a user can per​ form. ricThe role clas​ si​ fi​ ca​ tion is an es​ tab​ lished group​ ing of per​ mis​ sions.

These con​ fig​ u​ ra​ tions can be im​ ple​ mented using the REST API.38 Management Authenticated user creation lifecycle The ACI fab​ ric man​ ages ac​ cess priv​ i​ leges at the man​ aged ob​ ject (MO) level. A priv​ i​ lege is an MO that en​ ables or re​ stricts ac​ cess to a par​ tic​ u​ lar func​ tion within the sys​ tem. For ex​ am​ ple. Read/write re​ stric​ tions pre​ vent any ten​ ant from see​ ing any other ten​ ant’s con​ fig​ u​ ra​ tion. fab​ ric-equip​ ment is a priv​ i​ lege flag. This flag is set by the APIC on all ob​ jects that cor​ re​ spond to equip​ ment in the phys​ i​ cal fab​ ric. or GUI. The com​ bi​ na​ tion of user priv​ i​ leges. and se​ cu​ rity do​ mains with ac​ cess rights in​ her​ i​ tance en​ ables ad​ min​ is​ tra​ tors to con​ fig​ ure AAA func​ tions at the man​ aged ob​ ject level in a very gran​ u​ lar fash​ ion. and ac​ count​ ing (AAA) func​ tions of the Cisco ACI fab​ ric. sta​ tis​ tics. CLI. APIC poli​ cies man​ age the ac​ cess. au​ then​ ti​ ca​ tion. roles. Multiple Tenant Support A core APIC in​ ter​ nal data ac​ cess con​ trol sys​ tem pro​ vides multi-ten​ ant iso​ la​ tion and pre​ vents in​ for​ ma​ tion pri​ vacy from being com​ pro​ mised across ten​ ants. .

choose Admin > AAA. then users in the solar ten​ ant can ac​ cess the VMM do​ main ac​ cord​ ing to their ac​ cess rights. 1 On the menu bar. Cisco rec​ om​ mends that you con​ fig​ ure se​ cu​ rity do​ mains and ac​ cess lev​ els prior to de​ ploy​ ment of ten​ ants. it will be pos​ si​ ble to com​ pletely sep​ ar​ ate con​ fig​ ur​ a​ tion of ap​ pli​ ca​ tion work​ loads while only pro​ vid​ ing ac​ cess to those who need it. Note: From here. you can create a custom role from the Actions menu. choose Security Management > Roles. If you need to make se​ lec​ tive changes to allow ac​ cess out​ side of a user's se​ cu​ rity do​ main. By using se​ cu​ rity do​ mains. The poli​ cies can​ not cur​ rently be ap​ plied to in​ di​ vid​ ual ob​ jects de​ spite ref​ er​ ences at the ob​ ject level in​ side of the RBAC screen in the APIC. . be sure to set up a dis​ crete user ac​ cess pol​ icy just for that com​ mu​ ni​ ca​ tion. faults. A key con​ cept to keep in mind when con​ fig​ ur​ ing se​ cu​ rity do​ mains is that the con​ fig​ u-​ ra​ tion only ap​ plies at a ten​ ant level. sta​ tis​ tics. users can be or​ ga​ nized into var​ io ​us per​ mis​ sion struc​ tures. or event data. most com​ monly ap​ plied to ten​ ants. 2 In the Navigation pane. For ex​ am​ ple. Un​ less the ad​ min​ is​ tra​ tor as​ signs per​ mis​ sions to do so. poli​ cies. the users con​ tained in the se​ cu​ rity do​ main can ac​ cess the cor​ re​ spond​ ing VMM do​ main. ten​ ants are re​ stricted from read​ ing fab​ ric con​ fig​ ur​ a​ tion. Security Domains The se​ cu​ rity do​ main con​ cept is cru​ cial to proper op​ er​ at​ ion of the ACI fab​ ric. Using the ten​ ancy ca​ pa​ bil​ it​ ies of the ACI fab​ ric in con​ junc​ tion with prop​ erly con​ fig​ ured se​ cu​ rity do​ mains. Cisco does not rec​ om​ mend that you pro​ vide user ac​ cess con​ fig​ u-​ ra​ tion by mod​ if​ ing per​ mis​ sions of the "all" do​ main. if a ten​ ant named solar is tagged with the se​ cu​ rity do​ main called sun and a VMM do​ main is also tagged with the se​ cu​ rity do​ main called sun. you can see each built-in role and the associated privileges.Management 39 faults. User Roles You can view the built-in user roles as well as cre​ ate cus​ tom roles to meet spe​ cific re​ quire​ ments. or events. Additionally. If a vir​ tual ma​ chine man​ age​ ment (VMM) do​ main is tagged as a se​ cu​ rity do​ main. Changes in the "all" do​ main will af​ fect ac​ cess per​ mis​ sions for all users.

For ex​ am​ ple. Adding Users You might need to add new users within the ACI fab​ ric. The "mgmt" se​ cu​ rity do​ main is for man​ age​ ment traf​ fic poli​ cies. "com​ mon". Give the security domain a name and an optional description. choose Local and click Submit if Local is not already chosen. for a multi-ten​ ant en​ vi​ ron​ ment. choose Actions > Create a Security Domain. Ac​ tive Di​ rec​ tory. Con​ fig​ ure Local Users: 1 On the menu bar. and "mgmt". For more in​ for​ ma​ tion about the MIT. or use some​ thing such as LDAP.​ html To cre​ ate a se​ cu​ rity do​ main: 1 On the menu bar. choose AAA Authentication. choose Admin > AAA.​ cisco. . RBAC poli​ cies are con​ fig​ ured under the "Admin" tab of the APIC GUI. You can add se​ cu​ rity do​ mains as nec​ es​ sary. such as DNS or di​ rec​ tory ser​ vices. choose Admin > AAA. The "all" se​ cu​ rity do​ main usu​ ally in​ cludes ac​ cess to every​ thing within the man​ age​ ment in​ for​ ma​ tion tree (MIT).​ com/​ c/​ en/​ us/​ products/​ collateral/​ cloud-systems-management/​ aci-fabric-controller/​ white-paper-c11-729586. ACI can have local users man​ u-​ ally en​ tered by an admin. perform the following actions: a. 2 In the Navigation pane. RA​ DIUS. 3 In the Work pane. Users are then cre​ ated and as​ signed to cer​ tain se​ cu​ rity do​ mains. a se​ cu​ rity do​ main would be cre​ ated for each ten​ ant. or TACACS+ to spec​ ify users that will be al​ lowed to man​ age cer​ tain parts of the ACI net​ work. in the Realm drop-down list. 4 In the Create a Security Domain dialog box.Management 40 Creation of a Security Domain There are three se​ cu​ rity do​ mains that are pro​ vi​ sioned by de​ fault on the ACI fab​ ric: "all". "Com​ mon" is usu​ ally used when there is a need for shared re​ sources be​ tween ten​ ants. choose Security Management > Security Management. 3 In the Work pane. see the doc​ um ​ent at the fol​ low​ ing URL: http://​ www. 2 In the Navigation pane. or ten​ ants.

then the LDAP provider must be con​ fig​ ured first. such as Read/Write for admin or tenant admin. The IP ad​ dress or host name of the LDAP server is needed. Re​ mote au​ then​ ti​ ca​ tion is cov​ ered in da​ men​ tals Guide. 6 41 In the Create Local User dialog box. Remote Authentication Cre​ at​ ing re​ mote user ac​ counts in the ACI is sim​ il​ ar to most other data cen​ ter sys​ tems. Select the roles to be given to this user. choose Security Management. perform the following actions: a. b. The next op​ tion is to con​ fig​ ure groups from which it is al​ lowed to read data and grab it for the pur​ poses of se​ lect​ ing re​ mote users granted ac​ cess to the ACI net​ work. along with the port it uses to com​ mu​ ni​ cate as well as any other rel​ e​ vant in​ for​ ma​ tion that will allow con​ nec​ tion to that server. c. If cre​ at​ ing an LDAP ac​ count. and click Next. de​ tail in the Cisco ACI Fun​ . 5 In the Work pane. Specify login information for the user and 7 Click Finish.Management 4 In the Navigation pane. Specify any security information that is necessary and click Next. choose Actions > Create Local User. The same is true for RA​ DIUS and TACACS+.

.

An​ other Ex​ port pol​ icy re​ quired on oc​ ca​ sion is for gath​ er​ ing tech​ ni​ cal sup​ port in​ for​ ma​ tion. By de​ fault. Since all APIC poli​ cies and con​ fig​ ur​ a​ tion data can be ex​ ported to cre​ ate back​ ups and tech sup​ port files for Dis​ as​ ter Re​ cov​ ery. Configuration Export (Backup) . Cisco rec​ om​ mends that a cur​ rent Backup be per​ formed be​ fore mak​ ing any major sys​ tem con​ fig​ ur​ a​ tion changes or ap​ ply​ ing soft​ ware up​ dates. This can be done from any ac​ tive and fully fit APIC within the ACI fab​ ric. and of​ fline analy​ sis. and so the bun​ dles will not be cov​ ered in de​ tail bleshoot​ here. see the Cisco APIC Trou​ ing Guide that is ref​ er​ enced in the Ap​ pen​ dix of this book. Tech​ ni​ cal sup​ port bun​ dles are con​ fig​ ured very sim​ i-​ lar to con​ fig​ ur​ a​ tion ex​ port poli​ cies. The backup and re​ store process does not re​ quire backup of in​ di​ vid​ ual com​ po​ nents. As with all things APIC. but the ad​ min​ is​ tra​ tor can op​ tion​ ally spec​ ify only a spe​ cific sub​ tree of the man​ age​ ment in​ for​ ma​ tion tree.Management 43 Import and Export Policies All of the stake​ hold​ ers at ACME in​ volved in the de​ ploy​ ment and ad​ min​ is​ tra​ tion of the new mo​ bile ap​ pli​ ca​ tion plat​ form need to know that they will be able to eas​ ily re​ cover from any loss of con​ fig​ ur​ a​ tion quickly and eas​ ily. Back​ ups are con​ fig​ urable through an ex​ port pol​ icy that al​ lows ei​ ther sched​ uled or im​ me​ di​ ate back​ ups to a re​ mote server (pre​ ferred) or. This is use​ ful in multi-ten​ ant de​ ploy​ ments where in​ di​ vid​ ual ten​ ants wish to backup or re​ store their re​ spec​ tive poli​ cies. If is​ sues are en​ coun​ tered within the sys​ tem. Tech​ ni​ cal sup​ port ex​ port poli​ cies can be con​ fig​ ured to run on-de​ mand or sched​ uled for re​ cur​ ring pur​ poses. a pol​ icy needs to be con​ fig​ ured to ex​ port or backup the con​ fig​ ur​ a​ tion poli​ cies. For de​ tails on tech​ ni​ cal sup​ port ex​ port poli​ cies. Con​ fig​ ur​ a​ tion Im​ port poli​ cies (pol​ icy re​ store) will be dis​ cussed later in this sec​ tion. Backup/ex​ port poli​ cies can be con​ fig​ ured to be run on-de​ mand or based on a re​ cur​ ring sched​ ule. in the case where an ex​ ter​ nal SCP/FTP server is not avail​ able. all poli​ cies and ten​ ants are backed up. you might need to con​ tact the Cisco Tech​ ni​ cal As​ sis​ tance Cen​ ter (TAC). trou​ bleshoot​ ing. ACI is a good choice on this front as well. Pro​ vid​ ing a tech​ ni​ cal sup​ port bun​ dle will give Cisco sup​ port en​ gi​ neers the in​ for​ ma​ tion needed to help iden​ tify and sug​ gest re​ me​ di​ at​ ion to is​ sues. back​ ups to be writ​ ten to the local APIC file sys​ tem.

Configuration Export Policy Add a Remote Location (SCP) 1 On the menu bar. a re​ mote lo​ ca​ tion is cre​ ated (the ex​ ter​ nal server to which you want to ex​ port in​ for​ ma​ tion). choose Actions > Create Remote Location 4 In the Create Remote Location dialog box. . Enter a Remote location name b. then an Ex​ port pol​ icy task. choose Remote Locations. Choose a Management EPG Note: default (Out-of-Band) 5 Click Submit. choose Admin > Import/Export. The pro​ ce​ dure below de​ tails how to cre​ ate a re​ mote lo​ ca​ tion and ex​ port pol​ icy in ACI to trans​ fer the con​ fig​ u​ ra​ tion file bun​ dle to an SCP server. and op​ tion​ ally a Sched​ uler (for re​ cur​ ring tasks). Choose a Protocol d. Enter a Remote port f. Enter a Username g. more than a backup job is re​ quired. perform the following actions: a.​ gz file that is cre​ ated. Enter a Password h. Enter a Hostname/IP address c. 3 In the Work pane. The in​ di​ vid​ ual XML files can be ex​ tracted and viewed after the con​ fig​ u​ ra​ tion bun​ dle is trans​ ferred to the desk​ top. An ex​ trac​ tion util​ ity is needed to de​ com​ press the tar. Enter a Remote path e. First. 2 In the Navigation pane.Management 44 Configuration Export (Backup) Since ACI is pol​ icy dri​ ven.

choose Admin > Import/Export. 2 In the Navigation pane. The State should change from "pending" to "success" when the export completes correctly. and spec​ if​ y​ ing a spe​ cific Dis​ tin​ guished Name (DN) if you want to backup only a sub​ set of the Man​ age​ ment In​ for​ ma​ tion Tree (MIT). choose Export Policies > Configuration. . Start Now = Yes d. (Optional) Confirm on the SCP server that the backup filename exists. Name = Export_Policy_Name b.Management 45 Create a One Time Export Policy The pro​ ce​ dure below de​ tails a con​ fig​ ur​ a​ tion ex​ port pol​ icy. Export Destination = Choose_the_Remote_location_created_above 5 Click Submit. choose the Operational tab. 3 In the Work pane. Two op​ tional con​ fig​ ur​ a​ tions are ap​ ply​ ing a Sched​ uler pol​ icy if you want to setup a re​ cur​ ring op​ er​ at​ ion. Format = XML c. choose Admin > Import/Export. a. b. choose Export Policies > Configuration > Export_Name. 2 In the Navigation pane. Verify Export Policy was Successful 1 On the menu bar. perform the following actions: a. but the pro​ ce​ dure for a tech​ ni​ cal sup​ port ex​ port pol​ icy is very sim​ il​ ar. 3 In the Work pane. choose Actions > Create Configuration Export Policy 4 In the Create Configuration Export Policy dialog box. 1 On the menu bar.

Replace . Enter a name and the filename for the import policy.gz".Management 46 Extract and View Configuration Files A con​ fig​ ur​ a​ tion ex​ port task ex​ ports a com​ pressed bun​ dle of in​ di​ vid​ ual XML files. 2 In the Navigation pane.gz). Choose the import type: Merge . b. 3 In the Work pane. perform the following actions: a. Do not include the file path. . such as "backupfile. highlight the file. and then select Extract. Configuration Import (Restore/Merge) To re​ store a con​ fig​ ur​ a​ tion from a pre​ vi​ ous backup: If the re​ mote lo​ ca​ tion does not exist. 4 In the Create Import Configuration Policy dialog box.Will replace all configurations with those only from the backup file. choose Actions > Create Import Configuration Policy. choose Admin > Import/Export. Will not overrite any existing policies. 2 Navigate to the folder where the config export is located (tar.tar. 1 On the menu bar. choose Import Policies > Configuration. These XML con​ fig​ ur​ a​ tion files can be ex​ tracted and viewed on an​ other work​ sta​ tion. 4 Examine the various XML configuration files for parameters that have been configured. select the archive utility of choice. 3 Double-click one of the XML files to view the contents in a browser. cre​ ate a re​ mote lo​ ca​ tion per the "Adding a Re​ mote Lo​ ca​ tion (SCP)" sec​ tion. 1 From the workstation where the exported bundle has been copied.Will merge backup configuration with existing running configurations.

The Replace Import Mode can be either: best-effort . . The import mode must be specified when you attempt to perform a Merge import.Management 47 c. The configuration data is imported per shard with each shard holding a certain part of the system configuration objects. Choose Start Now. d. A shard's entire configuration must be valid to be imported in this mode. The default is best-effort.Attempts to import each shard. skipping those which contain an ​ invalid configuration. skipping any invalid objects atomic . Choose the Remote Location that was previously created in the "Adding a Remote Location (SCP)" section. e.Each shard is imported.

.

49 Upgrading and Downgrading Firmware .

.

Upgrading and Downgrading Firmware Section Content • Firmware Management Firmware Versions Firmware Components Firmware Policies Firmware Groups Maintenance Groups Controller Firmware Catalogue Firmware • Upgrading and Downgrading Considerations • Upgrading the Fabric Downloading the Firmware Images Upgrading the APIC Controller Software Upgrading the Switch Software Using the GUI Upgrading the APIC Controller Software Using the CLI Upgrading the Switch Software Using the CLI Verifying Cluster Convergence Troubleshooting Failures During the Upgrade Process 51 .

.

. This changes when there are fixes for product defects in the software. should be on the same . They have se​ lected a tar​ get ver​ sion of soft​ ware for their de​ ploy​ ment.1(1d) Both the soft​ ware for the APIC and the fab​ ric nodes are de​ noted by the same ver​ sion​ ing scheme.Represents bug fixes to a feature release of APIC. and spine switches. For ex​ am​ ple.Represents a minor release with new software features. or features content. the sup​ port for the hard​ ware plat​ forms they have se​ lected. • mntnc . Ex​ am​ ple: APIC version: 1.Represents major changes in the product architecture. in part​ ner​ ship with Cisco. • minor . they have put a proac​ tive plan in place to re​ visit this de​ ci​ sion pe​ ri​ od​ ic ​ally to de​ ter​ mine if fu​ ture up​ grades are re​ quired.0(2j) cor​ re​ sponds to switch soft​ ware 11. Re​ lease notes for the APIC ver​ sions ref​ er​ ence the cor​ re​ spond​ ing switch ver​ sions and vice-versa. Ad​ di​ tion​ ally. and the ma​ tu​ rity of the soft​ ware re​ leases. has eval​ ua ​ted the re​ quire​ ments for their de​ ploy​ ment based on the soft​ ware fea​ tures re​ quired. Firmware Versions The soft​ ware ver​ sions for ACI are listed in the fol​ low​ ing for​ mat: major.0(2j). but no additional new features.Upgrading and Downgrading Firmware 53 Firmware Management ACME Inc. All com​ po​ nents of the ACI in​ fra​ struc​ ture in​ clud​ ing the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). platform. leaf switches.minor(mntnc) • major . APIC re​ lease 1.

such as 1. you can as​ sume that a main​ te​ nance ver​ sion up​ grade. For most de​ ploy​ ments.Upgrading and Downgrading Firmware 54 ver​ sion. a sin​ gle firmware group is ad​ e​ quate. dis​ parate ver​ sions may exist be​ tween APIC and the switches. Con​ trol over which switches up​ grade to the new ver​ sion can be de​ ter​ mined through main​ te​ nance groups.(1d) => 1. Firmware Components There are three main com​ po​ nents that can be up​ graded: • Switches (leaf and spine) • Application Policy Infrastructure Controller (APIC) • Catalog firmware Firmware Policies Firmware Groups Firmware Group poli​ cies on the APIC de​ fine which group of nodes on which firmware will be up​ graded.1. Controller Firmware The APIC con​ troller firmware pol​ icy al​ ways ap​ plies to all con​ trollers in the clus​ ter. While at the time of up​ grad​ ing. will have less im​ pact than a major/minor ver​ sion up​ grade. mak​ ing it pos​ si​ ble to defer an up​ grade task to a busi​ ness main​ te​ nance win​ dow. as there will be only bug fixes and no new fea​ tures added. The APIC GUI pro​ vides real-time sta​ tus in​ for​ - .1(1d). Maintenance Groups Main​ te​ nance Group poli​ cies de​ fine a group of switches that will be jointly up​ graded to the as​ so​ ci​ ated firmware set. do not op​ er​ ate the fab​ ric for ex​ tended pe​ ri​ ods of time in this state. but the up​ grade is al​ ways done se​ quen​ tially. When con​ sid​ er​ ing the im​ pact and risk of up​ grad​ ing. Main​ te​ nance groups can be up​ graded on de​ mand or ac​ cord​ ing to a sched​ ule.

The APIC. which per​ forms image man​ age​ ment. Con​ troller Firmware poli​ cies can be up​ graded on de​ mand or also ac​ cord​ ing to a sched​ ule. Catalogue Firmware Each firmware image in​ cludes a com​ pat​ i​ bil​ ity cat​ a​ log that iden​ ti​ fies sup​ ported switch mod​ els. has an image repos​ i​ tory for com​ pat​ i​ bil​ ity cat​ a​ logs. and switch im​ ages. and mod​ els that are al​ lowed to use that firmware image. The APIC main​ tains a cat​ a​ log of the firmware im​ ages. APIC con​ troller firmware im​ ages. Firmware upgrade policy relationships . switch types.Upgrading and Downgrading Firmware 55 ma​ tion about firmware up​ grades.

.

and allocate enough time to troubleshoot or perform a rollback. you should set up at least two separate maintenance groups. A user must have the fabric administrator role to perform firmware upgrade tasks. the controllers should be upgraded first. Confirm that the /firmware partition is not filled beyond 75%.Upgrading and Downgrading Firmware 57 Upgrading and Downgrading Considerations Be​ fore start​ ing the up​ grade or down​ grade process. Although it is possible to upgrade the fabric without impacting the dataplane. see the Import and Export Policies chapter. you might be required to remove some unused firmware files from the repository to accommodate the compressed image as well as provide adequate space to extract the image. If the partition is filled beyond 75%. In the navigation pane. • Verify free space.github. To help minimize the impact to hosts during an upgrade. The APIC automatically extracts the image. For information about exporting configurations. A common separation is by odd and even node IDs. Verify that the health state of all of the controllers in the cluster are Fully Fit before you proceed. Before starting any upgrade. • Maintenance windows. there should not be any . choose Controllers > apic_controller_name > Cluster. choose System > Controllers. Typically.io/acitroubleshooting-book/ • Configuration backup. • Permissions. you should perform an upgrade during a scheduled maintenance window according to your change control policy. choose System > Controllers. followed by the switch nodes. con​ sider the fol​ low​ ing things: • Before starting the upgrade process. your controllers should be in good health. Always refer to the relevant release notes of the destination firmware version for any changes to this order. On the menu bar. To resolve issues for controllers that are not fully fit see the Troubleshooting Cisco Application Centric Infrastructure document:https://datacenter. always export your configuration to an external source. In the navigation pane. • Maintenance groups. This window should account for any unforseen issues that might arise during the upgrade. • Upgrade order. Assuming that your hosts are dualconnected to at least one odd and one even leaf node. On the menu bar. choose Controllers > apic_controller_name > Storage.

Maintenance group creation is covered in detail later in the chapter. Device packages are not always tied to the APIC software.html . software is not specifically tied to the APIC or switch software version.com/web/techdoc/aci/acimatrix/matrix. • Upgrading a fabric with the Application Virtual Switch (AVS) deployed. Another consideration is that your leaf VPC pairs should contain one odd and one even node.cisco. You can confirm the device compatability for Layer 4 to Layer 7 devices using the online Application Centric Infrastructure (ACI) Compatability tool: http://www. The AVS • Device packages.Upgrading and Downgrading Firmware 58 impact to your hosts.

8 Repeat the steps for the switch image.50/aci-apic-dk9. 9 Once the download reaches 100%.0. perform the following actions: a. b. choose Firmware Repository.1.1. the list of all switches in the fabric and the status of when the firmware was last upgraded are displayed.1.iso • SCP Example: 192. choose Download Tasks.0.50:/tmp/aci-firmware/aci-apicdk9. d.1. In the URL field. Note: In the Work pane. such as "apic_1. choose Fabric Node Firmware. 6 In the Navigation pane. enter the URL from where the image must be downloaded.168. enter your username and password. 4 In the Work pane.Upgrading and Downgrading Firmware 59 Upgrading the Fabric Downloading the Firmware Images You must down​ load both the con​ troller soft​ ware pack​ age and switch soft​ ware pack​ age for the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) from Cisco. c.3d". choose Admin > Firmware. in the Navigation pane. .3d. 2 In the Navigation pane. enter a name for the switch image. choose Actions > Create Outside Firmware Source. 3 In the Navigation pane. Click Submit. To down​ load the firmware im​ ages using the APIC GUI: 1 On the menu bar. 7 In the Work pane. 5 In the Create Outside Firmware Source dialog box. For the Protocol radio buttons. In the Source Name field. • HTTP Example: http://192.3d.iso For SCP.168.1.​ com. choose the Operational tab to view the download status of the images. choose Download Tasks. click the Secure copy or HTTP radio button.

2 Pull the software using SCP: admin@apic1:~> scp username@IP_address_with_the_image:/absolute_path_to_image_plus_image_filename &#10! 3 Place the image into the image repository: admin@apic1:~> firmware add ver_no. To up​ grade the APIC con​ troller soft​ ware using the GUI: 1 On the menu bar.Upgrading and Downgrading Firmware 60 10 In the Work pane. the downloaded version numbers and image sizes are displayed. You do not need to up​ grade the cat​ al​ og firmware image sep​ ar​ ately.iso is added to the repos​ it​ ory. click Controller Firmware. 4 Verify the software has been added to the respository: admin@ifc1:~> firmware list Name : aci-apic-dk9. To down​ load the firmware im​ ages using the APIC CLI: 1 SSH to an APIC and log in as "admin". choose Actions > Upgrade Controller Firmware Policy. perform the following actions: . 4 In the Upgrade Controller Firmware Policy dialog box.1(1d) Upgrading the APIC Controller Software The cat​ al​ og firmware image is up​ graded when an APIC con​ troller image is up​ graded. 2 In the Navigation pane.1.iso &#10! The firmware image ver_no.bin Type : controller Version : 1. choose Admin > Firmware.1. 3 In the Work pane.1d.

the APICs must have com​ pleted up​ grad​ ing and have a health state of Fully Fit. In the Apply Policy field. In the Target Firmware Version field. Click Submit to complete the task.Upgrading and Downgrading Firmware 61 a. choose the image version to which you want to upgrade. a Wait​ ing for Clus​ ter Con​ ver​ gence mes​ sage is dis​ played in the Sta​ tus col​ umn for each APIC as it up​ grades. b. 5 Verify the status of the upgrade in the Work pane by clicking Controller Firmware in the Navigation pane. Once the con​ troller re​ boots. choose Admin > Firmware. the browser dis​ plays an error mes​ sage. Alternately. If the clus​ ter does not im​ me​ di​ ately con​ verge. To up​ grade the switch soft​ ware using the GUI: 1 On the menu bar. Each APIC con​ troller takes about 10 min​ utes to up​ grade. and is not fully fit. from the drop-down list. The APIC con​ trollers are up​ graded se​ ri​ ally so that the con​ troller clus​ ter is avail​ able dur​ ing the up​ grade. enter the URL for the APIC controller that has already been upgraded. it drops from the clus​ ter. 6 In the browser URL field. The con​ trollers up​ grade in ran​ dom order. and sign in to the APIC controller as prompted. c. and it re​ boots with the newer ver​ sion while the other APIC con​ trollers in the clus​ ter are still op​ er​ at​ ional. Dur​ ing this pe​ riod. . When the APIC con​ troller that the browser is con​ nected to is up​ graded and it re​ boots. the up​ grade will wait until the clus​ ter con​ verges and is fully fit. Upgrading the Switch Software Using the GUI Be​ fore you up​ grade the switches. Then the clus​ ter con​ verges. and the next con​ troller image starts to up​ grade. click the Apply now radio button. Once a con​ troller image is up​ graded. it joins the clus​ ter again. The Sta​ tus di​ al​ og box dis​ plays the Changes Saved Suc​ cess​ fully mes​ sage. and the up​ grade process be​ gins. you can apply a schedule policy if you wish to defer the task to a specific date/time.

110". In the Group Name field. enter a comma-separated list or a range of node IDs to include in the group. In the Create Firmware Group dialog box. 3 If you have not created a firmware group. In the Work pane. 4 If you have not created maintenance groups. in the Navigation pane. create one group with the even-numbered devices and the other group with the odd-numbered devices. 108. In the Group Node IDs field. b. perform the following actions: i. c. enter the name of the maintenance group. In the Scheduler drop-down list. enter the name of the firmware group. perform the following actions: i. 106. choose Action > Create Maintenance Group. v. "102. The Work pane displays details about the firmware policy that was created earlier. In the Target Firmware Version drop-down list. choose the Policy tab. b. 104. choose Fabric Node Firmware > Firmware Groups > new_firmware_group. perform the following substeps: a. the switches that are operating in the fabric are displayed. In the Group Node IDs field. ii. In the Create Maintenance Group dialog box. "Even-Nodes". you can choose to create a schedule for upgrading or leave the drop-down list blank so that you can upgrade on demand. "101. Note: In the Work pane. For example.Upgrading and Downgrading Firmware 62 2 In the Navigation pane. iv. For example. enter a comma-separated list or a range of node IDs to include in the group. In the Navigation pane. c. For example. iii. Choose Actions > Create Firmware Group. iii. 103-105. In the Work pane. which is the default mode. For example. choose the firmware version to which you will upgrade. iv. Click Submit. perform the following substeps: a. Note: Cisco recommends that you create two maintenance groups for all of the switches. . click Maintenance Groups. choose Fabric Node Firmware. d. ii. Click Submit. In the Group Name field. To verify that the firmware group was created. click the Pause Upon Upgrade Failure radio button. For the Run Mode radio buttons. 108".

Repeat this step for the second maintenance group. The switch upgrade takes up to 12 minutes for each group. vii. the upgrade process will upgrade only one switch at a time out of the two switches in a VPC domain. view all of the switches that are listed. 6 In the Upgrade Now dialog box. You do not need to up​ grade the cat​ al​ og firmware image sep​ ar​ ately. To verify that the maintenance group was created. you will see all the switches listed under the controller node. . up to 20 switches at a time will get upgraded. The switches will reboot when they upgrade. the Status displays that all the switches in the group are being upgraded simultaneously. 8 In the Navigation pane. Note: The Work pane displays details about the maintenance policy. 5 Right-click one of the maintenance groups that you created and choose Upgrade Now. view the upgrade image details listed against each switch. The default concurrency in a group is set at 20. In the Current Firmware column. and click the name of the maintenance group that you created. and the controllers in the cluster will not communicate for some time with the switches in the group. Once the switches rejoin the cluster after rebooting. Upgrading the APIC Controller Software Using the CLI The cat​ al​ og firmware image is up​ graded when an APIC con​ troller image is up​ graded. the scheduler pauses and manual intervention is required by the APIC administrator. Cisco rec​ om​ mends that you to per​ form the firmware up​ grade from the GUI. click Fabric Node Firmware.Upgrading and Downgrading Firmware 63 vi. 7 Click OK. In case of any failures. for Do you want to upgrade the maintenance group policy now?. and then the next set of 20 switches are upgraded. the APIC per​ forms ad​ di​ tional ver​ if​ i​ ca​ tion and in​ tegrity checks on the soft​ ware image. Therefore. Note: In the Work pane. in the Navigation pane. If there are any VPC configurations in the cluster. Verify that the switches in the fabric are upgraded to the new image. choose Fabric Node Firmware > Maintenance Groups > new_maintenance_group. connectivity drops. a group named "Odd-Nodes". When you use the GUI. Note: In the Work pane. click Yes. For example.

"in​ progress". Upgrading the Switch Software Using the CLI .0(1. that con​ troller has prob​ a​ bly up​ graded and is re​ boot​ ing.0(1.0(1.202a) inqueue -----------------0 2 controller 1.0(1. Ex​ am​ ple: admin@apic1:~> firmware upgrade status Node-Id Role Current- Target- Upgrade- Progress-Percent Firmware Firmware Status (if inprogress) --------- ----------- ------------ -----------------. The up​ grade oc​ curs in the back​ ground.Upgrading and Downgrading Firmware 64 To up​ grade the APIC con​ troller soft​ ware using the CLI: 1 List the current software in the respository that was previously downloaded. 3 Check the status of the upgrade.1.202a) inqueue 0 3 controller 1. Ex​ am​ ple: admin@apic1:~> firmware upgrade controllers ver_no. or "com​ ple​ teok".bin The APIC con​ trollers are up​ graded se​ ri​ ally so that the con​ troller clus​ ter is avail​ able dur​ ing the up​ grade.0(1. When the APIC con​ troller to which you have con​ nected com​ pletes up​ grad​ ing and re​ boots.---------- 1 controller 1.200) apic-1.1d.200) apic-1.1.0(1.1(1d) 2 Upgrade the firmware on the controllers.bin Type : controller Version : 1.200) apic-1. Ex​ am​ ple: admin@apic1:~> firmware list Name : aci-apic-dk9. If you see "un​ known" in this field. you can close the SSH win​ dow.202a) inprogress 0 The Up​ grade-Sta​ tus field will show "in​ queue".

1d.iso" to ". The output that appears from the following command will appear like the following sample: Ex​ am​ ple: admin@apic1:~> firmware upgrade status node node_id Node-Id Role Current- Target- Upgrade- Progress-Percent Firmware Firmware Status (if inprogress) --------- ----------- ------------------- -----------------. with the correct version number: Ex​ am​ ple: admin@apic1:~> firmware list Name : aci-n9000-dk9. To up​ grade the switch soft​ ware using the CLI 1 Check that the output of the following command appears like the output shown below. Ex​ am​ ple: admin@apic1:~> firmware upgrade switch node 101 ver_no.bin Firmware Installation on Switch Scheduled You must up​ grade each switch sep​ ar​ ately.Upgrading and Downgrading Firmware 65 Upgrading the Switch Software Using the CLI Be​ fore you up​ grade the switches. the APICs must have com​ pleted up​ grad​ ing and have a health state of Fully Fit. 3 Check the upgrade status for the switch.---------- 1017 leaf n9000-11.1(1d) The name changes from ".bin Type : switch Version : 11. 2 Upgrade the switches.869S1) n9000-11.1(1d) completeok -----------------100 .11.bin".1.0(1.

when an up​ grade fail​ ure is de​ tected. As the con​ troller and switches move through the up​ grade. as well as how many have up​ graded suc​ cess​ fully. by en​ ter​ ing the firmware up​ grade sta​ tus com​ mand. • Queued: There is a currently active window (schedule) and the node is • Inprogress: Upgrade is currently in progress on this node. • Scheduled: Upgrade is scheduled for this node. • CompleteNOK: Upgrade failed on this node. This may take a while. . These mes​ sages are dis​ played in the Sta​ tus field. and no more nodes in that group begin to up​ grade.Upgrading and Downgrading Firmware 66 You can check the sta​ tus of all nodes at once. When all the clus​ ters have con​ verged suc​ cess​ fully. you will see "No" in the Wait​ ing for Clus​ ter Con​ ver​ gence field of the Con​ troller Firmware screen. The sched​ uler ex​ pects man​ ual in​ ter​ ven​ tion to debug any up​ grade fail​ ures. Verifying Cluster Convergence You can mon​ it​ or the progress of the clus​ ter con​ ver​ gence after a sched​ uled main​ te​ nance. you must re​ sume the paused sched​ uler. Troubleshooting Failures During the Upgrade Process There is one sched​ uler per main​ te​ nance pol​ icy. before declaring failure). the sched​ uler pauses. • CompleteOK: Upgrade completed successfully. you will see mes​ sages about the num​ ber of nodes queued and the num​ ber in the process of up​ grad​ ing. which pre​ sents you with a se​ ries of mes​ sages dur​ ing the process of one clus​ ter con​ verg​ ing and then the next clus​ ter. Once man​ ual in​ ter​ ven​ tion is com​ plete. You view the Con​ troller Firmware screen on the GUI. By de​ fault. The fol​ low​ ing are the pos​ si​ ble up​ grade states for a node: • NotScheduled: No upgrade is currently scheduled for this node. • Inretryqueue: Node is queued again for upgrade retry (5 attempts are made requesting permission to upgrade. 4 Repeat Steps 2 and 3 for each additional switch.

For ad​ di​ tional trou​ bleshoot​ ing pro​ ce​ dures. • Is the switch maintenance group paused? The group will be paused if any switch fails its upgrade. switches which have not already started their upgrade will be in “queued” state. Until the controller cluster is healthy. you should work through the steps for ver​ if​ y​ ing a pause in the sched​ uler. If the sys​ tem takes longer than about 60 min​ utes for a switch to dis​ play “wait​ ing​ For​ Clus​ ter​ Health = no” in the API or "Wait​ ing for Clus​ ter Con​ ver​ gence" show​ ing "No" in the GUI. that means the controller cluster is not healthy. see the Trou​ bleshoot​ ing Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture doc​ um ​ent at the fol​ low​ ing URL: https://​ datacenter.Upgrading and Downgrading Firmware 67 If you no​ tice that switches are in the “queued” state.​ io/​ aci-troubleshooting-book/​ . then check the fol​ low​ ing: • Is the controller cluster healthy? The controller cluster must be healthy.​ github. If you see “waitingForClusterHealth = yes” in the API or "Waiting for Cluster Convergence" showing "Yes" in the GUI.

.

69 Fabric Connectivity .

.

Access Policies Domains VLAN Pools AEPs Policy Types Interface Policies Switch Policies Interface Policy Groups Switch Policy Groups Interface Profiles Switch Profiles Best Practices • Adding New Devices to the Fabric Sample Configuration Creating VLAN Pools Create VLAN Pool Create Physical Domain Create an Attachable Access Entity Profile (AEP) Interface Policies Interface Policy Groups Interface Profile​ ​ Switch Profiles Reusability Sample vPC Creation Create VLAN Pool ​ Create a Physical Domain 71 .Fabric Connectivity Section Content • Understanding Fabric Policies • Fabric .

Fabric Connectivity 72 Create Access Entity Profile Interface Policies Switch Profile Create vPC domain Validate Operation of Configured vPC • Server Connectivity Cisco UCS B-Series Servers Standalone Rack Mount Servers or Non-Cisco Servers • Virtual Machine Networking Understanding VM Networking in ACI ACI VM Integration Workflow VMware Integration VMM Policy Model Interaction Publishing EPGS to a VMM Domain Connecting VMs to the EPG Port Groups on vCenter Verifying Virtual Endpoint Learning Verifying VM Endpoint Learning on the APIC from the CLI VMware Integration Use Case • Deploying the Application Virtual Switch Prerequisites Getting Started Install the AVS VIB Manual Installation DHCP Relay Attachable Access Entity Profile (AEP) and AVS VMM Domains for vCenter AVS Switching Modes Create the VMM Domain for AVS Verify AVS Deployment on vCenter Add vSphere Hosts to the AVS .

Fabric Connectivity Verify AVS on ESX VXLAN Load Balancing IGMP Snooping Policy for AVS • External Connectivity ​Extending ACI to External Layer 2 Extending an ACI Bridge Domain Outside of the Fabric Extending Endpoint Groups Outside the ACI Fabric Extending ACI to External Layer 3 Supported Routing Protocols Configure MP-BGP Spine Route Reflectors Layer 3 Integration Through Tenant Network with OSPF NSSA External Layer 3 for Multiple Tenants • Application Migration Use Case Extending the Network to ACI 73 .

.

and stor​ age ar​ rays. It is im​ por​ tant that other teams such as server teams un​ der​ stand these con​ cepts as well. the in​ ter​ face poli​ cies folder is where port be​ hav​ ior is con​ fig​ ured. con​ tracts are con​ sid​ ered the “what/when/why”. fab​ ric poli​ cies. con​ fig​ ur​ ing do​ mains. as they will be main​ tain​ ing these poli​ cies for the pur​ poses of in​ ter​ nal con​ nec​ tions be​ tween fab​ ric leaf nodes. The fab​ ric tab in the APIC GUI is used to con​ fig​ ure sys​ tem-level fea​ tures in​ clud​ ing. it is time to start cre​ at​ ing con​ nec​ tiv​ ity poli​ cies within the ACI fab​ ric. par​ tic​ ul​ arly in the case of their build processes for adding ad​ di​ tional ca​ pac​ ity. Do​ mains and AEPs are also cre​ ated in the ac​ cess poli​ cies view. AEPs can be con​ sid​ ered the "where" and do​ mains can be thought . and switch and port be​ hav​ ior. The ac​ cess poli​ cies sub​ sec​ tion is split into fold​ ers sep​ ar​ at​ ing out dif​ fer​ ent types of poli​ cies and ob​ jects that af​ fect fab​ ric be​ hav​ ior. it is also key to un​ der​ stand the im​ pli​ ca​ tions of delet​ ing poli​ cies on the ACI fab​ ric. like port speed. as they will be in​ ter​ act​ ing with them. demon​ strate how to add and pre-pro​ vi​ sion switches. but not lim​ ited to. de​ vice dis​ cov​ ery and in​ ven​ tory man​ age​ ment. and basic man​ age​ ment poli​ cies. Fabric . and ac​ cess poli​ cies. The fab​ ric pane is split into three sec​ tions: in​ ven​ tory. con​ nec​ tions to ex​ ter​ nal en​ ti​ ties such as servers. and walk through the steps and ob​ jects re​ quired when new de​ vices are added to the fab​ ric to ef​ fec​ tively op​ er​ ate an ACI fab​ ric. For ex​ am​ ple.Fabric Connectivity 75 Understanding Fabric Policies Now that ACME has been pro​ vi​ sioned with ACI fab​ ric and in​ fra​ struc​ ture space has been con​ fig​ ured be​ tween the leaf and spine switches. While many poli​ cies are reusable.Access Policies Domains EPGs are con​ sid​ ered the “who” in ACI. di​ ag​ nos​ tic tools. Un​ der​ stand​ ing how fab​ ric and ac​ cess poli​ cies con​ fig​ ure the fab​ ric is key for the ACME net​ work teams. The fab​ ric ac​ cess poli​ cies pro​ vide the fab​ ric with the base con​ fig​ ur​ a​ tion of the ac​ cess ports on the leaf switches. This chap​ ter will re​ view the key ob​ jects in the ac​ cess poli​ cies sub​ sec​ tion of the fab​ ric tab – many of which are reusable. ac​ cess priv​ il​ eges. or whether or not to run pro​ to​ cols like LACP on leaf switch in​ ter​ faces is set. net​ work​ ing equip​ ment.

and VMM do​ mains. For ex​ am​ ple. One or more do​ mains are added to an AEP. ex​ ter​ nal routed do​ mains. Do​ mains act as the glue be​ tween the con​ fig​ ur​ a​ tion done in the fab​ ric tab to the pol​ icy model and EPG con​ fig​ ur​ a​ tion found in the ten​ ant pane.76 Fabric Connectivity of as the “how” of the fab​ ric. Phys​ ic ​al do​ mains are gen​ er​ ally used for bare metal servers or servers where hy​ per​ vi​ sor in​ te​ gra​ tion is not an op​ tion. There are four dif​ fer​ ent do​ main types: phys​ ic ​al do​ mains. AEPs are tied to in​ ter​ face pol​ icy groups. VLANs are in​ stan​ ti​ ated on leaf switches based on AEP con​ fig​ ur​ a​ tion. Ex​ ter​ nal routed do​ mains are used for Layer 3 con​ nec​ tions. check out the fol​ low​ ing video ti​ tled "How De​ vices Con​ nect to the Fab​ ric: Un​ der​ stand​ ing Cisco ACI Do​ mains": https://​ www. and the ten​ ant ad​ min​ is​ tra​ tors as​ so​ ci​ ate do​ mains to EPGs. For ex​ am​ ple. For​ ward​ ing de​ ci​ sions are still based on con​ tracts and the pol​ icy model.​ youtube. AEPs are con​ fig​ ured under global poli​ cies. The fab​ ric op​ er​ at​ or cre​ ates the do​ mains. an ex​ ter​ nal bridged do​ main could be used to con​ nect an ex​ ist​ ing switch trunked-up to a leaf switch. Dif​ fer​ ent do​ main types are cre​ ated de​ pend​ ing on how a de​ vice is con​ nected to the leaf switch. not sub​ nets and VLANs. . Ex​ ter​ nal bridged do​ mains are used for Layer 2 con​ nec​ tions. ex​ ter​ nal bridged do​ mains. and are used to group do​ mains with sim​ il​ ar re​ quire​ ments. VXLAN and mul​ ti​ cast ad​ dress pools are also con​ fig​ urable.​ com/​ watch?​ v=_​ iQvoC9zQ_​ A VLAN Pools VLAN pools con​ tain the VLANs used by the EPGs the do​ main will be tied to. For an in-depth white​ board ex​ pla​ na​ tion on do​ mains. the fab​ ric knows where the var​ io ​us de​ vices in the do​ main live and the APIC can push the VLANs and pol​ icy where it needs to be. an ex​ ter​ nal routed do​ main could be used to con​ nect a WAN router to the leaf switch. AEPs At​ tach​ able Ac​ cess En​ tity Pro​ files (AEPs) can be con​ sid​ ered the "where" of the fab​ ric con​ fig​ ur​ a​ tion. By group​ ing do​ mains into AEPs and as​ so​ ci​ at​ ing them. A do​ main is as​ so​ ci​ ated to a sin​ gle VLAN pool.

and pro​ files. In​ ter​ face pro​ files con​ tain blocks of ports . there may be a pol​ icy that dic​ tates 10GE. There are three types of in​ ter​ face pol​ icy groups de​ pend​ ing on link type: in​ di​ vid​ ual. con​ fig​ ur​ ing vPC do​ mains. Interface Policies First. there should ide​ ally be a set of pol​ icy groups cre​ ated once and reused as new de​ vices are con​ nected to the fab​ ric. pol​ icy groups. and a 1GE link level pol​ icy must be cre​ ated for de​ vices con​ nected at that speed. and vPC. poli​ cies should be cre​ ated once and reused when con​ nect​ ing new de​ vices to the fab​ ric. there should be a pol​ icy that dic​ tates CDP is dis​ abled. Max​ im ​iz​ ing reusabil​ ity of pol​ icy and ob​ jects makes day-to-day op​ er​ at​ ions ex​ po​ nen​ tially faster and eas​ ier to make large-scale changes. Pol​ icy groups do not ac​ tu​ ally spec​ ify where the pro​ to​ cols and port be​ hav​ ior should be im​ ple​ mented. Switch Policies There are also poli​ cies for switches . For ex​ am​ ple. cov​ ered in the fol​ low​ ing para​ graphs. Ide​ ally. Port Chan​ nel.and are also tied to the in​ ter​ face pol​ icy groups de​ scribed in the . Interface Profiles In​ ter​ face pro​ files help tie the pieces to​ gether. are are later tied to in​ ter​ face pol​ icy groups. which are called ex​ plicit vPC pro​ tec​ tion groups in the APIC GUI.Fabric Connectivity 77 Policy Types Most of the poli​ cies fold​ ers have sub​ fold​ ers. under the in​ ter​ face poli​ cies folder there are fold​ ers for con​ fig​ ur​ a​ tion called poli​ cies. The "where" hap​ pens by as​ so​ ci​ at​ ing one or more in​ ter​ face pro​ files to a switch pro​ file.these can be reused as new de​ vices are con​ nected to the leaf switches. CDPen​ abled. Note that the ports on the leaf switches de​ fault to 10GE. Switch Policy Groups Switch pol​ icy groups allow lever​ ag​ ing of ex​ ist​ ing switch po​ lices like Span​ ning Tree and mon​ it​ or​ ing poli​ cies. In​ ter​ face pol​ icy groups use the poli​ cies de​ scribed in the pre​ vi​ ous para​ graph to spec​ ify how links should be​ have. For ex​ am​ ple. These are also reusable ob​ jects as many de​ vices are likely to be con​ nected to ports that will re​ quire the same port con​ fig​ ur​ a​ tion. Interface Policy Groups In​ ter​ face pol​ icy groups are tem​ plates to dic​ tate port be​ hav​ ior and are as​ so​ ci​ ated to an AEP. Note the in​ ter​ face pol​ icy groups sim​ ply dic​ tate pol​ icy. For ex​ am​ ple. and a pol​ icy that dic​ tates CDP is en​ abled .in​ ter​ face se​ lec​ tors .for ex​ am​ ple. in​ ter​ face poli​ cies are cre​ ated to dic​ tate in​ ter​ face be​ hav​ ior. Just like poli​ cies.

switch.78 Fabric Connectivity pre​ vi​ ous para​ graphs. and in​ ter​ face poli​ cies: Relationships to allow a physical interface or interfaces to be attached to an EPG Layer 2 In​ ter​ face Pol​ icy In Cisco ACI ver​ sion 1. Again. This as​ so​ ci​ a​ tion pushes the con​ fig​ u​ ra​ tion to the in​ ter​ face. . this is just an ar​ bi​ trary port. such as e1/1. a new con​ fig​ urable In​ ter​ face Pol​ icy was added to allow a per port-VLAN sig​ nif​ i​ cance. To con​ nect de​ vices to the ACI fab​ ric we can use un​ tagged traf​ fic. switch pro​ files allow the se​ lec​ tion of one or more leaf switches and as​ so​ ci​ ate in​ ter​ face pro​ files to con​ fig​ ure the ports on that spe​ cific node. The fol​ low​ ing fig​ ure high​ lights the re​ la​ tion​ ship be​ tween the var​ i​ ous global. the pro​ file must be as​ so​ ci​ ated to a spe​ cific switch pro​ file (dis​ cussed in the next para​ graph) to con​ fig​ ure the ports. Switch Profiles Lastly. and cre​ ates a Port Chan​ nel or vPC if one has been con​ fig​ ured in the in​ ter​ face pol​ icy.VLAN en​ cap​ su​ la​ tion or VXLAN en​ cap​ su​ la​ tion.1.

and 10GE port speed. For example. Two EPGs belonging to a single Bridge Domain cannot share the same encapsulation ID on a given leaf switch. it can be hard to remember what all the defaults are. but might help sim​ plify day-to-day op​ er​ at​ ion of the ACI fab​ ric. There are many "default" policies out of the box. global con​ fig​ ur​ a​ tion as​ sumes that ten​ ants do not share leaf switches and there​ fore there is no VLAN over​ lap​ ping within the same leaf. traffic will get affected. • Policies Reuse policies whenever possible. How​ ever. use names that clearly describe the setting. Best Practices Cisco has es​ tab​ lished sev​ eral best prac​ tices for fab​ ric con​ fig​ ur​ a​ tion. 1GE port speed. which is why policies should be clearly named to avoid making a mistake when adding new devices to the fabric. For example. This al​ lows ten​ ants to re-use VLAN en​ cap​ su​ la​ tion IDs thor​ ough the fab​ ric with​ out al​ low​ ing com​ mu​ ni​ ca​ tion be​ tween ten​ ants. there should be policies for LACP active/passive/off. These are not re​ quire​ ments and might not work for all en​ vi​ ron​ ments or ap​ pli​ ca​ tions. For example. In ACI. a policy that enables LACP in mode active could be called "LACPActive". a port and VLAN pair (P. That is. When naming policies.V) is registered internally instead of just a VLAN encapsulation ID. However. there can be a set of interface policy groups for all VMware ESXi servers connected via 10GE vPCs. This increases the consumption of hardware resources at a per switch level. and a different set of interface policy groups for all bare metal servers running . EPGs can use the same VLAN en​ cap​ su​ la​ tion as long as EPGs are bound to sep​ ar​ ate switches. It is expected that the port will flap when the Layer 2 interface policy changes between global and local. with the de​ fault con​ fig​ ur​ a​ tion (global). A set of interface policy groups should be created for each type of similar devices connected. • Per Port-VLAN limitations and considerations When per port-VLAN is used.Fabric Connectivity 79 In tra​ di​ tional net​ work​ ing one of the lim​ it​ a​ tions re​ lated to VLAN en​ cap​ su​ la​ tion is scal​ a-​ bil​ ity and re-us​ abil​ ity due to the limit of 4096 VLANs in net​ work​ ing de​ vices.

such as overlapping VLAN pools.Fabric Connectivity 80 1GE with CDP disabled. If a VMM domain needs to be leveraged across multiple tenants. • Domains Build one physical domain per tenant for bare metal servers or servers without hypervisor integration requiring similar treatment. • AEPs Multiple domains can be associated to a single AEP for simplicity's sake. or to limit the scope of the presence of VLANs across the fabric. create a switch profile for each vPC pair (if using vPC). and additionally. There are some cases where multiple AEPs may need to be configured to enable the infrastructure VLAN. Build one physical domain per tenant for external connectivity. Create a switch profile for each leaf switch individually. . each AEP will have its own set of interface policy groups. a single VMM domain can be created and associated with all leaf ports where VMware ESXi servers are connected. Since interface policy groups are tied to a single AEP.

these var​ i​ ous pro​ files are linked to​ gether and have de​ pen​ den​ cies. . This sec​ tion will walk through set​ ting up pro​ files from scratch.Fabric Connectivity 81 Adding New Devices to the Fabric This sec​ tion will demon​ strate how to con​ fig​ ure ACI to re-use the fab​ ric ac​ cess poli​ cies. As out​ lined in the pre​ vi​ ous sec​ tion. The fol​ low​ ing di​ a​ gram re​ it​ er​ ates the ob​ ject re​ la​ tion​ ships: Object relationships Whereas a tra​ di​ tional com​ mand line in​ ter​ face on a switch gen​ er​ ally re​ quires a port-byport confugu​ ra​ tion. ACI al​ lows de​ f​ i​ n​ i​ tion of ob​ jects and poli​ cies that can be re-used. sim​ pli​ fy​ ing day-to-day op​ er​ a​ tion of the fab​ ric. The re-us​ abil​ ity of these poli​ cies makes it pos​ si​ ble to repli​ cate the con​ fig​ u​ ra​ tion of a switch very eas​ ily. with a focus on how to re-use these pro​ files across the fab​ ric. The fol​ low​ ing di​ a​ gram de​ picts how this re-us​ abil​ ity sim​ pli​ fies the op​ er​ a​ tion of the fab​ ric over time.

the con​ fig​ u​ ra​ tion of a cou​ ple of switches does not re​ quire many processes or au​ toma​ tion.82 Fabric Connectivity Policy Re-use In any data cen​ ter. the op​ er​ a​ tor is faced with the cost of de​ sign​ ing processes to man​ age these de​ vices. . or spe​ cial​ ized ap​ pli​ ca​ tions. Lever​ ag​ ing the Cisco ACI pol​ icy model. This is what is de​ picted as the pol​ icy reuse in​ flec​ tion point in the pre​ vi​ ous di​ a​ gram. and will in​ clude a re​ view of the ob​ jects as they are con​ fig​ ured. and vPC-con​ nected de​ vices from scratch. The fol​ low​ ing steps rep​ re​ sent the use case of adding a new bare metal server con​ nected to a leaf switch. These are the steps to be taken in the APIC GUI when new de​ vices are con​ nected to the leaf switches to en​ sure the ac​ cess ports on the leaf switches have the right switch​ port con​ fig​ u​ ra​ tion. These can be net​ work man​ age​ ment tools. As the data cen​ ter size in​ creases. au​ toma​ tion be​ comes more and more crit​ i​ cal as it has a di​ rect im​ pact on the cost of busi​ ness op​ er​ a​ tions. scripts. Port Chan​ nel-con​ nected de​ vices. and the ver​ i​ fi​ ca​ tion steps to en​ sure proper con​ fig​ u​ ra​ tion. an op​ er​ a​ tor can lever​ age pro​ files to stream​ line the op​ er​ a​ tion of adding de​ vices and man​ ag​ ing those de​ vices. In tra​ di​ tional net​ works. Sample Configuration The fol​ low​ ing sec​ tions will walk through sam​ ple con​ fig​ u​ ra​ tion of set​ ting up in​ di​ vid​ u​ ally con​ nected de​ vices. when changes that im​ pact a large set of de​ vices need to be made.

This al​ lows for sev​ eral key ad​ van​ tages from a server de​ ploy​ ment per​ spec​ tive: • You can create resilient Layer 2 topologies based on link aggregation • You do not need STP • You have increased bandwidth. With​ out vPC's.Fabric Connectivity 83 Be​ fore get​ ting into the con​ fig​ u​ ra​ tion of vPC's. we pro​ vide a log​ i​ cal topol​ ogy that greatly sim​ pli​ fies server pro​ vi​ sion​ ing and man​ age​ ment. At a high level. or a spe​ cial con​ fig​ u​ ra​ tion on the NIC dri​ ver or the ker​ nel that al​ lows it to in​ tel​ li​ gently loadbal​ ance traf​ fic using an al​ go​ rithm. vPC Topology In the fig​ ure above. it is im​ por​ tant to un​ der​ stand what vPC's are and how they are dif​ fer​ ent from tra​ di​ tional meth​ ods of server con​ nec​ tiv​ ity. By con​ fig​ ur​ ing ports on two dif​ fer​ ent switches as the same port-chan​ nel and using an in​ ter-switch mes​ sag​ ing chan​ nel (such as the in​ ter-switch port-chan​ nel in the green box on the left hand side) to cover re​ dun​ dancy sce​ nar​ ios. as all links are actively forwarding • Your server configurations are simplified since the configurations simply appears as port-channels without the need for special software. a sin​ gle server is dual homed to two dif​ fer​ ent switches for re​ dun​ dancy. which are a pop​ u​ lar server con​ nec​ tiv​ ity method​ ol​ ogy. the ben​ e​ fits they pro​ vide and how vPC's in the ACI fab​ ric dif​ fer from how they are de​ ployed on Cisco Nexus switches run​ ning NX-OS soft​ ware. This sec​ tion of the chap​ ter at​ tempts to clar​ ify at a high level what vPC's are. vPC ex​ tends link ag​ gre​ ga​ tion to two sep​ a​ rate phys​ i​ cal switches. from a driver or kernel-tuning standpoint . the server will likely use an ac​ tive-standby con​ fig​ u​ ra​ tion.

to pro​ vide sim​ i​ lar ben​ e​ fits. such as Cisco UCS fab​ ricin​ ter​ con​ nects.84 Fabric Connectivity vPCs can also be used to con​ nect other down​ stream de​ vices. Legacy connectivity compared to vPC The com​ po​ nents of a tra​ di​ tional vPC do​ main are il​ lus​ trated below: Traditional vPC topology . The fig​ ure below shows a sin​ gle tra​ di​ tional Layer 2 switch con​ nected to a VPC en​ abled Cisco switch pair.

The fab​ ric it​ self serves as the peer-link. . there is no re​ quire​ ment for set​ ting up vPC peer-links. typ​ i​ cally on the out-of-band man​ age​ ment port. vPC topology in ACI The key dif​ fer​ ences to note here are that rel​ a​ tive to tra​ di​ tional vPC de​ sign. vPC con​ fig​ u​ ra​ tions need to be done man​ u​ ally by the op​ er​ a​ tor and re​ quire a pair of ded​ i​ cated "in​ ter-switch" links also called a peer-link. There are also no keepalives being sent on the man​ age​ ment ports.Fabric Connectivity 85 As il​ lus​ trated above. in Cisco switch​ ing prod​ ucts run​ ning NX-OS soft​ ware. that is used to de​ ter​ mine peer live​ li​ ness to de​ tect a vPC peer-switch fail​ ure. The rich in​ ter​ con​ nec​ tiv​ ity be​ tween fab​ ric nodes makes it un​ likely that peers will have an ac​ tive path be​ tween them. The ACI fab​ ric greatly sim​ pli​ fies VPC con​ fig​ u​ ra​ tions. Mak​ ing con​ fig​ u​ ra​ tion changes in such sce​ nar​ ios with​ out the con​ fig-sync fea​ ture en​ abled may lead to sce​ nar​ ios where there are mis​ matched vPC pa​ ra​ me​ ters be​ tween the vPC pri​ mary and the vPC sec​ ondary switches that may cause par​ tial con​ nec​ tiv​ ity loss dur​ ing the change it​ self if a type-1 in​ con​ sis​ tency is de​ tected. Note that at​ tempt​ ing to cable a leaf switch to an​ other leaf switch will lead to a "wiring mis​ match" fault in the GUI and re​ sult in a black​ listed port that will have to be man​ u​ ally re​ cov​ ered. There is also a peer-keepalive link.

peers assume master-slave roles. it is assumed to have crashed. static vs lacp LACP: Lag ID The fol​ low​ ing di​ ag ​rams il​ lus​ trate how the ACI fab​ ric for​ wards traf​ fic from a vPC do​ main to a non-vPC con​ nected host in the fab​ ric. Hence if the peer switch becomes unreachable.Fabric Connectivity 86 The fol​ low​ ing are some other key be​ hav​ ioral changes to vPC as it ap​ plies to the ACI fab​ ric rel​ at​ ive to clas​ sic vPC that are im​ por​ tant for op​ er​ at​ ors to un​ der​ stand: • Configurations are automatically synchronized to avoid an error-free configuration by the APIC which is the central point of control for all configurations in the ACI fabric. A list of type-1 parameters used for consistency checking for a given vPC domain specific to the ACI fabric are listed below. • Role is used in case of vpc type-1 consistency failure. Slave switch brings down all its vPC ports. the slave switch brings down all its vPC links if the MCT goes down. • In the ACI fabric. • Role election still happens. it is very unlikely that all the redundant paths between vPC peers fail at the same time. • In traditional vPC solution. • Global type-1 parameters: STP • Interface type-1 parameters: STP: Only BPDU Guard is configurable EthPM Port speed Duplex mode Port mode MTU Native VLAN PCM: Channel mode. . and vice-versa. The slave switch does not bring down vPC links.

VLAN pools de​ fine a range of VLAN IDs that will be used by the EPGs. 2 S3 does a table lookup and routes with vPC Virtual IP (VIP). 4 S3 delivers the pkt to locally attached host H2. H1 -> H2 1 H1 sends a pkt towards H2 on one of its PC link (S1 in this case). .Fabric Connectivity 87 vPC forwarding Uni​ cast packet flow H2 -> H1 1 H2 sends a pkt towards H1 on its link to S3. con​ fig​ ur​ ing newly-con​ nected bare metal servers first re​ quires cre​ ation of a phys​ i​ cal do​ main and then as​ so​ ci​ a​ tion of the do​ main to a VLAN pool. As men​ tioned in the pre​ vi​ ous sec​ tion. 4 S2 delivers the pkt to locally attached host H1. 3 Spine switch sees multiple routes for VIP and picks one of them (S2 in this case). 2 S1 does a table lookup and routes with IP of S3. 3 Spine switch routes to S3. Creating VLAN Pools In this ex​ am​ ple.

using iden​ ti​ fiers like VLANs.1Q or VXLAN en​ cap​ su​ la​ tion. b. There are two allocation modes: dynamic or static. vir​ tual port IDs. The range of VLANs used in the con​ fig​ u​ ra​ tion ex​ am​ ple is 100-199. As de​ picted in the fol​ low​ ing fig​ ure. Each server will be tag​ ging using 802. and NVGRE tags. phys​ i​ cal port IDs. In ACI. Encapsulation normalization Create VLAN Pool 1 On the menu bar.Fabric Connectivity 88 The servers are con​ nected to two dif​ fer​ ent leaf nodes in the fab​ ric. perform the following actions: a. choose Fabric > Access Policies. the APIC selects VLANs from the pool dynamically. Optionally. VXLAN. 802. VXLAN VNIDs. ACI has to know be​ fore​ hand how to clas​ sify pack​ ets into the dif​ fer​ ent EPGs. When a packet ar​ rives ingress to a leaf switch in the fab​ ric. c. The leaf switches nor​ mal​ ize the traf​ fic by strip​ ping off tags and reap​ ply​ ing the re​ quired tags on fab​ ric egress.1Q VLAN tags. it is im​ por​ tant to un​ der​ stand that the de​ f​ i​ n​ i​ tion of VLANs as they per​ tain to the leaf switch ports is uti​ lized only for iden​ ti​ fi​ ca​ tion pur​ poses. NVGRE. This is common in VMM integration mode where the actual . choose Pools > VLAN. 4 In the Create VLAN Pool dialog box. Define a meaningful name for the VLAN pool. provide a description for the VLAN pool. 3 In the Work pane. choose Actions > Create VLAN Pool. Note: When dynamic allocation is selected. 2 In the Navigation pane. the ACI fab​ ric can also act as a gate​ way be​ tween dis​ parate en​ cap​ su​ la​ tion types such as un​ tagged traf​ fic.

Define a meaningful name for the profile. Select the VLAN pool you just created.365-08:00" rn="rtinfraVlanNs-[uni/l2dom-JC-L2-Domain]" status="" tCl="l2extDomP" tDn="uni/l2domJC-L2-Domain"/> <fvnsRtVlanNs childAction="" lcOwn="local" modTs="2015-02-23T16:13:22. choose Actions > Create Physical Domain. d. perform the following actions: a. b. Static allocation is typically used when the pool will be referenced from a static source like a static path binding for an EPG for use with bare metal servers.538-08:00" name="" rn="from-[vlan-100]-to-[vlan-199]" status="" to="vlan199" uid="8131"/> </fvnsVlanInstP> Create Physical Domain A phys​ ic ​al do​ main acts as the link be​ tween the VLAN pool and the Ac​ cess En​ tity Pro​ file (AEP).538-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-vlan-pool" ownerKey="" ownerTag="" status="" uid="8131"> <fvnsRtVlanNs childAction="" lcOwn="local" modTs="2015-02-25T11:35:33. only the pro​ file name and the VLAN pool are con​ fig​ ured. The encap blocks are used to define the range of VLANs in the VLAN pool. 2 In the Navigation pane. The cre​ ation of the AEP and its as​ so​ ci​ at​ ion will be cov​ ered later in this sec​ tion. When con​ fig​ ur​ ing in this order. 3 4 In the Work pane.Fabric Connectivity 89 VLAN ID is not important. choose Physical and External Domains > Physical Domains. Remember it's the EPG itself which policies are applied to. . In the Create Physical Domain dialog box. XML Ob​ ject <fvnsVlanInstP allocMode="static" childAction="" configIssues="" descr="" dn="uni/infra/vlanns-[bsprint-vlan-pool]-static" lcOwn="local" modTs="2015-02- 23T15:58:33. 1 On the menu bar.007-08:00" rn="rtinfraVlanNs-[uni/phys-bsprint-PHY]" status="" tCl="physDomP" tDn="uni/physbsprint-PHY"/> <fvnsEncapBlk childAction="" descr="" from="vlan-100" lcOwn="local" modTs="2015-02- 23T15:58:33. Note multiple ranges can be added to a single pool. The do​ main also ties the fab​ ric con​ fig​ ur​ a​ tion to the ten​ ant con​ fig​ ur​ a​ tion. while the do​ mains are cre​ ated under the fab​ ric tab. choose Fabric > Access Policies. as the ten​ ant ad​ min​ is​ tra​ tor is the one who as​ so​ ci​ ates do​ mains to EPGs.

Define the a meaningful name for the profile. 1 On the menu bar. 3 In the Work pane. 6 Click Submit.945-08:00" rn="rtdomP-[uni/infra/attentp-bsprint-AEP]" status="" tCl="infraAttEntityP" tDn="uni/infra/attentp-bsprint-AEP"/> </physDomP> Create an Attachable Access Entity Profile (AEP) The AEP links the phys​ ic ​al do​ main and its VLAN Pool to the in​ ter​ face poli​ cies. enter a description. c. perform the following actions: a. 2 In the Navigation pane. Select the physical domain that was previously configured. choose Actions > Create Attached Entity Profile. b.065-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsvlanNs" state="formed" stateQual="none" status="" tCl="fvnsVlanInstP" tDn="uni/infra/vlanns[bsprint-vlan-pool]-static" tType="mo" uid="8131"/> <infraRsVlanNsDef childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:22. choose Fabric > Access Policies.906-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-PHY" ownerKey="" ownerTag="" status="" uid="8131"> <infraRsVlanNs childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:22. choose Global Policies > Attached Acess Entity Profle. The con​ fig​ ur​ a​ tion for an AEP is straight​ for​ ward. 5 Click Next. . Optionally. Click + to associate the domain to the AEP. d. 4 In the Create Attached Entity Profile dialog box.Fabric Connectivity 90 XML Ob​ ject <physDomP childAction="" configIssues="" dn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:21.065-08:00" rType="mo" rn="rsvlanNsDef" state="formed" stateQual="none" status="" tCl="fvnsAInstP" tDn="uni/infra/vlanns-[bsprint-vlan-pool]-static" tType="mo"/> <infraRtDomP childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.

980-08:00" rn="rtattEntP-[uni/infra/funcprof/accportgrp-bsprint-AccessPort]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-bsprint-AccessPort"/> <infraRsDomP childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 25T11:35:33.961-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsdomP[uni/phys-bsprint-PHY]" state="formed" stateQual="none" status="" tCl="physDomP" tDn="uni/phys-bsprint-PHY" tType="mo" uid="8131"/> </infraAttEntityP> Create Interface Policies Next.874-08:00" monPolDn="uni/fabric/monfab-default" rn="nscont" status=""> <infraRsToEncapInstDef childAction="" deplSt="" forceResolve="no" lcOwn="local" modTs="2015-02-23T16:13:52. de​ fine the in​ ter​ face pro​ files and show​ case the re-us​ abil​ ity of the fab​ ric poli​ - .874-08:00" rn="dompcont" status=""> <infraAssocDomP childAction="" dompDn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:52.874-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-AEP" ownerKey="" ownerTag="" status="" uid="8131"> <infraContDomP childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.570-08:00" monPolDn="uni/fabric/monfab-default" profileDn="" rn="source[uni/l2dom-JC-L2-Domain]" status=""/> <fabricCreatedBy childAction="" creatorDn="uni/phys-bsprint-PHY" deplSt="" domainDn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:52.570-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsdomP- [uni/l2dom-JC-L2-Domain]" state="formed" stateQual="none" status="" tCl="l2extDomP" tDn="uni/l2dom-JC-L2-Domain" tType="mo" uid="8754"/> <infraRsDomP childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:52.961-08:00" monPolDn="uni/fabric/monfab-default" profileDn="" rn="source-[uni/phys-bsprint-PHY]" status=""/> </infraRsToEncapInstDef> </infraContNS> <infraRtAttEntP childAction="" lcOwn="local" modTs="2015-02-24T11:59:37.961-08:00" rn="assocdomp-[uni/phys-bsprint-PHY]" status=""/> <infraAssocDomP childAction="" dompDn="uni/l2dom-JC-L2-Domain" lcOwn="local" modTs="2015-02-25T11:35:33.961-08:00" monPolDn="uni/fabric/monfabdefault" rType="mo" rn="rstoEncapInstDef-[allocencap-[uni/infra]/encapnsdef- [uni/infra/vlanns-[bsprint-vlan-pool]-static]]" state="formed" stateQual="none" status="" tCl="stpEncapInstDef" tDn="allocencap-[uni/infra]/encapnsdef[uni/infra/vlanns-[bsprint-vlan-pool]-static]" tType="mo"> <fabricCreatedBy childAction="" creatorDn="uni/l2dom-JC-L2-Domain" deplSt="" domainDn="uni/l2dom-JC-L2-Domain" lcOwn="local" modTs="2015-02- 25T11:35:33.Fabric Connectivity 91 XML Ob​ ject <infraAttEntityP childAction="" configIssues="" descr="" dn="uni/infra/attentp- bsprint-AEP" lcOwn="local" modTs="2015-02-23T16:13:52.570-08:00" rn="assocdomp-[uni/l2dom-JC-L2-Domain]" status=""/> </infraContDomP> <infraContNS childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.

Create Link Level Policies Link level poli​ cies dic​ tate con​ fig​ ur​ a​ tion like the speed of ports. Optionally. 2 In the Navigation pane. provide a description for the policy. choose Actions > Create CDP Interface Policy. 4 In the Create Link Level Policy dialog box. choose Interface Policies > Policies > CDP Interface. but ide​ ally there are al​ ready poli​ cies in place that can sim​ ply be se​ lected. Leaf switch ports default to 10GE. choose Fabric > Access Policies. 5 Click Submit. Define a meaningful name for the policy such as 'CDP-Enable'.Fabric Connectivity 92 cies. 3 In the Work pane. Define the meaningful name for the policy. choose Fabric > Access Policies. Select the interface speed. 2 In the Navigation pane. XML Ob​ ject <fabricHIfPol autoNeg="on" childAction="" descr="" dn="uni/infra/hintfpol-1G-Auto" lcOwn="local" linkDebounce="100" modTs="2015-01-14T06:47:15. b.331- 08:00" rn="rtinfraHIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example"/> </fabricHIfPol> Create a CDP Interface Policy 1 On the menu bar. d. Change the de-bounce interval if required. choose Actions > Create Link Level Policy. . c. 1 On the menu bar. Select the auto negotiation mode for the interface.693-08:00" name="1G-Auto" ownerKey="" ownerTag="" speed="1G" status="" uid="15374"> <fabricRtHIfPol childAction="" lcOwn="local" modTs="2015-01-14T06:48:48. In​ ter​ face poli​ cies can be re-used as needed by dif​ fer​ ent in​ ter​ face pro​ file de​ f​ in ​i​ t​ ion re​ quire​ ments. This sec​ tion will il​ lus​ trate cre​ ation of new pro​ files. e.081- 08:00" rn="rtinfraHIfPol-[uni/infra/funcprof/accportgrp-UCS-1G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-1G-PG"/> <fabricRtHIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. 4 In the Create CDP Interface Policy dialog box. choose Interface Policies > Policies > Link Level. 3 In the Work pane. perform the following actions: a. perform the following actions: a.

provide a description for the policy. provide a description for the policy.Fabric Connectivity 93 b. choose Fabric > Access Policies. choose Interface Policies > Policies > LLDP Interface. c.081- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-UCS-1G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-1G-PG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-24T11:59:37.470-08:00" name="CDP-Enable" ownerKey="" ownerTag="" status="" uid="15374"> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-01-14T07:23:54. choose Actions > Create LLDP Interface Policy. Choose either admin state enabled or disabled.154- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" status="" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-01-14T06:48:48. XML Ob​ ject <cdpIfPol adminSt="enabled" childAction="" descr="" dn="uni/infra/cdpIfP-CDP-Enable" lcOwn="local" modTs="2015-01-14T06:47:25. 2 In the Navigation pane. perform the following actions: a. 3 In the Work pane.331- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example"/> </cdpIfPol> Create an LLDP Interface Policy 1 On the menu bar. b. ​ . c. 5 Click Submit.980- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-bsprint-AccessPort]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-bsprintAccessPort"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. Define a meaningful name for the policy. Optionally. 4 In the Create LLDP Interface Policy dialog box. d. 5 Click Submit. Choose the transmit state. Choose the receive state. Optionally.957- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-UCS-10G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-10G-PG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11.

4 In the Create LACP Policy dialog box.Fabric Connectivity 94 XML Object descr="" <lldpIfPol adminRxSt="enabled" adminTxSt="enabled" childAction="" dn="uni/infra/lldpIfP-LLDP-Enable" lcOwn="local" modTs="2015-02-11T07:40:35. 1 On the menu bar. c. d. provide a description for the policy.154- 08:00" rn="rtinfraLldpIfPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG" status="" /> <lldpRtLldpIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. 5 Click Submit. Define a meaningful name for the policy.331- 08:00" rn="rtinfraLldpIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example" </lldpIfPol> l​ ows Create an LACP Interface Policy ​ faces. Optionally. choose Interface Policies > Policies > LACP 3 In the Work pane.664-08:00" status="" name="LLDP-Enable" ownerKey="" ownerTag="" status="" uid="15374"> /> <lldpRtLldpIfPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. . specify the minimum and maximum number of links in the Port Channel.3ad) that al r​ a​ tion re​ quire​ ments an op​ tional ne​ go​ ti​ at​ ion pro​ to​ col to be run on Port Chan​ nel in​ ter pre-emp​ tively de​ fined to be used as needed for the var​ io ​us con​ fig​ u in the data cen​ ter. . choose Fabric > Access Policies. 2 In the Navigation pane. These can be Link Ag​ gre​ ga​ tion Con​ trol Pro​ to​ col is part of an IEEE spec​ if​ i​ ca​ tion (802. Note if LACP is enabled on the leaf switch. choose Actions > Create LACP Policy. Optionally. Select the LACP mode required for the server. perform the following actions: a. b. LACP must also be enabled on the server or other connected device.

provide a description for the policy. Define a meaningful name for the policy.547-08:00" mode="active" name="LACP-Active" ownerKey="" ownerTag="" status="" uid="8131"> <lacpRtLacpPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. choose Actions > Create Spanning Tree Interface Policy.susp-individual" descr="" dn="uni/infra/lacplagp-LACP-Active" lcOwn="local" maxLinks="16" minLinks="1" modTs="2015-02-24T11:58:36. d.154- 08:00" rn="rtinfraLacpPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" status="" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG"/> </lacpLagPol> Create an LACP Member Profile (optional) Op​ tion​ ally. Note: ACI does not run Span​ ning Tree on the fab​ ric be​ tween the leaves and spines. 1 On the menu bar. If required. 4 In the Create LACP Member Policy dialog box. change the priority. 3 In the Work pane. 1 On the menu bar. 5 Click Submit. c. 2 In the Navigation pane. The Span​ ning Tree in​ ter​ face pol​ icy sim​ ply de​ fines the port be​ hav​ ior. choose Fabric > Access Policies.Fabric Connectivity 95 XML Ob​ ject <lacpLagPol childAction="" ctrl="fast-sel-hot-stdby. If required. Optionally. perform the following actions: a. change the transmit rate. b. . 3 In the Work pane.graceful-conv. the LACP mem​ ber pro​ file pro​ vides the abil​ ity to pro​ vide pri​ or​ ity spec​ if​ i​ ca​ tions to mem​ bers of an LACP group. choose Interface Policies > Policies > LACP Member. It is a com​ mon best prac​ tice to en​ able BPDU guard on in​ ter​ faces con​ nected to servers. choose Actions > Create LACP Member Policy. Create a Spanning Tree Interface Policy (optional) The Span​ ning Tree pol​ icy dic​ tates the be​ hav​ ior of south​ bound leaf port Span​ ning Tree fea​ tures. choose Fabric > Access Policies. 2 In the Navigation pane. choose Interface Policies > Policies > Spanning Tree Interface.

Fabric Connectivity

96

4

In the Create Spanning Tree Interface Policy dialog box, perform the following
actions:
a. Define a meaningful name for the policy.
b. Optionally, provide a description for the policy.
c. Enable BPDU filter and/or BPDU guard.

5

Click Submit.

Create a Storm Control Policy (optional)

A traf​
fic storm oc​
curs when pack​
ets flood the LAN, cre​
at​
ing ex​
ces​
sive traf​
fic and de​
grad​
ing net​
work per​
for​
mance. The traf​
fic storm con​
trol fea​
ture can be used to pre​
vent dis​
rup​
tions on ports by a broad​
cast, mul​
ti​
cast, or uni​
cast traf​
fic storm on phys​
i​
cal in​
ter​
faces.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policies > Storm Control.

3

In the Work pane, choose Actions > Create Storm Control Policy.

4

In the Create Storm Control Policy dialog box, perform the following actions:
a. Define a meaningful name for the policy.
b. Optionally, provide a description for the policy.
c. Specify how the control policy is to be applied, either through percentage of
the total bandwidth or as a packet per second definition that matches the
requirement for the data center

5

Click Submit.

Creating a Layer 2 Interface Policy to enable per port-VLAN

1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policies > L2 Interface.

3

In the Work pane, choose Actions > Create L2 Interface Policy.

4

In the Create L2 Interface Policy dialog box, perform the following actions:
a. Give the L2 Interface name and an optional description.
b. Select VLAN scope to Port Local scope to enable per port-VLAN.

Create Interface Policy Groups
The in​
ter​
face pol​
icy groups com​
prise the in​
ter​
face poli​
cies as a func​
tional group that is
as​
so​
ci​
ated to an in​
ter​
face. The fol​
low​
ing di​
ag
​ram shows how pre​
vi​
ously cre​
ated items
are grouped under the pol​
icy group.

Fabric Connectivity

97

Policies contained in a policy group
Once all the in​
ter​
face poli​
cies have been de​
fined, the in​
di​
vid​
ual poli​
cies can be brought
to​
gether to form a pol​
icy group that will be linked to the in​
ter​
face pro​
file. The pol​
icy
group is de​
fined from a mas​
ter de​
f​
i​
n​
i​
tion that en​
com​
passes being one of the fol​
low​
ing:

Access Policy Group

Port Channel Policy Group

VPC Policy Group

Create Access Port Policy Group

The ac​
cess port pol​
icy is de​
fined for an in​
di​
vid​
ual link (non-Port Chan​
nel or vPC).
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create Access Policy Group.

4

In the Create Access Policy Group dialog box, perform the following actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Use the profiles created previously that are relevant for this policy group.

5

Click Submit.

Fabric Connectivity

98

Create Port Channel Interface Policy Group

Port Chan​
nel​
ing also load-bal​
ances traf​
fic across the phys​
ic
​al in​
ter​
faces that are mem​
bers of the chan​
nel group. For every group of in​
ter​
faces that needs to be con​
fig​
ured
into a port chan​
nel, a dif​
fer​
ent pol​
icy group has to be cre​
ated. This pol​
icy group de​
fines
the be​
hav​
iour. For ex​
am​
ple, if ports 1/1-4 are to be con​
fig​
ured into one port chan​
nel, and ports 1/5-8 into a sep​
ar​
ate port chan​
nel, each of those groups would re​
quire
the cre​
ation of a sep​
ar​
ate pol​
icy group.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create PC Interface Policy Group.

4

In the Create PC Interface Policy Group dialog box, perform the following
actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Select the policies created previously that are relevant for this PC policy
group.

5

Click Submit.

Create VPC Interface Policy Group

Note: This ob​
ject must be unique for each VPC cre​
ated.
A vir​
tual PortChan​
nel (vPC) al​
lows links that are phys​
ic
​ally con​
nected to two dif​
fer​
ent
de​
vices to ap​
pear as a sin​
gle Port Chan​
nel to a third de​
vice. In the world of ACI, pairs of
leaf switches may be con​
fig​
ured in a vPC do​
main so that down​
stream de​
vices can be
ac​
tive-ac​
tive dual-homed.
For every group of in​
ter​
faces that are to be con​
fig​
ured into a vPC, a dif​
fer​
ent in​
ter​
face
pol​
icy group needs to be cre​
ated. The vPC pol​
icy group con​
tains both the de​
f​
in
​i​
t​
ion for
the be​
hav​
iour of the port chan​
nel, and the iden​
ti​
fier. For ex​
am​
ple, if ports 1/1-4 are to
be con​
fig​
ured into one vPC across two switches, and ports 1/5-8 into a sep​
ar​
ate vPC
across two switches, each of those groups would re​
quire the de​
f​
in
​i​
t​
ion of a sep​
ar​
ate
pol​
icy group.
Note: For vPC you will also re​
quire a unique vPC do​
main de​
f​
in
​i​
t​
ion be​
tween the two
paired switches. More de​
tails to fol​
low.
1

On the menu bar, choose Fabric > Access Policies.

Fabric Connectivity

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create VPC Interface Policy Group.

4

In the Create VPC Interface Policy Group dialog box, perform the following

99

actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Choose the policies created previously that are relevant for this vPC policy
group.
5

Click Submit.

Interface Profile
The in​
ter​
face pro​
file in ACI links the pol​
icy groups that de​
fine how the in​
ter​
face is
going to be​
have, and as​
signs them to spe​
cific ports via the con​
cept of in​
ter​
face se​
lec​
tor. In turn, the in​
ter​
face pro​
file is even​
tu​
ally tied to a switch pro​
file to spec​
ify which
leaf switch the ref​
er​
enced ports should be con​
fig​
ured. As we con​
tinue the process of
defin​
ing the port pro​
files, you can ob​
serve how we have started at the bot​
tom of this
ob​
ject tree con​
fig​
ur​
ing the dif​
fer​
ent pro​
files. The pur​
poses for all these in​
di​
vid​
ual poli​
cies that tie to​
gether is to max​
i​
mize pol​
icy re-use.

Interface Profile links to Interface Selector and Interface Policy Group
The di​
a​
gram in the pre​
vi​
ous sec​
tion pro​
vides a vi​
sual de​
scrip​
tion of what can be ac​
com​
plished by group​
ing the poli​
cies that have been de​
fined under the in​
ter​
face pro​
file,
and then as​
signed to ports with in​
ter​
face se​
lec​
tors and the ac​
cess port pol​
icy groups.

100 Fabric Connectivity

Create Interface Profile

The in​
ter​
face pro​
file is com​
posed of two pri​
mary ob​
jects. The in​
ter​
face se​
lec​
tor and
the ac​
cess port pol​
icy group. The in​
ter​
face se​
lec​
tor de​
fines what in​
ter​
faces will apply
the ac​
cess port pol​
icy. The ports that share the same at​
trib​
utes can then be grouped
under the same in​
ter​
face pro​
file.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

Next, add the in​
ter​
face se​
lec​
tors that are as​
so​
ci​
ated to this in​
ter​
face pro​
file.
Create Interface Selector

1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

3
4

In the Work pane, choose Actions > Create Access Port Selector.
In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter the interface IDs.
d. Choose the interface policy group that should be associated to these ports.

5

Click Submit.

Create Interface Profile for Port Channel

If a server has two or more up​
links to a leaf switch, the links can be bun​
dled into a Port
Chan​
nel for re​
siliency and load dis​
tri​
bu
​t​
ion. In order to con​
fig​
ure this in ACI, cre​
ate an
in​
ter​
face pol​
icy group of type Port Chan​
nel to bun​
dle the in​
ter​
faces. Dif​
fer​
ent Port
Chan​
nels re​
quire dif​
fer​
ent pol​
icy groups.

Fabric Connectivity 101

Port Channel Policy Group
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

Next, cre​
ate an in​
ter​
face port se​
lec​
tor. Since you will be con​
fig​
ur​
ing a Port Chan​
nel,
the op​
er​
a​
tor will add all of the in​
ter​
faces re​
quired in the Port Chan​
nel in​
ter​
face. In this
ex​
am​
ple in​
ter​
faces Eth​
er​
net 1/1-2 will be con​
fig​
ured in one Port Chan​
nel and in​
ter​
faces Eth​
er​
net 1/3-4 will be con​
fig​
ured in an​
other Port Chan​
nel.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

3

In the Work pane, choose Actions > Create Access Port Selector.

102 Fabric Connectivity

4

In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter interface IDs for the first port channel.
d. Choose the interface policy group.

5

Click Submit.

6

Repeat this process for the second Port Channel (if you have another Port
Channel to add).

Create an Interface Profile for Virtual Port Channel

A vPC do​
main is al​
ways made up of two leaf switches, and a leaf switch can only be a
mem​
ber of one vPC do​
main. In ACI, that means that the de​
f​
i​
n​
i​
tion of the poli​
cies is sig​
nif​
i​
cant be​
tween the two switches. The same pol​
icy can be reused be​
tweeen the two
switches, and through the vPC do​
main the pair as​
so​
ci​
a​
tion can be de​
fined. vPC Switch
do​
main mem​
bers should be taken into con​
sid​
er​
a​
tion when con​
fig​
ur​
ing firmware main​
te​
nance groups. By keep​
ing this in mind, firmware up​
grades should never im​
pact both
vPC switch peers at the same time. More de​
tails on this can be found in the Up​
grad​
ing
and Down​
grad​
ing Firmware sec​
tion.
For this rea​
son, a switch pro​
file that would rep​
re​
sent two sep​
a​
rate switch IDs needs to
be cre​
ated. The re​
la​
tion​
ship of these switches to the two ports in the same pol​
icy
group is de​
fined through the in​
ter​
face pro​
file.

vPC Policy Group

Fabric Connectivity 103

The same process would have to be re​
peated for every grouped in​
ter​
face on each side
that will be a mem​
ber of the vPC.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

6

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

7
8

In the Work pane, choose Actions > Create Access Port Selector.
In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter interface IDs.
d. Select of the interface policy group to be used for the vPC port behavior.

9

Click Submit.

Create a vPC Domain for Virtual Port Channel

When con​
fig​
ur​
ing a vPC, there is one ad​
di​
tional step to be con​
fig​
ured once to put two
leaf switches into the same vPC do​
main.

Creating a vPC Domain
1

On the menu bar, choose Fabric > Access Policies.

104 Fabric Connectivity

2

In the Navigation pane, choose Switch Policies > VPC Domain > Virual Port
Channel default.

3

In the Work pane, choose Actions > Explicit VPC Protection Group.

4

In the Explicit VPC Protection Group dialog box, perform the following actions:
a. Define a meaningful name for the vPC domain.
b. Provide a unique ID to represent the vPC domain.
c. Choose the first switch you want to be part of the vPC domain.
d. Choose the second switch you want to be part of the vPC domain.

5

Click Submit.

Switch Profiles
A switch pro​
file groups all the in​
ter​
face pro​
files that de​
fine the be​
hav​
ior of its re​
spec​
tive switch ports. A switch pro​
file could be the de​
f​
in
​i​
t​
ion of a sin​
gle switch or it could
be the de​
f​
in
​i​
t​
ion of mul​
ti​
ple switches. As a best prac​
tice, there should be a switch pro​
file for each leaf switch, and an ad​
di​
tional switch pro​
file for each vPC do​
main pair of
leaf switches.
The in​
ter​
face pro​
files that you have cre​
ated can be as​
so​
ci​
ated to the switch through a
sin​
gle switch pro​
file or they can be as​
so​
ci​
ated through dif​
fer​
ent switch pro​
files. If you
have var​
io
​us racks that are iden​
ti​
cal in the way the in​
ter​
face ports are con​
fig​
ured, it
could be ben​
e​
fi​
cial to uti​
lize the same switch pro​
file. This would make it pos​
si​
ble to
mod​
ify the con​
fig​
ur​
a​
tion of many switches dur​
ing op​
er​
at​
ions with​
out hav​
ing to con​
fig​
ure each switch in​
di​
vid​
ua
​lly.

Reusability
The ca​
pa​
bil​
ity of pol​
icy reusabil​
ity is cru​
cial to re-em​
pha​
size from an op​
er​
at​
ional per​
spec​
tive. If a pro​
file has been de​
fined to con​
fig​
ure a port as 1GB speed for ex​
am​
ple, that
pro​
file can be reused for many in​
ter​
face pol​
icy groups. When look​
ing at whole switch
con​
fig​
ur​
a​
tions, the re-us​
abil​
ity of the pro​
file can be ex​
tended to sim​
plify data cen​
ter
op​
er​
at​
ions and en​
sure com​
pli​
ance. The fol​
low​
ing fig​
ure il​
lus​
trates the reusabil​
ity of
pro​
files across racks of switches.

If all these racks are con​ fig​ ured in the same fash​ ion (mean​ ing they are wired in the same way) the same poli​ cies could be reused by sim​ ply as​ sign​ ing the switches to the same switch pro​ file. each of the top of rack switches is based on the same switch pro​ file. If a pro​ file has been reused across many de​ vices. The fol​ low​ ing topol​ ogy will be con​ fig​ ured: . make sure to check where it is being used be​ fore you delete the pro​ file or pol​ icy. It is also im​ por​ tant to be aware of the im​ pli​ ca​ tion of delet​ ing pro​ files. It would then in​ herit the pro​ file tree and be con​ fig​ ured the exact same way as the other racks. Sample vPC Creation The fol​ low​ ing pro​ ce​ dure demon​ strates what a vPC bringup looks like and also API POST con​ fig​ u​ ra​ tion as​ s​ es​ ment of the vPC.Fabric Connectivity 105 Policy re-use at scale In the pre​ vi​ ous di​ a​ gram.

choose Actions > Create VLAN Pool. REST :: /api/node/class/fvnsVlanInstP. . choose Fabric > Access Policies. In the Create Physical Domain dialog box. choose Actions > Create Physical Domain. Define a meaningful name for the domain. 3 4 In the Work pane. Click Static Allocation for the allocation mode. perform the following actions: a.xml CLI :: moquery -c fvnsVlanInstP Create a Physical Domain 1 On the menu bar. 4 In the Create VLAN Pool dialog box. provide a description for the pool. b. b. Optionally. 3 In the Work pane. Note: For this example the pool will be from VLAN 100 to VLAN 199. c. 2 In the Navigation pane.106 Fabric Connectivity Sample Topology Create VLAN Pools 1 On the menu bar. perform the following actions: a. choose Fabric > Access Policies. Define a meaningful name for the pool. choose Physical and External Domains > Physical Domains. Choose the VLAN pool that you previously created. 2 In the Navigation pane. choose Pools > VLAN.

2 In the Navigation pane. 3 4 In the Work pane.Fabric Connectivity 107 REST :: /api/node/class/physDomP. choose Interface Policies > Policies > Link Level. 6 Click Submit. d. . b. Choose the physical domain that you previously created. 2 In the Navigation pane. 5 Click Submit. Provide a unique ID to represent the AEP. Define a meaningful name for the pool. perform the following actions: a. perform the following actions: a. c.xml CLI :: moquery -c infraAttEntityP Interface Policies Create Link Level Policy 1 On the menu bar. 5 Click Next. provide a description for the policy. choose Fabric > Access Policies. b. Choose the auto negotiation for the interface. Choose the speed to match the interface requirement. 3 In the Work pane. choose Actions > Create Link Level Policy. choose Fabric > Access Policies. Optionally. Define a meaningful name for the AEP.xml CLI :: moquery -c physDomP Create Access Entity Profile 1 On the menu bar. Click + to add a domain that will be associated to the interfaces. In the Create Attached Entity Profile dialog box. 4 In the Create Link Level Policy dialog box. c. d. choose Actions > Create Attached Entity Profile. choose Global Policies > Attachable Access Entity Profles. REST :: /api/node/class/infraAttEntityP.

choose Actions > Create LACP Policy. Define a meaningful name for the policy group. Choose a Link Level Policy. Choose an AEP to associate the policy group to. Define a meaningful name for the policy. 4 In the Create LACP Policy dialog box. b. choose Fabric > Access Policies. c. d. choose Interface Policies > Policy Groups. perform the following actions: a. 3 In the Work pane. REST :: /api/node/class/lacpLagPol. Optionally. provide a description for the policy group. choose Fabric > Access Policies.xml CLI :: moquery -c infraAccBndlGrp .108 Fabric Connectivity REST :: /api/node/class/fabricHIfPol. 2 In the Navigation pane. 3 In the Work pane.xml CLI :: moquery -c fabricHIfPol Create LACP Policy 1 On the menu bar. However. Choose an LACP Policy. c. perform the following actions: a. choose Actions > Create vPC Interface Policy Group. 2 In the Navigation pane. choose Interface Policies > Policies > LACP. In mode click on Active. b. 4 In the Create vPC Interface Policy Group dialog box. Optionally. 5 Click Submit.xml CLI :: moquery -c lacpLagPol Create vPC Interface Policy Group 1 On the menu bar. Note: LACP is recommended for vPCs. REST :: /api/node/class/infraAccBndlGrp. ensure LACP is configured on the device connected to the leaf switch. e. 5 Click Submit. provide a description for the pool.

c. d. c. Select an interface policy group. 6 In the Navigation pane. 7 8 In the Work pane. 3 In the Work pane. 4 In the Create Switch Profile dialog box. Optionally. 5 Click Submit. ii. Name: 103-104 (example node numbers). Define a meaningful name for the profile. Optionally. choose Interface Policies > Profiles > Profiles > ACIVPC-int-profile. choose Actions > Create Switch Profile. choose Interface Policies > Profiles. choose Switch Policies > Profiles. REST :: /api/node/class/infraAccPortP. Select the proper interfaces. provide a description for the policy group.Fabric Connectivity 109 Create an Interface Profile 1 On the menu bar. i. 9 Click Submit. choose Actions > Create Access Port Selector. In the Create Access Port Selector dialog box. b. choose Fabric > Access Policies. b. . Define a meaningful name for the policy group. choose Fabric > Access Policies. 3 In the Work pane. provide a description for the profile. b. 2 In the Navigation pane.xml CLI :: moquery -c infraAccPortP Create a Switch Profile 1 On the menu bar. Blocks: Select switch 103 and switch 104. perform the following actions: a. Define a meaningful name for the profile. In switch selectors click on the + symbol. Optionally. 4 In the Create Interface Profile dialog box. provide a description. choose Actions > Create Interface Profile. 2 In the Navigation pane. perform the following actions: a. perform the following actions: a.

xml CLI :: moquery -c infraNodeP Create vPC domain 1 On the menu bar. click + to create a vPC protection group. choose Switch Policies > VPC Domain > Virual Port Channel default. You can also val​ id ​ate the op​ er​ at​ ion of the vPC di​ rectly from the CLI of the switch it​ self. REST :: /api/node/class/fabricExplicitGEp. choose POD 1 > Interfaces > vPC Interfaces. c. perform the following actions: b. REST :: /api/node/class/infraNodeP. a. choose Actions > Explicit VPC Protection Group. 2 In the Navigation pane. Provide a unique ID for the vPC domain. the status should be displayed and you should see successful establishment of the vPC domain. 6 Select the previously created interface selector. . 2 In the Navigation pane. In the Explicit VPC Protection Group dialog box. 4 Click Submit.xml CLI :: moquery -c fabricExplicitGEp Validate Operation of Configured vPC 1 On the menu bar.110 Fabric Connectivity 5 Click Next. choose Fabric > Inventory. there will be a table that shows the status of the vPC interface. d. choose Fabric > Access Policies. Select the first switch ID that is part of this vPC pair: 103. 3 In the Work pane. If configured correctly. In the Explicit VPC Protection Groups section. Select the second switch ID that is part of this vPC pair: 104. Define a meaningful name for the vPC domain. f. 7 Click Finish. 3 In the Work pane. e.

-----success success ------------ . forwarding via vPC peer-link vPC domain id : 100 vPC keep-alive status : Disabled Peer status : peer adjacency formed ok Configuration consistency status Per-vlan consistency status Type-2 consistency status : success : success : success vPC role : primary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------id -1 Port ---- Status Active vlans -----. Leaf-3# show vpc Legend: (*) .----------.local vPC is down.Fabric Connectivity 111 If you con​ nect to the con​ sole or the out of band man​ age​ ment in​ ter​ face of the leaf node you should be able to see the sta​ tus with the com​ mand show vpc.-------------------------------------------------up - vPC status ---------------------------------------------------------------------id Port Status Consistency Reason Active vlans 1 Po1 up - -- ---- -----.

xml <polUni> <fvTenant descr="" dn="uni/tn-Cisco" name="Cisco" ownerKey="" ownerTag=""> <fvAp descr="" name="CCO" ownerKey="" ownerTag="" prio="unspecified"> .Switch Selector --> <infraNodeP name="switchProfileforVPC_201"> <infraLeafS name="switchProfileforVPC_201" type="range"> <infraNodeBlk name="nodeBlk" from_="201" to_="201"/> </infraLeafS> <infraRsAccPortP tDn="uni/infra/accportprof-intProfileforVPC_201"/> </infraNodeP> <infraNodeP name="switchProfileforVPC_202"> <infraLeafS name="switchProfileforVPC_202" type="range"> <infraNodeBlk name="nodeBlk" from_="202" to_="202"/> </infraLeafS> <infraRsAccPortP tDn="uni/infra/accportprof-intProfileforVPC_202"/> </infraNodeP> <!-.112 Fabric Connectivity The fol​ low​ ing REST API call can be used to build vPCs and at​ tach vPCs to sta​ tic port bind​ ings.Interface Policy Group --> <infraFuncP> <infraAccBndlGrp name="intPolicyGroupforVPC" lagT="node"> <infraRsAttEntP tDn="uni/infra/attentp-AttEntityProfileforCisco"/> <infraRsCdpIfPol tnCdpIfPolName="CDP_ON" /> <infraRsLacpPol tnLacpLagPolName="LACP_ACTIVE" /> <infraRsHIfPol tnFabricHIfPolName="10GigAuto" /> </infraAccBndlGrp> </infraFuncP> </infraInfra> </polUni> https://{{hostName}}/api/node/mo/uni.xml <polUni> <infraInfra> <!-. URL: https://{{apic-ip}}/api/policymgr/mo/.Interface Profile --> <infraAccPortP name="intProfileforVPC_201"> <infraHPortS name="vpc201-202" type="range"> <infraPortBlk name="vpcPort1-15" fromCard="1" toCard="1" fromPort="15" toPort="15"/> <infraRsAccBaseGrp tDn="uni/infra/funcprof/accbundle-intPolicyGroupforVPC"/> </infraHPortS> </infraAccPortP> <infraAccPortP name="intProfileforVPC_202"> <infraHPortS name="vpc201-202" type="range"> <infraPortBlk name="vpcPort1-1" fromCard="1" toCard="1" fromPort="1" toPort="1"/> <infraRsAccBaseGrp tDn="uni/infra/funcprof/accbundle-intPolicyGroupforVPC"/> </infraHPortS> </infraAccPortP> <!-.

Fabric Connectivity 113 <fvAEPg descr="" matchT="AtleastOne" name="Web" prio="unspecified"> <fvRsPathAtt encap="vlan-1201" instrImedcy="immediate" mode="native" tDn="topology/pod-1/protpaths-201-202/pathep-[vpc201-202]” /> </fvAEPg> <fvAEPg descr="" matchT="AtleastOne" name="App" prio="unspecified"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/protpaths-201-202/pathep-[vpc201-202]” /> </fvAEPg> </fvAp> </fvTenant> </polUni> .

.

These poli​ cies are all gov​ erned by in​ ter​ face pol​ icy groups. Cisco Dis​ cov​ ery Pro​ to​ col (CDP) must be en​ abled on any ACI fab​ ric in​ ter​ faces that are con​ nect​ ing to UCS Fab​ ric In​ ter​ con​ nects. as well as some third party servers that all need to be con​ nected to the ACI fab​ ric. If the work​ load is a bare metal server. For more in​ for​ ma​ tion on the process needed to con​ fig​ ure links to UCS as ei​ ther a vPC or a tra​ di​ tional port chan​ nel. has sev​ eral dif​ fer​ ent mod​ els of servers in their data cen​ ters.Fabric Connectivity 115 Server Connectivity Server con​ nec​ tiv​ ity is nec​ es​ sary for all ap​ pli​ ca​ tion work​ loads to func​ tion prop​ erly on the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. the type of Layer 2 con​ nec​ tion needed on the Fab​ ric In​ ter​ con​ nect fac​ ing ports must be de​ ter​ mined first. A best prac​ tice is to lever​ age a vir​ tual pri​ vate cloud (vPC) to con​ nect the UCS en​ vi​ ron​ ment so as to cre​ ate a multichas​ sis ether​ chan​ nel. In this sce​ nario. In the case of Cisco Uni​ fied Com​ put​ ing Sys​ tem (UCS). see the Adding New De​ vices to the Fab​ ric sec​ tion. See the Con​ fig​ ur​ ing Man​ age​ ment Pro​ to​ cols chap​ ter for more in​ for​ ma​ tion. Standalone Rack Mount Servers or Non-Cisco Servers Any non-UCS server ar​ chi​ tec​ ture can also be con​ nected di​ rectly to the ACI fab​ ric or to a Cisco Nexus 2000 Fab​ ric Ex​ ten​ der (FEX). in​ di​ vid​ ual link and fab​ ric switch fail​ ures are mit​ i-​ gated to main​ tain a higher ex​ pected up time. fab​ ric ac​ cess poli​ cies must be pro​ vi​ soned to match these re​ quire​ ments. traf​ fic can be clas​ si​ fied on a per port basis and as​ so​ ci​ ated AEPs and EPGs can be mapped ap​ pro​ pri​ ately to match the en​ cap​ su​ lated traf​ fic. the Link Level Dis​ cov​ ery Pro​ to​ col (LLDP) must be dis​ abled. the kind of traf​ fic ex​ pected out of the server links needs to be de​ ter​ mined. You can pre-pro​ vi​ sion these poli​ cies as part of the ini​ tial man​ age​ ment tasks. The fab​ ric con​ nec​ tiv​ ity re​ quire​ ments that are dic​ tated by the server in​ fra​ struc​ ture must be care​ fully con​ sid​ ered. Cisco UCS B-Series Servers If UCS B-se​ ries Fab​ ric In​ ter​ con​ nects are being con​ nected to your ACI fab​ ric and Cisco UCS. When being con​ nected to the ACI fab​ ric. ACME Inc. When con​ nect​ ing UCS to the ACI fab​ ric. such as Cisco UCS B and C se​ ries. If a sup​ ported hy​ - .

Re​ stric​ tions that are pre​ sent in NX-OS mode such that non-host-fac​ ing ports are not sup​ ported. a Vir​ tual Ma​ chine Man​ ager (VMM) do​ main must be prop​ erly con​ fig​ ured. .116 Fabric Connectivity per​ vi​ sor is to be used. Uti​ liz​ ing a FEX is an al​ ter​ na​ tive way to con​ nect host de​ vices into the ACI fab​ ric. Ports must only be con​ nected to hosts. all host-fac​ ing ports are treated the same way as if they were di​ rectly at​ tached to the ACI fab​ ric. and then as​ so​ ci​ ated with the cor​ re​ spond​ ing ports on the fab​ ric as a hy​ per​ vi​ sor that is fac​ ing through EPG and AEP map​ ping. When uti​ liz​ ing a FEX. and con​ nec​ tiv​ ity to any other net​ work de​ vice will not func​ tion prop​ erly. The key is to map the ex​ pected traf​ fic clas​ si​ fi​ ca​ tion to the ports that are con​ nected to the server in​ fra​ struc​ ture. are still true.

A leaf switch does not support overlapping VLAN pools. A virtual machine controller administrator provides an APIC administrator with a virtual machine controller authentication credential. the user defines the interface and the encapsulation. such as VMware vCenter. • A pool represents a range of traffic encapsulation identifiers. This is the pool type that is required for VMM integration. The encapsulation must be within the range of a pool that is associated with a domain with which the EPG is associated. For static EPG deployment. A pool is a shared resource and can be consumed by multiple domains. The ACI pro​ vides the abil​ ity to man​ age both vir​ tual and phys​ ic ​al end​ points with the same set of poli​ cies. This chap​ ter will look at var​ io ​us op​ er​ at​ ional tasks that will be per​ formed through​ out the daily op​ er​ at​ ions. Multiple controllers can use the same credentials. A VMware vCenter domain can associate only to a dynamic pool. . The Application Policy Infrastructure Controller (APIC) communicates with the VMM to publish network policies that are applied to virtual workloads. Static pools . The fol​ low​ ing list de​ scribes some vir​ tual ma​ chine man​ ager (VMM) sys​ tem terms: • A virtual machine controller is an external VMM entity. and Microsoft Systems Center Virtual Machine Manager (SCVMM). and multicast addresses.Managed internally by the APIC to allocate VLANs for endpoint groups (EPGs). Different overlapping VLAN pools must not be associated with the same attachable entity profile (AEP). and the domain has a relation to the pool. such as VLAN and VXLAN IDs. VMware vShield. multiple controllers of the same type can use the same credential. The pool contains a range of encapsulated VLANs and VXLANs.The EPG has a relation to the domain. • The two types of VLAN-based pools are as follows: Dynamic pools .Fabric Connectivity 117 Virtual Machine Networking Understanding VM Networking in ACI One of the most com​ mon uses of the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) will be to help man​ age and de​ ploy ap​ pli​ ca​ tions in vir​ tual en​ vi​ ron​ ments. such as VMM and Layer 4 to Layer 7 services. • Credentials represent the authentication credentials to communicate with virtual machine controllers.

how​ ever. see the fol​ low​ ing doc​ u​ ments: • Cisco APIC Getting Started Guide VMware Integration When in​ te​ grat​ ing ACI into your VMware in​ fra​ struc​ ture you have two op​ tions for de​ ploy​ ing net​ work​ ing.118 Fabric Connectivity When cre​ at​ ing dy​ namic VLAN pools for VMM in​ te​ gra​ tion. ACI VM Integration Workflow ACI VM Integration Workflow For de​ tailed in​ for​ ma​ tion on how to de​ ploy the VMware vSphere Dis​ trib​ uted Switch with the Cisco APIC. This in​ cludes cre​ at​ ing the VLANs on Uni​ fied Com​ put​ ing Sys​ tem (UCS). the VLAN range must also be cre​ ated on any in​ ter​ me​ di​ ate de​ vices. Both pro​ vide sim​ i​ lar basic vir​ tual net​ work​ ing func​ tion​ al​ ity. lever​ ag​ ing the VMware vSphere Dis​ trib​ uted Vir​ tual Switch (DVS) or the Cisco Ap​ pli​ ca​ tion Vir​ tual Switch (AVS). the AVS pro​ vides . VMware do​ mains can be de​ ployed. such as tra​ di​ tional switches or blade switches.

For or​ ga​ ni​ za​ tions in​ ter​ ested in using the stan​ dard DVS pro​ vided by VMware. has cho​ sen to lever​ age the ad​ di​ tional fea​ tures pro​ vided by AVS. This serves as a ref​ er​ ence for the ways the var​ i​ ous poli​ cies are re​ lated to each other.​ cisco. VMM Policy Model Interaction . ACME Inc. such as VXLAN and mi​ croseg​ men​ ta​ tion sup​ port.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ getting-started/​ video/​ cisco_​ apic_​ create_​ vcenter_​ domain_​ profile_​ using_​ gui.​ html VMM Policy Model Interaction Shown below are some of the var​ i​ ous ACI poli​ cies which are in​ volved with set​ ting up VM In​ te​ gra​ tion. please refer to the fol​ low​ ing doc​ u​ ments: http://​ www.Fabric Connectivity 119 ad​ di​ tional ca​ pa​ bil​ i​ ties.

4 In the Work pane. 3 In the Navigation pane. 6 Click Submit. For an EPG to be pushed to a VMM do​ main. Connecting VMs to the EPG Port Groups on vCenter 1 Connect to your vCenter using the VI Client. a. choose Actions > Add VM Domain Association. choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > Application_EPG_Name > Domains (VMs and Bare-Metals). For de​ tails on how to cre​ ate EPGs. 3 Click on the Network Adapter. choose Tenants > ALL TENANTS 2 In the Work pane. choose the VMM Domain Profile that you previously created. 5 In the Add VM Domain Association dialog box.120 Fabric Connectivity Publishing EPGs to a VMM Domain This sec​ tion will de​ tail how to pub​ lish an ex​ ist​ ing end​ point group (EPG) to a Vir​ tual Ma​ chine Man​ ager (VMM) do​ main. 2 From the Host and Clusters view. and in the Network Connection dropdown box select the Port Group which corresponds to your EPG. right click on your Virtual Machine and select "Edit Settings". There is no communication delay or traffic loss by keeping the default selections. Cisco recommends keeping the default option of On Demand. For the Deployment & Resolution Immediacy. see the Ten​ ants sec​ tion. a do​ main bind​ ing within the ten​ ant EPG must be cre​ ated. It should display in the format of [TENANT|APPLICATION_PROFILE|EPG|VMM_DOMAIN_PROFILE] . Note: The EPG will now be available as a Port Group to your VMM. This provides the best resource usage in the fabric by only deploying policies to Leaf nodes when endpoints assigned to this EPG are connected. choose the Tenant_Name. 1 On the menu bar.

Verifying VM Endpoint Learning on the APIC from the CLI You can ver​ ify the same info as above from the CLI by using the 'mo​ query' (Man​ aged Ob​ ject Query) com​ mand and adding two fil​ ters. One for the Dis​ tin​ guished Name (DN) name of your EPG. it means one of the fol​ low​ ing: • The VM is running on a host which is not attached to the Distributed Switch • There may be a communication between your APIC and vCenter either through managed by the APIC. Note: The current tab should display CLIENT ENDPOINTS. you should ver​ ify the APIC has learned your vir​ tual end point. From here you should be able to find your Virtual Machine by filtering the "Learning Source" column for rows with values of "Learned VMM". Verifying Virtual Endpoint Learning Once the VMs are con​ nected to the ap​ pro​ pri​ ate port group/EPG. choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > Application_EPG_Name. choose the Operational tab. choose the Tenant_Name. 1 On the menu bar. All endpoints either virtual or physical will be displayed. 2 In the Work pane. choose Tenants > ALL TENANTS. 4 In the Work pane. 3 In the Navigation pane.Fabric Connectivity 121 If you do not see your ACI EPG in the Net​ work Con​ nec​ tion list. the OOB or INB management network. se​ lect​ ing "Save As" and look​ ing at the XML ob​ ject. and one for the Class Name of 'fvCEp' (Fab​ ric Vec​ tor Client End​ point) moquery -c fvCEp --dn uni/tn-<TENANT_NAME>/ap-<APP_PROFILE_NAME>/epg-<EPG_NAME> You can de​ ter​ mine the DN of your EPG by right click​ ing on the EPG in the GUI. From this file you will see the DN entry for the par​ tic​ ul​ ar EPG: .

229+11:00 rn : cep-00:50:56:BB:8C:6A status : uid : 0 uuid : VMware Integration Use Case A VMWare ad​ min​ is​ tra​ tor in ACME re​ quests the net​ work team to trunk a set of VLANs down to the ESX hosts to pro​ vide con​ nec​ tiv​ ity to their DVS switches. .122 Fabric Connectivity <imdata totalCount="1"><fvAEPg uid="15374" triggerSt="triggerable" status="" scope="2588672" prio="unspecified" pcTag="49159" name="epg-od" monPolDn="uni/tn-common/monepg-default" modTs="2015-02-06T06:46:24. use this DN with the mo​ query to re​ turn the list of client En​ points for this EPG: admin@apic1:~> moquery -c fvCEp --dn uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od Total Objects shown: 1 # fv.CEp name : 00:50:56:BB:8C:6A childAction : dn : uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od/cep-00:50:56:BB:8C:6A encap : vlan-211 id : 0 idepdn : ip : 10.10 lcC : learned. the net​ work team de​ cides to lever​ age a new method​ ol​ ogy to be more agile and lever​ age the on-de​ mand pro​ vi​ sion​ ing of re​ sources where and when they are needed.vmm lcOwn : local mac : 00:50:56:BB:8C:6A mcastAddr : not-applicable modTs : 2015-02-06T06:48:52. as well as pro​ vid​ ing un​ lim​ ited Layer 2 mo​ bil​ ity for all the VM hosts within the ACI fab​ ric.10.10. Rather than trunk​ ing VLANs on a per server basis.729+11:00" matchT="AtleastOne" lcOwn="local" dn="uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od" descr="" configSt="applied" configIssues="" childAction=""/></imdata> Next.

ACME chose VMM in​ te​ gra​ tion as the pre​ ferred de​ ploy​ ment model as it is the most ef​ fec​ tive method of break​ ing down or​ ga​ ni​ za​ tional chal​ lenges. How​ ever.800). APIC dy​ nam​ ic ​ally pro​ vi​ sions all EPGs and makes them avail​ able to the ESX hosts as a port-group. doing on-de​ mand re​ source al​ lo​ ca​ tion. the APIC dy​ nam​ ic ​ally com​ mu​ ni​ cates with vCen​ ter to make EPGs avail​ able through port groups. . Dur​ ing a vMo​ tion event.Fabric Connectivity 123 To do so. This is their dy​ namic VLAN pool. the net​ work ad​ mins work with the VMware ad​ mins to de​ cide on a range of VLANs that will be pro​ vided dy​ nam​ ic ​ally by APIC to the ESX hosts that need them. As the VMware admin pro​ vi​ sions ESX hosts and se​ lects the ap​ pro​ pri​ ate port-groups for VMs. It is im​ por​ tant to note that ACME can still choose to de​ ploy tra​ di​ tional VLAN trunk​ ing down to VMware DVS switches by sta​ t​ i​ cally pro​ vi​ sion​ ing EPGs on a per-port basis. the APIC ad​ min​ is​ tra​ tor pro​ ceeds to con​ fig​ ure VMM in​ te​ gra​ tion in the APIC GUI by pro​ vid​ ing the vCen​ ter cre​ den​ tials to APIC. This al​ lows VMware ad​ mins to main​ tain con​ trol and move vir​ tual NICs into these port-groups on de​ mand. and get​ ting en​ hanced vis​ i​ bil​ ity and teleme​ try into both the vir​ tual and phys​ i​ cal en​ vi​ ron​ ments. The APIC also con​ fig​ ures VLAN IDs on the leaf-switches as needed. and still reap the ad​ van​ tages of the Layer 2-any​ where ACI fab​ ric. Once this is de​ cided. Note: The APIC does not au​ to​ mat​ ic ​ally move VM​ NICs into the port-group. VMs are al​ lowed to move any​ where within the ACI fab​ ric with no re​ stric​ tions other than those im​ posed by vCen​ ter. APIC is au​ to​ mat​ ic ​ally in​ formed of the VM move and then up​ dates the end​ point track​ ing table to allow seam​ less com​ mu​ ni​ ca​ tion. They de​ cide on an un​ used VLAN range of (600 .

.

• A dynamic VLAN pool has been created with enough VLAN IDs to accommodate one VLAN per EPG you plan on deploying to each VMM domain. • VMware vCenter is installed. • One or more vSphere hosts are available for deployment to the AVS. These can be down​ loaded from CCO at the fol​ low​ ing lo​ ca​ tion: Down​ loads Home > Prod​ ucts > Switches > Vir​ tual Net​ work​ ing > Ap​ pli​ ca​ tion Vir​ tual Switch .Fabric Connectivity 125 Deploying the Application Virtual Switch Prerequisites • All switch nodes have been discovered by the fabric • INB or OOB management connectivity is configured. Each ver​ sion of AVS soft​ ware in​ cludes the VIB files for all sup​ ported vSphere ver​ sions.1. The AVS pack​ age for ei​ ther ver​ sion will in​ clude vSphere In​ stal​ la​ tion Bun​ dles (VIBs).2. This al​ lows ei​ ther de​ vice to be up​ graded in​ de​ pen​ dently. The ini​ tial AVS soft​ ware re​ leased was ver​ sion 4. • A DNS server policy has been configured to enable connection to a VMM using a hostname.0 is not sup​ ported).1 and 5. Just like any soft​ ware.5 (vSphere 5. Al​ ways refer to the AVS re​ lease notes to con​ firm if any spe​ cial con​ sid​ er​ at​ ions may exist. and available.1. fol​ lowed by tem Com​ pat​ i​ bil​ ity List doc​ um ​ent to en​ sure your ver​ sion 5.2. Getting Started The AVS soft​ ware was de​ signed to op​ er​ ate in​ de​ pen​ dently of the APIC soft​ ware ver​ sion. new ver​ sions of the AVS will be re​ leased to in​ clude new fea​ tures and im​ prove​ ments. Refer to the ACI Ecosys​ de​ sired ver​ sion of AVS is com​ pat​ ib ​le with the APIC and vSphere ver​ sions being run. configured. As of this pub​ li​ ca​ tion there are two VIBs to sup​ port vSphere ver​ sions 5.

This can be achieved in a va​ ri​ ety of ways.2.vib Install the AVS VIB Be​ fore set​ ting up the AVS con​ fig​ ur​ a​ tion on the APIC.3. The easiest way to copy the VIB to the host is to leverage the VMware VI Client. For a few hosts.1.1. Right click on the desired datastore and select Browse.3. From here.2. re​ ferred to as the Vir​ tual Eth​ er​ net Mod​ ule (VEM).3.2.2.2. all of which are dis​ cussed in Cisco Ap​ pli​ ca​ tion Vir​ tual stal​ la​ tion Guide.126 Fabric Connectivity The VIBs can be iden​ ti​ fied as fol​ lows: AVS 4.1.2. but for 10+ Switch In​ hosts it may be eas​ ier to lever​ age the Vir​ tual Switch Up​ date Man​ ager (VSUM) to help au​ to​ mate the in​ stal​ la​ tion process.vib cross_cisco-vem-v172-5.1 Bundle cross_cisco-vem-v172-5.2.3.VIB can be uploaded directly to the host's datastore. the AVS soft​ ware must be in​ stalled in vSphere.0-3.1. this can eas​ ily be done man​ ua ​lly.1.0-3.1.2. Manual Installation 1 Copy the VIB file to a host.1. 2 SSH into the vSphere host on which the AVS VIB is to be installed. 3 Install or upgrade the VIB using the esxcli command: To in​ stall the AVS VIB: esxcli software vib install -v /<path>/<vibname> --maintenance-mode --no-sig-check . navigate to the Host > Configuration > Storage > Datastore_X.2.2.3.1.1 Bundle cross_cisco-vem-v165-4.1.0-3.1.vib AVS 5.3.0-3.1.vib cross_cisco-vem-v165-4.2.1.2.

0 Releasebuild-1623387 4 Confirm the VEM is loaded and running. Reboot Required: false VIBs Installed: Cisco_bootbank_cisco-vem-v172-5.1.1.3.0-3.2.Fabric Connectivity 127 To Up​ grade an ex​ ist​ ing AVS VIB: esxcli software vib update -v /<path>/<vibname> --maintenance-mode --no-sig-check A sam​ ple out​ put is shown below: # esxcli software vib install -v /vmfs/volumes/datastore1/cross_cisco-vem-v1725.1 VIBs Removed: VIBs Skipped: /vmfs/volumes/53cab6da-55209af3-0ef2-24e9b391de3e # vem version Running esx version -1623387 x86_64 VEM Version: 5.0-3.2. .2.3.1. This will allow the AVS to cre​ ate a vmk Vir​ tual Tun​ nel End​ point (VTEP) in​ ter​ face on each host.vib --maintenance-mode --no-sig-check Installation Result Message: Operation finished successfully.1 VSM Version: System Version: VMware ESXi 5.3.3.3.2.1.1.3.0-3. This VTEP in​ ter​ face will be used for the Open​ Flex con​ trol chan​ nel and/or the VXLAN tun​ nel source be​ tween the VTEP and ACI fab​ ric.5.1.2.1. # vem status VEM modules are loaded Switch Name Num Ports Used Ports Configured Ports MTU Uplinks vSwitch0 3072 6 128 1500 VMNIC0 VEM Agent (vemdpa) is running DHCP Relay The first task is to cre​ ate a DHCP relay pol​ icy in the infra ten​ ant on the APIC.2.

this will allow the AVS VTEP in​ ter​ faces to pull an ad​ dress from this pool. 4 5 In the Work pane. choose infra > Networking > Bridge Domains > default > DHCP Relay Labels. choose the infra. 2 In the Work pane. STEPS 1 On the menu bar.​ 0/​ 16). choose Tenants > ALL TENANTS. choose the infra. perform the following actions: a. The way APIC would do it is through over the infra vlan. From the Name drop-down list. 3 In the Navigation pane. choose infra > Networking > Protocol Policies > DHCP > Relay Policies.​ 0. 4 5 In the Work pane. . In the Create DHCP Relay Policy dialog box.128 Fabric Connectivity If the de​ fault TEP Ad​ dress pool was used dur​ ing ini​ tial setup (10. such as "avs-dhcp-relay-pol". Leave other fields blank and click OK. The APIC IP ad​ dresses au​ to​ mat​ ic ​ally func​ tion as the DHCP providers and DO NOT need to be ex​ plic​ itly added. choose Tenants > ALL TENANTS. perform the following actions: a. Change the Scope to TENANT. choose Actions > Create DHCP Relay Policy. 2 In the Work pane. For this rea​ son it is crit​ ic ​al to ex​ tend the Infra VLAN from a leaf through to each vSphere host that will host the AVS. 6 Click Submit. b. Create the Infra DHCP Relay Policy Cre​ ate a DHCP relay pol​ icy that will setup the DHCP server (the APIC in this case) and a label using the Infra Ten​ ant and de​ fault EPG. In the Create DHCP Relay Policy dialog box. choose the DHCP Relay Policy that you created previously. Create the DHCP Relay Label 1 On the menu bar. 6 Click Submit. b. 3 In the Navigation pane.​ 0. Provide a name for the Relay policy. choose Actions > Create DHCP Relay Policy.

. b. but this guide will de​ tail cre​ at​ ing the AEP sep​ ar​ ately first. This will be covered later in the Publishing EPGs to VMM Domains chapter. and CDP. 2 In the Navigation pane. 3 In the Work pane. Re​ fer​ ring back to the "VMM Pol​ icy Model In​ ter​ ac​ tion" di​ ag ​ram from the "VM Net​ work​ ing Overview" chap​ ter. Name: Provide any name to identify the AEP.Fabric Connectivity 129 Attachable Access Entity Profile (AEP) and AVS An im​ por​ tant com​ po​ nent used by the AVS is the At​ tach​ able En​ tity Pro​ file (AEP). 4 In the Create Attachable Access Entity Profile dialog box. select the Interface Policy Group your AEP will be associated to. From the next page of the wizard. the AEP is what ties the VMM do​ main to the phys​ ic ​al in​ ter​ faces where the vSphere hosts are con​ nected. See the "Adding New De​ vices to the Fab​ ric" chap​ ter for more de​ tail on cre​ at​ ing the in​ ter​ face pol​ icy group and in​ ter​ face pro​ files. Create a New AEP 1 On the menu bar. i. The AEP can be cre​ ated on-the-fly dur​ ing the cre​ ation of the VMM do​ main it​ self. choose Fabric > Access Policies. the En​ able In​ fra​ struc​ ture VLAN check box must be checked for the AEP pol​ icy. Note: In​ ter​ face Pol​ icy Group cre​ ation is cov​ ered else​ where in this guide. This is to en​ sure that the traf​ fic of in​ ter​ est (DHCP re​ quest/offer can flow through the in​ fra​ struc​ ture VLAN to the AVS). Es​ sen​ tially the In​ ter​ face Pol​ icy Group is a col​ lec​ tion of In​ ter​ face poli​ cies which de​ fine In​ ter​ faces Se​ lec​ tors and prop​ er​ ties. Click the All Interfaces radio button for the desired Interface Policy Group. iii. This can be com​ pared to a "switch​ port trunk al​ lowed VLAN xxx" com​ mand in tra​ di​ tional net​ work​ ing. Fill in the AEP wizard information then click Next. such as "AVS-AEP". Enable Infrastructure VLAN: Check this box. ii. choose Actions > Create Attachable Access Entity Profile. choose Global Policies > Attachable Access Entity Profile. This procedure assumes your Interface Policy Group has already been created. perform the following actions: a. Re​ gard​ less of using an ex​ ist​ ing AEP or cre​ at​ ing a new one. The AEP de​ fines which VLANs will be per​ mit​ ted on a host fac​ ing in​ ter​ face. Domains (VMs or Baremetal): Leave blank. such as speed/ne​ go​ ti​ at​ ion. LLDP.

This al​ lows the same poli​ cies ap​ plied to phys​ ic ​al end​ points. vCen​ ter VMM Do​ mains are cre​ ated using ei​ ther the VMware DVS or Cisco AVS. You can​ not change from one to the other. to also be ap​ plied to vir​ tual end​ points. • No Local Switch only supports VLAN encapsulation. a. 2 In the Navigation pane. AVS Switching Modes The AVS can op​ er​ ate in the fol​ low​ ing switch​ ing modes: • Local Switching: Supports VXLAN encapsulation or VLAN encapsulation. choose the existing AEP b. This switching mode allows Inter-EPG traffic to be switched locally on the AVS. Note: As men​ tioned early in this chap​ ter. the In​ fra​ struc​ ture VLAN is re​ quired for AVS com​ mu​ ni​ ca​ tion to the fab​ ric using the Open​ Flex con​ trol chan​ nel. choose Global Policies > Attachable Access Entity Profile. VMM Domains for vCenter A Vir​ tual Ma​ chine Man​ ager (VMM) do​ main de​ fines a vir​ tual in​ fra​ struc​ ture that will be in​ te​ grated into ACI. A new VMM Do​ main will be cre​ ated from scratch to sup​ port AVS de​ ploy​ ment. . choose Fabric > Access Policies.130 Fabric Connectivity Modify an Exisiting AEP 1 On the menu bar. check the Enable Infrastructure VLAN check box. In the Navigation pane. In the Work pane. This switching mode sends all traffic (Inter-EPG included) to the Leaf switch.

All traf​ fic be​ tween the AVS up​ links and ACI fab​ ric will be en​ cap​ su​ lated by VXLAN and trans​ fered using the in​ fra​ struc​ ture VLAN. If VLAN en​ cap​ su​ la​ tion is pre​ ferred. 1 On the menu bar. 3 In the Work pane. you will need to en​ sure every VLAN in the VM Do​ main VLAN pool has been ex​ tended be​ tween the fab​ ric and AVS host. . When using VXLAN en​ cap​ su​ la​ tion. This in​ cludes cre​ at​ ing the VLANs on in​ ter​ me​ di​ ate de​ vices such as UCS and the vNICs for any AVS vSphere hosts. you can cre​ ate the VMM do​ main for the AVS. Create the VMM Domain for AVS Now that the DHCP server pol​ icy has been cre​ ated and AEP cre​ ated/mod​ i​ fied. choose VM NETWORKING. choose Actions > Create VCenter Domain. 2 In the Navigation pane. only the infra VLAN is re​ quired to be ex​ tended to the AVS hosts.Fabric Connectivity 131 ​ AVS Switching Modes: Non-Local and Local switching mode The de​ ci​ sion be​ tween using VLAN or VXLAN en​ cap​ su​ la​ tion will man​ date dif​ fer​ ent VLAN ex​ ten​ sion re​ quire​ ments out​ side of the fab​ ric. choose the Policies tab.

vCenter: Add the vCenter details. perform the following actions: a. vCenter Credentials: Create a credential set with administrator/root acces to vCenter f. Name: This value will be used as the AVS "Switchname" displayed in vCenter. . Click Submit. Attachable Access Entity Profile: <Choose the AEP previously created/modified> e. b. Switching Preference: <Choose Local or No Local Switching> • For No Local Switching mode: Multicast Address: <Assign a multicast address to represent your AVS> Multicast Address Pool: <Create a unique Multicast Address Pool large enough to include each AVS vSphere host.> • For Local Switching mode: Encapsulation: <Choose VLAN or VXLAN based on preference> For VLAN Encapsulation VLAN Pool: <Choose/Create a VLAN pool> For VXLAN Encapsulation Multicast Address: <Assign a multicast address to represent your AVS> Multicast Address Pool: <Create a unique Multicast Address Pool large enough to include each AVS vSphere host.> d. Virtual Switch: Cisco AVS c.132 Fabric Connectivity 4 In the Create vCenter Domain dialog box. 5 • Name: Friendly name for this vCenter • Hostname/IP Address: <DNS or IP Address of vCenter> • DVS Version: vCenter Default • Datacenter: <Enter the exact Datacenter name displayed in vCenter> • Management EPG: <Set to oob or inb Management EPG> • Associated Credentials: <Choose the Credential set previously created> • Click OK to complete the creation of the vCenter.

Note: For blade switch sys​ tems such as UCS. 2 Expand this folder to find your AVS. 1 From the vCenter client. In UCS terms. choose any vSphere hosts to add to the AVS. To do this you will need at least one un​ used phys​ ic ​al in​ ter​ face (VMNIC) to act as the up​ link on each host.. you then need to at​ tach hosts to it. .. Add vSphere Hosts to the AVS After the AVS has been cre​ ated in vCen​ ter. navigate to HOME > INVENTORY > NETWORKING. skipping the migration of any virtual adapters or virtual machine networking at this time.. this re​ quires the vNIC within the ser​ vice pro​ file to have all rel​ e​ vant VLANs ac​ tive on the vNICs. 3 In the Add Host dialog box. navigate to HOME > INVENTORY > NETWORKING and confirm a new Distributed Virtual Switch folder has been created.Fabric Connectivity 133 Verify AVS Deployment on vCenter 1 In the vCenter client. and select an unassigned VMNIC uplink. 4 Click Next until the wizard completes. the VMNIC in​ ter​ face used must have all nec​ es​ sary VLANs al​ lowed on the in​ ter​ face. 2 Right click on the newly created AVS switch (not the folder) and choose Add Host. and a few default Port Groups including "uplink" and "vtep". AVS up​ links can not be shared with any other ex​ ist​ ing vSwitch or vDS.

255.255.0.0/16 pool that is created during the APIC setup to provision the IP address.0.99. we see that the VMkernel port has received the IP address from the APIC.168.255. The APIC uses the same 10.0.255.168.255 IP Family IP Address MAC Address IPv4 MTU TSO MSS Enabled Type 65535 true STATIC true STATIC true DHCP 172.0. you should see a new vmk interface created on your distributed switch within vCenter and assigned to the 'vtep' port group.54 00:25:b5:00:00:29 1500 IPv4 192.0 vmk2 255.16.0 172.176.0 vmk1 255.255 9 10.95 00:50:56:65:3d:b3 1500 65535 . ~ # esxcfg-VMKNIC -l Interface Port Group/DVPort Netmask Broadcast vmk0 Management Network 255. This vmk is your Virtual Tunnel Endpoint (vtep) interface and should have pulled a DHCP address from the APIC from the TEP subnet. This implies that we are ready for Opflex communication in between the AVS and the APIC.176.16.134 Fabric Connectivity Adding Virtual host physical NICs to participate in Virtual Switch 5 Assuming the ACI fabric can reach the vSphere host over the infra VLAN. As can be seen from the screenshot below.0.255.255.255 vmotion 192.99.16.54 00:50:56:61:1c:92 1500 IPv4 65535 10.

there should be one mul​ ti​ cast group per de​ ployed EPG on the host.0.Fabric Connectivity 135 Verify AVS on ESX On the ESX com​ mand line.127.0.0.0.0. In the out​ put below.0. Also ver​ ify that the GIPO ad​ dress is the same as the mul​ ti​ cast ad​ dress that was used while cre​ at​ ing the VMM do​ main.92 1 These mul​ ti​ cast ad​ dresses will cor​ re​ spond to the EPG de​ tails found in the APIC GUI antX > Ap​ pli​ ca​ tion Pro​ files > Ap​ pli​ ca​ tion​ Pro​ fileX > End Point under Ten​ ants > Ten​ Groups > End​ Point​ GroupX and click the Op​ er​ a​ tional Tab > Client End Points.1. ~ # vemcmd show openflex Status: 12 (Active) Dvs name: comp/prov-VMware/ctrlr-[AVS-TEST]-VC/sw-dvs-87 Remote IP: 10.76 1 225.0.0. Ver​ ify that the ‘sta​ tus: 12 (Ac​ tive)’ is seen as well as the switch​ ing mode.1 Ver​ ify on the AVS host .0. there are three dif​ fer​ ent Vir​ tual Ma​ chines con​ nected to dif​ fer​ ent EPGs.30 Port: 8000 Infra vlan: 4093 FTEP IP: 10.0. issue the ‘vem​ cmd show open​ flex’ com​ mand. you should have an equal num​ ber of VMKNICs and up​ links. A max​ im ​um of eight VMKNICs can be .58 1 225. VXLAN Load Balancing VXLAN load bal​ anc​ ing is au​ to​ mat​ ic ​ally en​ abled as soon as more than one VMKNIC is con​ nected to the Cisco AVS. Each VMKNIC can use only one up​ link port. ~ # vemcmd show epp multicast Number of Group Additions 3 Number of Group Deletions 0 Multicast Address EPP Ref Count 225.32 Switching Mode: LS NS GIPO: 225.

you need to add four VMKNICs to en​ able VXLAN load-bal​ anc​ ing. the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) al​ ready cre​ ated one VMKNIC when the host was added to the dis​ trib​ uted vir​ tual switch (DVS). Note that each VMK in​ ter​ face added is as​ signed a unique DHCP ad​ dress from the fab​ ric TEP pool.136 Fabric Connectivity at​ tached to a Cisco AVS switch. you will need to cre​ ate an ad​ di​ tional vir​ tual adapter (VMK) for each AVS up​ link. In VXLAN load bal​ anc​ ing. the VMKNICs pro​ vide a unique MAC ad​ dress to pack​ ets of data that can then be di​ rected to use cer​ tain phys​ i​ cal NICs (VM​ NICs). You need to have as many VMKNICs as the host has VM​ NICs. Each vmk in​ ter​ face cre​ ated for the AVS should be at​ tached to the vtep port group and con​ fig​ ured for DHCP. Each of the VMKNICs that you cre​ ate has its own soft​ ware-based MAC ad​ dress. For ex​ am​ ple. Distributed Switch configured with APIC port groups . In VMware vSphere Client. up to a max​ i​ mum of eight. In the screen​ shot below you can see the four VMNIC up​ links to the AVS and the four vmk vir​ tual in​ ter​ faces to pro​ vide equal load bal​ anc​ ing traf​ fic across all up​ links. if the host has five VM​ NICs.

c. The IGMP snoop​ ing pol​ icy needs to be en​ abled on the infra ten​ ant. b. choose Create IGMP Snooping Policy. . a. then IGMP pol​ icy is not needed since the blade switch will flood the mul​ ti​ cast traf​ fic on all the rel​ e​ vant ports. Click the Fast Leave check box. Note: Verify if IGMP snooping is working properly on the vSphere host CLI using 'vemcmd show epp multicast' as shown above. 2 In the Work pane. Provide a name for the policy. choose infr > Networking > Bridge Domain > default. Click the Enable Querier check box. USC-B FI has IGMP snoop​ ing en​ abled. This will cause flood​ ing of the mul​ ti​ cast traf​ fic to all end​ points. choose Tenants > ALL TENANTS. In the IGMP Snoop Policy drop-down list. we have to con​ fig​ ure an IGMP querier pol​ icy on the APIC. Create an IGMP Snooping Policy for AVS 1 On the menu bar. If we dis​ able IGMP snoop​ ing on UCS or other in​ ter​ me​ di​ ate blade switches. 4 Click Submit. d.Fabric Connectivity 137 IGMP Snooping Policy for AVS Cisco UCS-B Series Considerations with AVS Deployments This sec​ tion of the ar​ ti​ cle will focus the nec​ es​ sary steps to en​ able AVS through the Cisco UCS-B se​ ries. such as "IGMP_Infra". By de​ fault. The al​ ter​ nate method would be to cre​ ate an IGMP pol​ icy on UCS to dis​ able IGMP snoop​ ing. 3 In the Navigation pane. choose infra. Due to this.

.

such as a VLAN resource and policy content-addressable memory . In the Path field. perform the following actions: a. vPC or Interface dialog box. to fur​ ther ex​ tend con​ nec​ tiv​ ity to a re​ mote data cen​ ter. There​ fore. 2 In the Work pane. all of the traf​ fic re​ ceived on this leaf port with the con​ fig​ ured VLAN ID will be mapped to the EPG. must con​ fig​ ure some Layer 2 con​ nec​ tiv​ ity. To as​ sign a Layer 2 con​ nec​ tion sta​ ti​ c ​ally on an ACI leaf port to an EPG: 1 On the menu bar. the con​ fig​ ured pol​ icy for this EPG will be en​ forced. as the traf​ fic clas​ si​ fi​ ca​ tion will be based on the en​ cap​ su​ la​ tion re​ ceived on a port. vPC or Interface. specify a port as well as a VLAN ID.Fabric Connectivity 139 External Connectivity Extending ACI to External Layer 2 As men​ tioned in the in​ tro​ duc​ tion of this book. 5 In the Deploy Static EPG on PC. 3 In the Navigation pane. b. The end​ points do not need to be di​ rectly con​ nected to the ACI leaf. is a multi​ na​ tional com​ pany with mul​ ti​ ple data cen​ ters. choose the Tenant_Name. The immediacy also determines when the hardware resource. choose Tenants > ALL TENANTS. choose Action > Deploy Static EPG on PC. 4 In the Work pane. ACME Inc. Extending Endpoint Groups Outside the ACI Fabric The sim​ plest way to ex​ tend an end​ point group (EPG) out​ side of the ACI fab​ ric is to sta​ t-​ i​ cally as​ sign a leaf port and VLAN ID to an ex​ ist​ ing end​ point group. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). ACME Inc. and as such. This is nec​ es​ sary for ex​ tend​ ing Layer 2 con​ nec​ tiv​ ity to a Data Cen​ ter In​ ter​ con​ nect (DCI) plat​ form. Click one of the Deployment Immediacy radio buttons. Deployment immediacy determines when the actual configuration will be applied on the leaf switch hardware. After doing so. or sim​ ply to ex​ tend a Layer 2 do​ main out​ side of the fab​ ric to con​ nect in an ex​ ist​ ing Layer 2 net​ work in a non-ACI fab​ ric.

Click one of the Mode radio buttons.140 Fabric Connectivity (CAM) to support the related contract for this EPG. Untagged .The tagged option means that the leaf node expects incoming traffic to be tagged with the specified VLAN ID previously established. but (unlike the untagged mode) 802. This is the default deployment mode.1P . The mode option specifies whether the ACI leaf expects incoming traffic to be tagged with a VLAN ID or not. c. Associate the physical domain to the EPG in question. Tagged . This option can be used to connect a leaf port to a bare metal server whose network interface cards (NICs) typically generate untagged traffic. 802. will be consumed on the leaf switch.1P will allow other 'tagged' EPGs to be statically bound to the same interface. Similar to the switchport access vlan vlan_ID command. A port can have only one EPG statically bound to a port as untagged.1P option refers to traffic tagged with 802. with this option you can only assign the interface to one EPG. Multiple EPGs can be statically bound to the same interface as long as the encap VLAN/VXLAN ID is unique. ii. Choose this mode if the traffic from the host is tagged with a VLAN ID. .The untagged option means that the leaf expects untagged traffic without a VLAN ID. Any traffic received on links with this mode classification will have the following conditions applied to them: d. 802.The 802.1P mode is useful when its necessary to handle the traffic on one EPG as untagged to the interface (similar to the switchport trunk native vlan vlan_ID command). The option On Demand instructs the leaf switch to program the EPG and its related policy in the hardware only when traffic matching this policy is received for this EPG. Create a physical domain and VLAN pool that are associated to this physical domain. i. e.1P headers. iii. The option Immediate means that the EPG configuration and its related policy configuration will be programmed in the hardware right away.

For the External Bridged Domain drop-down list. Create an attachable access entity profile (AEP) to map the interfaces and policies together. 2 In the Work pane. Associate the Layer 2 outside connection with the bridge domain and a VLAN ID. you will also need to se​ lect which traf​ fic you would like to tra​ verse be​ tween the two EPGs. choose Action > Create Bridged Outside. This VLAN ID must be in the range of the VLAN pool that is used for the Layer 2 outside connection. See the Adding New De​ vices to the Fab​ ric sec​ tion for more in​ for​ ma​ tion on how to con​ fig​ ure an AEP and a phys​ ic ​al do​ main. this method will also allow the end​ points to share the same sub​ net and de​ fault gate​ way. To cre​ ate an ex​ ter​ nal Layer 2 do​ main: 1 On the menu bar. 3 In the Navigation pane. This new EPG will be part of the ex​ ist​ ing bridge do​ main. In the Create Bridged Outside dialog box. Extending an ACI Bridge Domain Outside of the Fabric A Layer 2 out​ side con​ nec​ tion is as​ so​ ci​ ated with a bridge do​ main and is de​ signed to ex​ tend the whole bridge do​ main. cre​ ate a new ex​ ter​ nal EPG to clas​ sify this ex​ ter​ nal traf​ fic. With two sep​ ar​ ate EPGs. choose Tenant_Name > Networking > External Bridged Network. choose Tenants > ALL TENANTS. 4 5 In the Work pane. perform the following actions: a. . not just an in​ di​ vid​ ual EPG under the bridge do​ main. Sim​ il​ ar to the pre​ vi​ ous ex​ am​ ple of adding an end​ point to a pre-ex​ ist​ ing EPG. Clas​ sify any out​ side con​ nec​ tions or end​ points into this new ex​ ter​ nal EPG.Fabric Connectivity 141 f. This Layer 2 outside connection will put this VLAN and the bridge domain of the ACI fabric under the same Layer 2 domain. create a Layer 2 domain if one does not already exist. To ac​ com​ plish an ex​ ten​ sion of the bridge do​ main to the out​ side. a Layer 2 out​ side con​ nec​ tion must be cre​ ated for the bridge do​ main. Dur​ ing this process. This VLAN must be configured on the external Layer 2 network. to the out​ side net​ work. choose the Tenant_Name. i.

and pro​ vides IP con​ nec​ tiv​ ity be​ tween a ten​ ant pri​ vate net​ work and an ex​ ter​ nal IP net​ work. as well as to the In​ ter​ net to pro​ vide ac​ cess to the mo​ bile ap​ pli​ ca​ tion. d. Add a Layer 2 border leaf node and Layer 2 interface for a Layer 2 outside connection. You should now have the de​ sired reach​ ab ​il​ ity be​ tween the in​ side and out​ side Layer 2 seg​ ments. This is a policy object that tells the APIC to allow certain encap (VLANs) on selected ports. Each Layer 3 ex​ ter​ nal con​ nec​ tion is as​ so​ ci​ ated with one ten​ ant . which gen​ er​ ally does not di​ rectly at​ tach to the fab​ ric. For more information on how to create a Access Attachable Entity Profile. This helps avoid any overlapping in the VLAN range between VLANs used for an EPG and those in use for a Layer 2 outside connection. After adding a Layer 2 border leaf and Layer 2 interface. e. create a VLAN pool to associate to the VLAN on the Layer 2 outside connection. This is a means to specify the range of the VLAN IDs that will be used for creating a Layer 2 outside connection. and there​ fore there must be con​ nected to the ex​ ter​ nal net​ work. Create an AEP. ACME must be able to con​ nect to both their in​ ter​ nal cor​ po​ rate back​ bone. Configure a contract to allow communication between your existing endpoints in the existing EPG and your new external Layer 2 EPG. the communication between this external Layer 2 EPG and your existing internal EPG will be allowed. see the Adding New Devices to the Fabric section. c. This in​ te​ gra​ tion is pos​ si​ ble with Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) at the ten​ ant pol​ icy level. In the Navigation pane. b.142 Fabric Connectivity ii. After specifying this contract as the provided contract for your internal EPG. if it does not already exist. choose External Bridged Networks > Networks and specify a contract to govern this policy as the consumed contract. Extending ACI to External Layer 3 The most im​ por​ tant com​ po​ nent of any ap​ pli​ ca​ tion is the user or cus​ tomer. Layer 3 con​ nec​ tiv​ ity to a de​ vice like a router is known as an Ex​ ter​ nal Routed Net​ work. click Next to start creating a Layer 2 EPG. While creating the Layer 2 domain. Simply provide a name for the Layer 2 EPG. All of the traffic entering the ACI fabric with the designated VLAN (the VLAN ID provided in step 1) will be classified into this Layer 2 EPG.

Any ACI leaf can be​ come a bor​ der leaf. These ap​ pli​ ca​ tion com​ po​ nents might have re​ quire​ ments for ex​ ter​ nal con​ nec​ tiv​ ity into them. Other de​ vices. The re​ quire​ ment of the Layer 3 ex​ ter​ nal net​ work is only needed when a group of de​ vices in the ap​ pli​ ca​ tion pro​ file re​ quire Layer 3 con​ nec​ tiv​ ity to a net​ work out​ side of the ACI fab​ ric. As the op​ er​ a​ tor of the fab​ ric. The fol​ low​ ing fig​ ure shows part of a sim​ pli​ fied fab​ ric: A sample application profile for a three-tiered application with contracts between the tiers For ex​ am​ ple. web servers need a con​ nec​ tion to the out​ side world for users to reach them. of an ap​ pli​ ca​ tion into end​ point groups (EPGs). It is im​ por​ tant to note that the spine nodes can​ not have con​ nec​ tions to ex​ ter​ nal routers. can still con​ nect to the bor​ der leaves.Q tagging 3 Switched Virtual Interface (SVI) . the ex​ ter​ nal Layer 3 con​ nec​ tion can be one of the fol​ low​ ing types: 1 Physical Layer 3 interface 2 Subinterface with 8021. With ACI. you can pro​ vide the ten​ ant ad​ min​ is​ tra​ tor with the abil​ ity to in​ ter​ face to an ex​ ter​ nal Layer 3 con​ nec​ tion in var​ i​ ous ways by using a uniquely-de​ fined Layer 3 con​ struct for the ten​ ant ap​ pli​ ca​ tion pro​ file or a shared com​ mon in​ fra​ struc​ ture. like servers. A bor​ der leaf is sim​ ply ter​ mi​ nol​ ogy to refer to a leaf that hap​ pens to be con​ nected to a Layer 3 de​ vice. or tiers. In large scale ACI de​ signs it might be pro​ duc​ tive to have ded​ i​ cated ACI leafs as bor​ der leafs. In the ACI fab​ ric. An ap​ pli​ ca​ tion pro​ file en​ ables an op​ er​ a​ tor to group the dif​ fer​ ent com​ po​ nents.Fabric Connectivity 143 pri​ vate net​ work. the con​ nec​ tiv​ ity is de​ fined by a con​ tract to a de​ fined ex​ ter​ nal Layer 3 end​ point group. Ex​ ter​ nal Layer 3 con​ nec​ tions are usu​ ally es​ tab​ lished on the bor​ der leaf con​ struct of the ACI.

• Private . the ex​ ter​ nal net​ work has learned of the in​ ter​ nal ACI net​ work 10. The net​ work team will pro​ vide the ex​ ter​ nal Layer 3 con​ nec​ tiv​ ity for their ten​ ants. and routes learned from one ex​ ter​ nal Layer 3 con​ nec​ tion can be ad​ ver​ tised out to a dif​ fer​ ent ex​ ter​ nal Layer 3 con​ nec​ tion.Indicates that this subnet will be contained only within the ACI fabric private network. The fol​ low​ ing fig​ ure de​ picts the logic of pub​ lic and pri​ vate net​ works: Application profile with external consumers. For the pri​ vate net​ works.​ 1. pub​ lic.​ 0/​ 24. . the fab​ ric only ad​ ver​ tises sub​ nets that are marked pub​ lic in the as​ so​ ci​ ated bridge do​ main. not just fab​ ric routes. In re​ leases prior to ver​ sion 1. ACI does not ad​ ver​ tise the net​ works through the rout​ ing pro​ to​ col to the ad​ ja​ cent Layer 3 router. • Shared . These sub​ nets can be clas​ si​ fied as pri​ vate. ACI is ca​ pa​ ble of act​ ing as a tran​ sit net​ work.​ 1.144 Fabric Connectivity Bridge do​ mains con​ tain one or more sub​ nets. public and private networks annotated With de​ vices con​ nect​ ing through the ex​ ter​ nal Layer 3 con​ nec​ tion. This be​ hav​ ior is known as a non-tran​ sit fab​ ric. or shared: • Public .1 of Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). One com​ mon mech​ a​ nism is to use sub-in​ ter​ faces on a router to cre​ ate dif​ fer​ ent Layer 3 do​ mains since each ten​ ant will likely not have their own ex​ ter​ nal router. as it is ad​ ver​ tised to the ad​ ja​ cent router through the Layer 3 ex​ ter​ nal con​ nec​ tion.1 and later.Indicates that this subnet will be advertised to the external router. Routes that are learned ex​ ter​ nally from the fab​ ric are not ad​ ver​ tised through other ports. and the net​ works are not reach​ able to de​ vices ex​ ter​ nal to the fab​ ric.Indicates that this subnet will be leaked to one or more private networks inside of the ACI fabric. In re​ lease ver​ sion 1.

Without MP-BGP. With static routes. con​ fig​ ure more than one spine as a router re​ flec​ tor node. Configure MP-BGP Spine Route Reflectors The ACI fab​ ric route re​ flec​ tors use mul​ ti​ pro​ to​ col bor​ der gate​ way pro​ to​ col (MP-BGP) to dis​ trib​ ute ex​ ter​ nal routes within the fab​ ric so a full mesh BGP topol​ ogy is not re​ quired. ad​ min​ is​ tra​ tors can setup con​ nec​ tiv​ ity to ex​ ter​ nal net​ works. or BGP) for the Layer 3 outside connections are not propagated within the ACI fabric. and use that AS number as the BGP AS number for the ACI fabric. the fab​ ric in​ fra​ struc​ ture op​ er​ at​ or must con​ fig​ ure a Route Re​ flec​ tor pol​ icy to des​ ig​ nate which spines act as the route re​ flec​ tor(s). the router learns only a summarization of routes. the user must find out the AS number on the router to which the ACI border leaf will connect. OSPF. With OSPF NSSA. the fab​ ric ad​ min​ is​ tra​ tor must se​ lect at least one spine switch that will be a route re​ flec​ tor. Given that the same AS number is used for both cases. the external routes (static. To en​ able route re​ flec​ tors in the ACI fab​ ric. and the ACI leaves that are not part of the border leaf does not have IP connectivity to any outside networks. • iBGP peering leaf and external router—With internal Border Gateway Protocol (iBGP). • OSPF NSSA—Using not-so-stubby area (NSSA) reduces the size of the Open Shortest Path First (OSPF) database and the need to maintain the overhead of the routing protocols with large tables of routes. For re​ dun​ dancy pur​ poses.Fabric Connectivity 145 Supported Routing Protocols The fol​ low​ ing rout​ ing pro​ to​ cols are sup​ ported at time of writ​ ing: • Static routes—You can define static routes to the outside world. To con​ nect ex​ ter​ nal Layer 3 de​ vices to the ACI fab​ ric. Using static routes reduces the sizing and complexity of the routing tables in the leaf nodes. and pro​ vide the au​ tonomous sys​ tem (AS) num​ ber for the fab​ ric. Once route re​ flec​ tors are con​ fig​ ured. ACI supports only one autonomous system (AS) number that has to match the one that is used for the internal Multiprotocol Border Gateway Protocol (MP-BGP) route reflector. including a default path out of the fabric. . you must also configure the static path back to the interanal network that you wish to be reachable from the outside world. but increases administrator overhead. OSPF NSSA advertises to the adjacent router the internal public subnets part of the Layer 3 external.

in the Date Time Policy dropdown list. In the BGP Route Reflector Policy drop-down list. perform the following actions: a. In the Create Pod Policy Group dialog box. Add the two spine nodes that will be members of this reflector policy and c. in the Fabric Policy Group drop-down list. choose Fabric > Fabric Policies. c. choose default. 7 Click Submit. To con​ fig​ ure the Route Re​ flec​ tor pol​ icy: 1 On the menu bar. In the BGP Route Reflector Policy drop-down list. d. which pairs the bor​ der leaf node with one of the route re​ flec​ tor nodes as a BGP peer. The in​ fra​ struc​ ture op​ er​ at​ or con​ fig​ ures each of the paired leaf nodes with the routes (or route pre​ fixes) that the nodes can ad​ ver​ tise. 6 In the Create Pod Policy Group dialog box.146 Fabric Connectivity When a ten​ ant re​ quires a Layer 3 con​ nec​ tion. e. perform the following actions: a. choose Pod Policies. choose Pod Policies > Policies > BGP Route Deflector default. the router should peer with mul​ ti​ ple leaf nodes. b. the in​ fra​ struc​ ture op​ er​ at​ or con​ fig​ ures the leaf node to which the WAN router is being con​ nected as bor​ der leaf node. choose Actions > Create Pod Policy Group. choose Pod Policies > Profiles > default. If a WAN router must ad​ ver​ tise more than 4000 routes. 4 In the Navigation pane. Click Submit. f. Each leaf node can store up to 4000 routes at time of writ​ ing. In the Work pane. . 2 In the Navigation pane. 3 In the Work pane. After the route re​ flec​ tors are con​ fig​ ured. b. In the Navigation pane. choose default. they can ad​ ver​ tise routes in the fab​ ric. choose default. Complete the remainder of the dialog box as appropriate to your setup. 5 In the Work pane. choose Create Pod Policy Group. Change the Autonomous System Number to match the required number for your network.

Fabric Connectivity 147 The fol​ low​ ing fig​ ure il​ lus​ trates the ob​ jects and their re​ la​ tion​ ships for ex​ ter​ nal Layer 3 con​ nec​ tions: Object relationships for Layer 3 outside objects Layer 3 Integration Through Tenant Network with OSPF NSSA The fol​ low​ ing fig​ ure shows a sim​ ple in​ te​ gra​ tion of a Layer 3 ex​ ter​ nal into ACI using OSPF: Logical topology for an external OSPF router communicating with two border leafs .

ii. 4. choose Tenants > ALL TENANTS. In the Nodes and Interfaces Protocol Profiles section. d. In the OSPF Area Control section.​ g. enter a name for the profile. b. In the Loopback Addresses section. perform the following actions: a.148 Fabric Connectivity The setup in​ cludes a sin​ gle router with two in​ ter​ faces con​ nected to leaf switches. Note: See the "Adding New De​ vices to The Fab​ ric" sec​ tion to setup the ac​ cess poli​ cies for the in​ ter​ faces of the leaves that are con​ nected to the router. click + to add a loopback address. iii. Click OK. Uncheck the Router ID as Loopback Address check box.1". enter a name for the profile. h.1". 2. click + to add a profile.0. In the Create Routed Outside dialog box. In the Create Node Profile dialog box. and click Update. choose Tenant_Name > Networking > External Routed Networks. choose Action > Create Routed Outside. 4 5 In the Work pane.254. enter the router's IP address as the ID.1. click the Send redistributed LSAs into NSSA area check box. such as "10. In the Name field. perform the following actions: i. perform the following actions: 1. In the Nodes section. choose the private network for this tenant. 6. In the Select Node dialog box. such as "10. In the OSPF Area Type section. To in​ te​ grate Layer 3 through a ten​ ant net​ work with OSPF/NSSA: 1 On the menu bar. In the OSPF Area ID field. such as Leaf-1. 3. e. In the Node ID drop-down list. Click the OSPF check box.254. Enter the loopback address. click + to add a node. f. 2 In the Work pane. . such as "1". choose a node. 5. In the Name field. click the NSSA Area radio button. choose the tenant. 3 In the Navigation pane. enter the OSPF area ID. In the Router ID field. c. In the Private Network drop-down list.

Click Next. In the Select Routed Interface dialog box. In the Network Type section. Click Submit. In the MTU (bytes) field. In the Name field. such as "1500" to match the example peering router. choose the interface on the leaf. such as "OSPFPoint2Point". 5. 3. In the Path drop-down list. In the Interfaces section. such as Point to Point. b. click + to create an OSPF interface profile.Fabric Connectivity 149 iv. 6. In the Create External Network dialog box.1/24". 4. such as e1/9 on Leaf-1.0. When defining the interaction with another OSPF router. choose Create OSPF Interface Policy.1. click + create an external network. d. enter the maximum MTU of the external network. This document does not explain the different OSPF parameters. such as "10. perform the following actions: . b. enter a name for the profile. In the Name field. k. In the Create OSPF Interface Policy dialog box. enter the IP address of the path that is attached to the layer 3 outside profile. Click OK. In the OSPF Interface Profiles section. perform the following actions: 1. click the radio button that matches the adjacent router. In the Create Interface Profile dialog box. you must specify the policy interaction. Complete the remainder of the dialog box as appropriate to your setup and click OK. In the External EPG Networks section. In the OSPF Policy drop-down list. v. vi. 2. d. 7. c. enter a name for the OSPF policy. i. click on the Routed Intefaces tab. Complete the remainder of the dialog box as appropriate to your setup. perform the following actions: a. Click OK. perform the following actions: a. In the IP Address field. c. Click the + sign to select a routed interface. j.

0. External Layer 3 for Multiple Tenants In ACI. enter 0. you must configure the external network EPG.150 Fabric Connectivity i. For routers ca​ pa​ ble of using sub-in​ ter​ faces.0. Click Finish. . Next. l. the ex​ ter​ nal Layer 3 con​ nec​ tion can be con​ fig​ ured to route via SVI. In the IP Address field.0/0 to permit the learning of any subnet and click OK. you can use var​ io ​us mech​ an ​isms to reuse the same ex​ ter​ nal Layer 3 router for mul​ ti​ ple ten​ ants. those can be used to pro​ vide mul​ ti​ ple ex​ ter​ nal Layer 3 con​ nec​ tion for mul​ ti​ ple VRFs and/or ten​ ants. If the ad​ ja​ cent router is a Cisco Nexus Se​ ries Switch with a Layer 2 trunk in​ ter​ face. The fab​ ric op​ er​ at​ or can con​ fig​ ure mul​ ti​ ple ex​ ter​ nal Layer 3 con​ nec​ tions using ei​ ther sub-in​ ter​ face or SVI and pro​ vide that to each ten​ ant.

can be found in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter of this book. They would like to mi​ grate with min​ im ​al ser​ vice in​ ter​ rup​ tion while tak​ ing ad​ van​ tage of ACI in​ no​ va​ tions where ap​ plic​ ab ​le. One key con​ sid​ er​ at​ ion should be when to switch over the SVI in​ ter​ faces from the ex​ ist​ ing en​ vi​ ron​ ment into the ACI fab​ ric and when to start ad​ ver​ tis​ ing routes to this SVI net​ work. you must start by con​ fig​ ur​ ing the Layer 2 out​ side net​ work to allow traf​ fic from the source VLAN to com​ mu​ ni​ cate with the same VLAN re​ sid​ ing on the ACI fab​ ric. servers or vir​ tu​ al​ iza​ tion hosts from out​ side the ACI fab​ ric.Fabric Connectivity 151 Application Migration Use Case When op​ er​ at​ ing the ACI fab​ ric. As ACME starts to use ACI in more of their data cen​ ters. As​ sum​ ing that the SVIs re​ side on the ex​ ter​ nal Layer 2 net​ work. ACME would like to per​ form the mi​ gra​ tion in mul​ ti​ ple stages. ACME must man​ age the mi​ gra​ tion of SVI in​ ter​ faces as well as the pol​ icy al​ low​ ing traf​ fic to tra​ verse a Layer 2 out​ side net​ work and then on to the ACI fab​ ric. Fur​ ther in​ for​ ma​ tion and steps on how to cre​ ate this Layer 2 and Layer 3 con​ nec​ tiv​ ity in​ clud​ ing the pol​ icy. onto the fab​ ric. At a high level. Extending the Network to ACI One of the ACME sites would like to mi​ grate from the legacy data cen​ ter ar​ chi​ tec​ ture to the next gen​ er​ at​ ion ACI Fab​ ric. In this ex​ am​ ple. you can then start the mi​ gra​ tion process of mov​ ing ap​ pli​ ca​ tion work​ loads onto the fab​ ric. Cisco rec​ om​ mends that you move the SVIs over to the ACI fab​ ric once a ma​ jor​ ity of the hosts have been mi​ grated over. Once they have suc​ cess​ fuly es​ tab​ lished con​ nec​ tiv​ ity be​ tween the out​ side Layer 2 net​ work (where the work​ load or host is com​ ing from) and the ex​ ist​ ing in​ ter​ nal fab​ ric EPG. there can be oc​ ca​ sions when you will have to mi​ grate work​ loads. You will also need to con​ fig​ ure Layer 3 con​ nec​ tiv​ ity from the fab​ ric out to the ex​ ist​ ing Layer 3 net​ works in prepa​ ra​ tion for full con​ nec​ tiv​ ity after SVI mi​ gra​ tion. One com​ mon ex​ am​ ple is when mi​ grat​ ing from a tra​ di​ tional data cen​ ter con​ fig​ ur​ a​ tion over to a pol​ icy-dri​ ven data cen​ ter using ACI. . it will be come nec​ es​ sary to per​ form these mi​ gra​ tions.

152 Fabric Connectivity The Legacy data cen​ ter prior to mi​ gra​ tion: Traditional pre-migration data center The ACI data cen​ ter fol​ low​ ing mi​ gra​ tion: Post-migration ACI based data center topology .

2 Provide a physical connection from aggregation switch #2 to the ACI border leaf #1. you log​ ic ​ally map a VLAN=EPG. In the APIC con​ troller you will now con​ fig​ ure a sin​ gle ten​ ant. For de​ tails on con​ fig​ ur​ ing the ac​ cess poli​ cies please ref​ er​ ence the Fab​ ric Con​ nec​ tivty sec​ tion of this book. choose the Tenant_Name. The in​ ter​ con​ nect from the legacy net​ work to the ACI fab​ ric will be ac​ com​ plished through stan​ dard Layer 2 ex​ ten​ sions (VLAN/VXLAN). In the Name field enter a name for the tenant. 1 2 On the menu bar. In the Create Tenant dialog box perform the following actions: a. or a sin​ gle in​ ter​ face. Con​ fig​ ure a sin​ gle pri​ vate net​ work. . In this state. This con​ nec​ tiv​ ity can be ac​ com​ plished in ei​ ther the form of a Vir​ tual Port Chan​ nel. 2 In the Work pane. b. Click Next. Note: Be​ fore con​ nect​ ing ex​ ter​ nal phys​ ic ​al con​ nec​ tions into the fab​ ric. 1 Provide a physical connection from aggregation switch #1 to the ACI border leaf #1. choose Tenants > ALL TENANTS. Con​ fig​ ure the ag​ gre​ ga​ tion links as a Layer 2 trunk. choose Tenants > Add Tenant. The cre​ ated ten​ ant will rep​ re​ sent the legacy data cen​ ter into the ACI fab​ ric.Fabric Connectivity 153 The first stage will pro​ vide con​ nec​ tiv​ ity from the legacy data cen​ ter to the ACI fab​ ric. the Fab​ ric Ac​ cess Poli​ cies for the ac​ cess ports that you will be used for the DCI must be con​ fig​ ured. 1 Trunk the VLAN representing the host connectivity. Pro​ vide phys​ ic ​al con​ nec​ tiv​ ity from the ex​ ist​ ing ag​ gre​ ga​ tion layer to the ACI bor​ der leafs. 3 Click Finish. Port Chan​ nel. 1 On the menu bar. This allows for the host VLAN to be extended into the fabric.

Click the ARP Flooing check box. Click Next. perform the following actions: a. b. 4 In the Work pane. 2 In the Work pane.154 Fabric Connectivity 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. 3 In the Navigation pane choose Application Profiles. click Flood. d. Note: The con​ cept of flood​ ing the un​ known uni​ cast and arp within the Fab​ ric is to allow for the Layer 2 se​ man​ tics from the legacy data cen​ ter to be ex​ tended into the ACI Fab​ ric. e. Click Submit. 5 In the Create Application Profile dialog box. choose Actions > Create Application Profile. choose the Tenant_Name. In the Forwarding drop-down list. choose the Tenant_Name. When a host in the legacy data cen​ ter sends an ARP re​ quest and/or floods an un​ known uni​ cast frame. . b. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name. For the Multi Destination Flooding radio buttons. c. choose Actions > Create Private Network. Con​ fig​ ure a sin​ gle ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. For the Layer 2 Unknown Unicast radio buttons. 5 In the Create Private Network dialog box. choose Tenants > ALL TENANTS 2 In the Work pane. click Flood in BD. By de​ fault BPDU frames are flooded within EPG. g. In the Name field enter a name for the Application Profile. 4 In the Work pane. the bridge do​ main will then mimic the be​ hav​ ior in the ACI Fab​ ric. In the Name field enter a name for the private network. choose Tenants > ALL TENANTS. In the Name field enter a name for the bridge domain. f. perform the following actions: a. Con​ fig​ ure a sin​ gle end​ point group: 1 On the menu bar. 6 Click Finish. choose Custom.

2 In the Work pane. b. the local VLAN num​ ber is not sig​ nif​ ic ​ant as it will be mapped to the tagged VLAN on the VPC to​ ward the Legacy data cen​ ter. Fol​ low​ ing the Stage 1 mi​ gra​ tion. Choose the Path Type. In the Bridge Domain field. In the Create Application EPG dialog box. perform the following actions: a.Fabric Connectivity 155 4 5 In the Work pane. From the ACI Fab​ ric per​ spec​ tive. Acme​ Out​ Side. c. See the "Fab​ ric Con​ nec​ tiv​ ity" sec​ tion. choose Tenants > ALL TENANTS. Click Submit. choose the Tenant_Name. 4 In the Work pane. Note: The EPG within the fab​ ric will map to a sin​ gle VLAN in the legacy data cen​ ter. choose the appropiate bridge domain. 5 In the Deploy Static EPG on PC. b. c. The use of a sin​ gle VLAN per EPG will pro​ vide a path for a net​ work-cen​ tric mi​ gra​ tion— min​ im ​iz​ ing im​ pact while in​ tro​ duc​ ing fab​ ric in​ no​ va​ tions. the Legacy host VLAN is now ex​ tended to the ACI fab​ ric and all hosts from the Fab​ ric point of view are in the EPG. The en​ cap​ su​ la​ tion VLAN should match the de​ fined legacy data cen​ ter VLAN de​ f​ in ​i​ t​ ion. d. Con​ fig​ ure a VPC for the con​ nec​ tiv​ ity to the legacy data cen​ ter. choose Actions > Deploy Static EPG on PC. VPC or Interface dialog box. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). This is what is known as nor​ mal​ iza​ tion where ACI uses the tagged vlan to map ex​ ter​ nal Layer 2 con​ nec​ tions into ACI End Point Groups. enter a name for the endpoint group. perform the following actions: a. Enter the encapsulation VLAN. choose Actions > Create Application EPG. con​ fig​ ure a sta​ tic trunk bind​ ing using the VPC under the EPG. Acme​ Out​ Side. 1 On the menu bar. Then. . Choose the Path. Click Finish. VPC or Interface. In the Name field.

Configure additional Tenants (optional). a. AEP. b. Configure additional Application Profiles (optional). see the "Fab​ ric Con​ nec​ tivty" sec​ tion. Note: En​ sure the ap​ pro​ pri​ ate re​ sources have been cre​ ated to sup​ port the con​ nec​ tiv​ ity. Create a static binding for each physical host to be migrated. mul​ ti​ ple EPGs will be cre​ ated with APIC con​ tracts that de​ fine reach​ a​ bil​ ity. i. Configure additional End Point Groups. Create the endpoint group "AcmeInSide". choose Tenant. e. choose Any-Any. Create a VMM domain to deploy the host within the ACI Fabric (optional). Configure contracts for inter-EPG communications. f. Configure additional Private Networks (optional).156 Fabric Connectivity The con​ nec​ tiv​ ity noted below rep​ re​ sents a BD=EPG=VLAN. and switch or in​ ter​ face pro​ files. i. b. ii. . "AcmeInSide". Configure additional Bridge Domains (optional). d. Dur​ ing this stage. For the Allow drop-down list. The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter or pro​ vided by the Legacy data cen​ ter. An EPG as a VLAN datacenter deployment The sec​ ond stage of the mi​ gra​ tion takes ad​ van​ tage of the fab​ ric pol​ icy model to de​ fine fur​ ther the ap​ pli​ ca​ tion mod​ el​ ing. For the Scope drop-down list. VLAN pools. c. The fol​ low​ ing list is an overview of the steps that you must per​ form: 1 In the APIC: a. For more in​ for​ ma​ tion. phys​ i​ cal do​ main or VMM Do​ main. 2 Begin migrating the host to the fabric EPG.

iv. choose AcmeBD. In the Flood Layer 2 drop-down list. Datacenter migration with Layer 3 provided by existing DC and Layer 2 extended to new ACI datacenter In the third stage of the mi​ gra​ tion the Layer 3 gate​ way will move from the legacy data cen​ ter to the ACI Fab​ ric. In the Unicast drop-down list. choose Flooding. the host con​ nec​ tiv​ ity across both the legacy data cen​ ter and ACI fab​ ric are now gov​ erned by the APIC pol​ icy (con​ tracts). b. i. Migrate the gateway from the legacy data center to the fabric. . The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter are pro​ vided by the ACI fab​ ric. ii. Note: The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter are pro​ vided by the legacy data cen​ ter. Configure a Layer 3 out. iii.Fabric Connectivity 157 Fol​ low​ ing the stage 2 mi​ gra​ tion. Note: The con​ cept of uni​ cast rout​ ing within the bridge do​ main al​ lows for the con​ fig​ u​ ra​ tion of the per​ va​ sive gate​ way across the fab​ ric. choose Routing. In the Bridge Domain drop-down list. choose Unknown Unicast. In the ARP drop-down list. The fol​ low​ ing list is an overview of the steps that you must per​ form: 1 In the APIC: a.

.

159 Tenants .

.

Tenants 161 Section Content • • ACI Tenancy Models Application Profile Application Profile Configuration Create a New Application Profile Modify Application Profile Remove Application Profile Verify Application Profile • Endpoint Group Endpoint Group Configuration Create a New Endpoint Group Modify Endpoint Group Remove Endpoint Group Verify Endpoint Group • Endpoint Verify Endpoint Group • Private Network Private Network Configuration Parameters Creating a New Private Network Modify Private Network Remove Private Network Verify Private Network .

162 Tenants • Bridge Domain Bridge Domain Configuration Parameters Create a new Bridge Domain Modify a Bridge Domain Remove a Bridge Domain Verify Bridge Domain • Tenant Networking Use Cases Common Private Network for All Tenants Multiple Private Networks with Intra-Tenant Communication Multiple Private Networks with Inter-Tenant Communication .

2 Separation of environments from a software development lifecycle perspective: Development. a ten​ ant is a unique cus​ tomer. This means dif​ fer​ ent things to dif​ fer​ ent peo​ ple (much like the term Cloud) based on their per​ spec​ tive. This in​ tro​ duces a war​ ranted level of cau​ tion around change con​ trol and ap​ pli​ ca​ tion im​ pact. mak​ ing a rout​ ing pro​ to​ col change on a router or Layer 3 switch could po​ ten​ tially af​ fect hun​ dreds of unique VLANs/sub​ nets. and so on. ACI has been de​ signed from the be​ gin​ ning to be “multi-ten​ ant”. In the case of a clas​ sic ser​ vice provider. and Production. Quality Assurance. In tra​ di​ tional net​ work​ ing en​ vi​ ron​ ments. They will be using ten​ ant con​ structs for the ap​ pli​ ca​ tion life​ cy​ cle of their cur​ rent de​ ploy​ ment. a ten​ ant that will be used for the au​ to​ mated test​ ing.Tenants 163 ACI Tenancy Models ACME Inc. ap​ pli​ ca​ tion owner. The in​ for​ ma​ tion se​ cu​ rity team will be able to in​ te​ grate this into the cor​ po​ rate LDAP sys​ tem. as men​ tioned in the in​ tro​ duc​ tion. The ten​ ant ob​ ject gives us the abil​ ity to draw a box around the log​ ic ​al and con​ crete ob​ jects that we use to pro​ vide a uni​ fied view of the con​ fig​ ur​ a​ tion de​ pen​ den​ cies for un​ der​ lay and over​ lay net​ works. main​ tain​ ing a sep​ a​ rate ten​ ant for the re​ sources that de​ vel​ op​ ers will be using to build the ap​ pli​ ca​ tion. . and fi​ nally a pro​ duc​ tion ten​ ant. busi​ ness unit. and database owners. security. will be using ten​ ancy for a cou​ ple of use cases. such as web. etc. networking. while in a typ​ ic ​al endcus​ tomer en​ vi​ ron​ ment a ten​ ant could be an op​ er​ at​ ing group. 3 Separation of duties by domain owner. app. they are also look​ ing to build an in​ fra​ struc​ ture which can be lever​ aged for sim​ il​ ar ini​ tia​ tives in the fu​ ture. The de​ ci​ sion on how to lever​ age ten​ ancy mod​ els is dri​ ven by a num​ ber of fac​ tors: 1 Overall IT operations and support models in your organization to manage application. servers. Ten​ ants will be used to draw vir​ tual bound​ aries for dif​ fer​ ent lines of busi​ ness. such as different business units. Lever​ ag​ ing the ACI pol​ icy model. the phys​ ic ​al hard​ ware is ab​ stracted from the log​ ic ​al con​ structs. and pre​ vent changes which would im​ pact other groups. Ad​ di​ tion​ ally. 4 Fault domain size and scope to limit the impact of failures.

Each server is re​ ferred to as an End​ point in ACI. There​ fore. In ACI. adds each com​ po​ nent. the ten​ ant poli​ cies are where you de​ fine ap​ pli​ ca​ tions. and the ob​ jects that de​ fine the ten​ ant poli​ cies such as Ap​ pli​ ca​ tion Pro​ files and End​ point Groups. The ten​ ant net​ work​ ing is used to de​ fine net​ work​ ing poli​ cies and will be ap​ plied to the un​ der​ ly​ ing hard​ ware in a trans​ par​ ent way thanks to the layer of ab​ strac​ tion pro​ vided by ACI using pri​ vate net​ works. End​ points are clas​ si​ fied in ACI to apply poli​ cies.164 Tenants A Ten​ ant in the ACI Ob​ ject model rep​ re​ sents the high​ est-level ob​ ject. a web​ site could use a 3-tier ap​ pli​ ca​ tion model. Al​ though the ten​ ant net​ work​ ing and the ten​ ant poli​ cies are de​ fined sep​ ar​ ately. When a user browses the web site. the ap​ pli​ ca​ tion servers com​ mu​ ni​ cate with the data​ base which could also be a clus​ ter of servers. the net​ work​ ing poli​ cies used by an ap​ pli​ ca​ tion are de​ fined with a re​ la​ tion​ ship be​ tween the End​ point Groups and the Bridge Do​ main. Fi​ nally. You cre​ ate end​ point groups with end​ points that share the same type of poli​ cies. Bridge Do​ mains and sub​ nets. ap​ pli​ ca​ tion servers and data​ base servers. The fol​ low​ ing image shows all of the com​ po​ nents that can be con​ fig​ ured within a ten​ ant. they might ac​ tu​ ally be com​ mu​ ni​ cat​ ing with a vir​ tual IP ad​ dress on a load bal​ ancer that in turn can dis​ trib​ ute the web re​ quest to a num​ ber of dif​ fer​ ent web servers. such as Pri​ vate Net​ works. In​ side. For ex​ am​ ple. An ap​ pli​ ca​ tion could con​ sist of a com​ bi​ na​ tion of phys​ ic ​al servers or VMs that we will call servers from now on. an ap​ pli​ ca​ tion can be formed by sev​ eral end​ point groups and they are grouped in an Ap​ pli​ ca​ tion Pro​ file. In the next sec​ tions of this chap​ ter these con​ cepts will be cov​ ered in de​ tail. The web servers in turn com​ mu​ ni​ cate with core ap​ pli​ ca​ tions that can be di​ vided amongst sev​ eral ap​ pli​ ca​ tions servers for load bal​ anc​ ing or high avail​ abil​ ity pur​ poses. bridge do​ mains and sub​ nets. you can dif​ fer​ en​ ti​ ate be​ tween the ob​ jects that de​ fine the ten​ ant net​ work​ ing. Below you can find an il​ lus​ tra​ tion with the dif​ fer​ ent ob​ jects that com​ pound a ten​ ant and how they are re​ lated. . such as with whom are they going to com​ mu​ ni​ cate and what type of com​ mu​ ni​ ca​ tion or re​ stric​ tions are re​ quired. com​ prised of web servers. In the fol​ low​ ing sec​ tions each di​ ag ​ram shows the progress of how ACME Inc.

Tenants 165

Tenant Logical Model
There are 3 Ten​
ants pre​
con​
fig​
ured in the sys​
tem by de​
faut:
1

Common – a special tenant with the purpose of providing “common” services to
other tenants in the ACI fabric. Global reuse is a core principle in the common
tenant. Some examples of common services are:
a. Shared Private Networks
b. Shared Bridge Domains
c. DNS
d. DHCP
e. Active Directory

2

Infra – The Infrastructure tenant that is used for all internal fabric
communications, such as tunnels and policy deployment. This includes switch
to switch (Leaf, Spine, Application Virtual Switch (AVS)) and switch to APIC.
The infra tenant does not get exposed to the user space (tenants) and it has its
own private network space and bridge domains. Fabric discovery, image
management, and DHCP for fabric functions are all handled within this tenant.

3

Mgmt - The management tenant provides convenient means to configure
access policies for fabric nodes. While fabric nodes are accessible and
configurable through the APIC, they can also be accessed directly using in-band
and out-of band connections. In-band and out-of-band policies are configured
under the mgmt tenant:

In-Band Management Access

Out-of-Band Management Access

Tenants 167

Application Profile
An ap​
pli​
ca​
tion pro​
file is a con​
ve​
nient log​
i​
cal con​
tainer for mul​
ti​
ple hosts (phys​
i​
cal or
vir​
tual). You can cre​
ate Ap​
pli​
ca​
tion Pro​
file con​
tain​
ers based on a va​
ri​
ety of cri​
te​
ria,
such as what func​
tion the ap​
pli​
ca​
tion pro​
vides, how the ap​
pli​
ca​
tion looks from the
end-user per​
spec​
tive, where they are lo​
cated within the con​
text of the data cen​
ter, or
any other log​
i​
cal group​
ing rel​
a​
tive to the im​
ple​
men​
ta​
tion. Ap​
pli​
ca​
tion Pro​
file servers
are grouped in EPGs de​
pend​
ing on the use of com​
mon poli​
cies.
Ap​
pli​
ca​
tion Pro​
files pro​
vide a mech​
a​
nism to un​
der​
stand groups of servers as a sin​
gle
ap​
pli​
ca​
tion. This ap​
proach makes an ACI ap​
pli​
ca​
tion aware and al​
lows us to check the
op​
er​
a​
tional state for an ap​
pli​
ca​
tion mon​
i​
tor​
ing all the servers that are part of an ap​
pli​
ca​
tion as a whole and be​
come in​
formed about rel​
e​
vant faults and health sta​
tus for that
par​
tic​
u​
lar ap​
pli​
ca​
tion. Each Ap​
pli​
ca​
tion Pro​
file cre​
ated can have a unique mon​
i​
tor​
ing
pol​
icy and QOS pol​
icy.
An Ap​
pli​
ca​
tion Pro​
file is a child ob​
ject of the Ten​
ant and a sin​
gle Ten​
ant can con​
tain
mul​
ti​
ple Ap​
pli​
ca​
tion Pro​
files.

Adding components to a Tenant - 1. Application Profile

168 Tenants

Application Profile Configuration
Name - The name of the ap​
pli​
ca​
tion pro​
file.
Tags - A tag or meta​
data is a non-hi​
er​
ar​
chi​
cal key​
word or term as​
signed to the fab​
ric
mod​
ule.
Mon​
i​
tor​
ing Pol​
icy - The mon​
it​
or​
ing pol​
icy name for the EPG se​
man​
tic scope (op​
tional).

Create a New Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles.

4

In the Work pane, choose Actions > Create Application Profile.

5

In the Create Application Profile dialog box, perform the following actions:
a. Enter an Application Profile Name.
b. Enter a TAG (optional).
c. Choose a Monitoring Policy (optional).

6

Click Submit.

Modify Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles >
Application_Profile.

4
5

In the Work pane, choose policy.
In the Create Application Profile dialog box, perform the following actions:
a. Enter an Application Profile Name.
b. Enter an appropriate TAG (optional).
c. Choose the Monitoring Policy (optional).

6

Click Submit.

Tenants 169

Remove Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles
> Application_Profile.

4

In the Work pane, choose Actions > Delete.

Verify Application Profile
REST :: /api/node/class/fvAp.xml
CLI :: moquery -c fvAp

Tenants 171

Endpoint Group
End​
point Groups are used to cre​
ate log​
i​
cal group​
ings of hosts or servers that per​
form
sim​
i​
lar func​
tions within the fab​
ric. Each End​
point Group cre​
ated can have a unique
mon​
i​
tor​
ing pol​
icy and QoS pol​
icy and must be as​
so​
ci​
ated with a Bridge Do​
main.
An End​
point group is a child ob​
ject of the Ap​
pli​
ca​
tion Pro​
file and an Ap​
pli​
ca​
tion Pro​
file can con​
tain mul​
ti​
ple End​
point Groups. Each end​
point within an End​
point Group is
sus​
cep​
ti​
ble to the same pol​
icy in the Fab​
ric.

Adding components to a tenant - 2. End Point Group in the Application Profile
All the End​
points in​
side an EPG can com​
mu​
ni​
cate with each other. How com​
mu​
ni​
ca​
tions be​
tween EPGs con​
tracts will be re​
quired is gov​
erned by con​
tracts and not tra​
di​
tional Layer 2/Layer 3 for​
ward​
ing con​
structs. For ex​
am​
ple, Host-A in EPG-A can have
the IP ad​
dress/mask of 10.​
1.​
1.​
10/​
24 and Host B in EPG B can have the IP ad​
dress/mask 10.​
1.​
1.​
20/​
24 (note that both hosts be​
lieve they are "in the same sub​
net"). In
this case they would not be al​
lowed to com​
mu​
ni​
cate un​
less a con​
tract that per​
mit​
ted
con​
nec​
tiv​
ity ex​
isted be​
tween EPG-A and EPG-B. Con​
tracts will be ex​
plained in greater
de​
tail in a fol​
low​
ing sec​
tion.
Note that there are some types of End​
point Groups within the fab​
ric that are not con​
tained under Ap​
pli​
ca​
tion Pro​
files such as, Ap​
pli​
ca​
tion End​
point Group, Ex​
ter​
nal Bridge
Net​
works (aka Lay​
er2 Ex​
ter​
nal), Ex​
ter​
nal Routed Net​
works (aka as Lay​
er3 Ex​
ter​
nal) and
Man​
age​
ment End​
point Groups. These End​
point Groups might have spe​
cial re​
quire​
-

172 Tenants

ments, for ex​
am​
ple, in Ex​
ter​
nal Bridge Net​
works, MAC ad​
dresses of the end​
points are
not learnt by the leaf switches.
End​
point Groups are linked to Bridge Do​
mains but they will re​
ceive a VLAN ID dif​
fer​
ent
from the bridge do​
main, un​
less Bridge Do​
main legacy mode is used.
It is im​
por​
tant to un​
der​
stand that a sin​
gle sub​
net can be ex​
tended across sev​
eral EPGs.
Each EPG is iden​
ti​
fied by an en​
cap​
su​
la​
tion VLAN or VXLAN so that the same sub​
net will
be using dif​
fer​
ent en​
cap​
su​
la​
tion IDs across the fab​
ric. This con​
cept is dif​
fer​
ent from
tra​
di​
tional net​
work​
ing.

Endpoint Group Configuration
Name - The name for the end​
point group.
Tag - A tag or meta​
data is a non-hi​
er​
ar​
chi​
cal key​
word or term as​
signed to the fab​
ric
mod​
ule.
Qos Class - The QoS pri​
or​
ity class iden​
ti​
fier.
The class can be:

Unspecified

Level1-Class 1 Differentiated Services Code Point (DSCP) value.

Level2-Class 2 DSCP value.

Level3-Class 3 DSCP value.

Cus​
tom Qos - The QoS traf​
fic pri​
or​
ity class iden​
ti​
fier. The Cus​
tom class is a user-con​
fig​
urable DSCP value.
Bridge Do​
main - The name of the bridge do​
main as​
so​
ci​
ated with this ob​
ject.
it​
or​
ing pol​
icy name for the EPG se​
man​
tic scope (op​
tional).
Mon​
it​
or​
ing Pol​
icy – The mon​
As​
so​
ci​
ated Do​
main Pro​
file – A source re​
la​
tion to an in​
fra​
struc​
ture do​
main pro​
file as​
so​
ci​
ated with ap​
pli​
ca​
tion end​
point groups.

Tenants 173

Sub​
net - An End​
point Group sub​
net is rel​
e​
vant when and only when con​
fig​
ur​
ing route
leak​
ing be​
tween VRF's/Pri​
vate Net​
work within a Ten​
ant (op​
tional).
Sta​
tic End​
point - The sta​
tic client end​
point rep​
re​
sents a silent client end​
point at​
tached
to the net​
work (op​
tional).

Create a New Endpoint Group
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles >
Application_Profile > Application EPGs.

4
5

In the Work pane, choose Actions > Create Application EPG.
In the Create Application EPG dialog box, perform the following.
a. Enter an Application EPG Name.
b. Enter an Tag (optional).
c. Enter an Qos Class (optional).
d. Enter an Custom Qos (optional).
e. Enter a Bridge Domain Name.
f. Choose a Monitoring Policy (optional).
g. Enter an Associated Domain Profile Name.

6

Click Finish.

Modify Endpoint Group
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles
> Application_Profile > Application EPGs > Application_EPG.

4

In the Work pane, select policy.
a. Enter an Application EPG Name.
b. Enter an Tag (optional).
c. Enter an Qos Class (optional).
d. Enter an Custom Qos (optional).

3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile > Application EPGs > Application_EPG. choose the Tenant_Name. Verify Endpoint Group REST :: /api/node/class/fvAEPg. g. 2 In the Work pane.. Enter an Associated Domain Profile Name.xml CLI :: moquery -c fvAEPg . f. choose Actions > Delete. Choose the appropriate Monitoring Policy if applicable (optional). Enter a Bridge Domain Name. choose Tenants > ALL TENANTS. 4 In the Work pane.174 Tenants e. Remove Endpoint Group 1 On the menu bar. 5 Click Finish.

c. a lo​ ca​ tion. Choose the Path Type. Click Submit. VPC or Interface. choose Actions > Deploy Static EPG on PC. 2 In the Work pane. Below is an ex​ am​ ple of a sta​ tic bind​ ing. and can be ei​ ther vir​ tual or phys​ ic ​al. Enter the encapsulation VLAN. and a de​ ploy​ ment Im​ me​ di​ acy mode as​ so​ ci​ ated with it. In order to show the end​ points that are con​ nected to the fab​ ric under cer​ tain EPGs: 1 On the menu bar. VPC or Interface dialog box. See the VVM sec​ tion for an ex​ am​ ple of a dy​ namic bind​ ing.Tenants 175 Endpoint End​ points are de​ vices that are con​ nected to the net​ work ei​ ther di​ rectly or in​ di​ rectly. choose the Tenant_Name. An End​ point is a child ob​ ject of the End​ point Group and an End​ point Group con​ struct can con​ tain mul​ ti​ ple End​ points. choose Tenants > ALL TENANTS. 1 On the menu bar. d. 4 In the Work pane. perform the following actions: a. and at​ trib​ utes. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). 2 In the Work pane. Choose the Path. an en​ cap​ su​ la​ tion. choose the Tenant_Name. The End​ points ref​ er​ enced within the fab​ ric can be ei​ ther sta​ tic (de​ fined within the APIC) or dy​ namic (au​ to​ mated by vCen​ ter/Open​ stack). choose Tenants > ALL TENANTS. Each end​ point has a path. b. End​ points have an ad​ dress (iden​ tity). 5 In the Deploy Static EPG on PC. You can add Sta​ tic End​ points by cre​ at​ ing Sta​ tic Bind​ ings within the End​ point Group. .

choose Operational.xml CLI :: moquery -c fvCEp . 4 In the Work pane. Verify Endpoint REST :: /api/node/class/fvCEp.176 Tenants 3 In the Navigation Pane choose Tenant_Name > Application Profiles > Application_Profile > Application EPGs > Application_EPG.

When there is a re​ quire​ ment to route traf​ fic be​ tween sep​ a​ rate Pri​ vate Net​ work in​ stances. see the overview sec​ tion of this chap​ ter. Private Network as part of the Tenant Logical Model The most com​ mon method to share Pri​ vate Net​ works be​ tween ten​ ants is through the com​ mon ten​ ant. Pri​ vate Net​ works cre​ ated in the com​ mon ten​ ant are shared glob​ ally within the fab​ ric. This will be dis​ cussed in de​ tail in the Bridge Do​ main and EPG con​ fig​ u​ ra​ tion sec​ tions. Pri​ vate net​ works are a child of the Ten​ ant ob​ ject. a Pri​ vate Net​ work that is in​ tended to be used by mul​ ti​ ple ten​ ants and is not cre​ ated in the com​ mon ten​ ant re​ quires ex​ plicit con​ fig​ u​ ra​ tion to be shared. For more in​ for​ ma​ tion about com​ mon ten​ ants.3. Adding components to a Tenant . How​ ever. It is a unique Layer 3 for​ ward​ ing and ap​ pli​ ca​ tion pol​ icy do​ main. . or con​ text. One or more bridge do​ mains are as​ so​ ci​ ated with a pri​ vate net​ work. pri​ vate Layer 3 net​ work. All of the end​ points within the Pri​ vate Net​ work must have unique IP ad​ dresses be​ cause it is pos​ si​ ble to for​ ward pack​ ets di​ rectly be​ tween these de​ vices if the pol​ icy al​ lows it.Tenants 177 Private Network A Pri​ vate Net​ work is also re​ ferred to as a VRF. spe​ cial con​ sid​ er​ a​ tion for sub​ net con​ fig​ u​ ra​ tion is needed.

d.Name of the OSPF timers pol​ icy as​ so​ ci​ ated with this ob​ ject. g. Choose the appropriate Monitoring Policy if applicable (optional). Enter an Private Network Name. End Point Re​ ten​ tion Pol​ icy . Creating a New Private Network 1 On the menu bar. Choose a Policy Enforcement (optional). When en​ traf​ fic. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. Pol​ icy En​ force​ ment .The name of the Pri​ vate Net​ work. c. Choose a BGP Policy Name (optional). Choose an OSPF Policy Name (optional). Un​ en​ forced al​ lows all traf​ fic within the Pri​ vate Net​ work. con​ tracts be​ tween EPGs are re​ quired to allow en​ forced. 2 In the Work pane. . icy as​ so​ ci​ ated with this ob​ ject. In the Create Private Network dialog box.The pre​ ferred pol​ icy con​ trol. choose Tenants > ALL TENANTS. e. perform the following actions: a. Mon​ i​ tor​ ing Pol​ icy .178 Tenants Private Network Configuration Parameters Name . The de​ fault is en​ forced. The val​ ues can be en​ forced or un​ forced is cho​ sen. i​ tor​ ing pol​ icy name for the Ten​ ant se​ man​ tic scope (op​ tional). 4 5 In the Work pane. choose the Tenant_Name.The net​ work do​ main name label. BGP Timers .The mon​ DNS Label . f. 6 Click Finish. b.The end point re​ ten​ tion pol​ icy name (op​ tional). La​ bels en​ able clas​ si​ fy​ ing which ob​ jects can and can​ not com​ mu​ ni​ cate with one an​ other (op​ tional). Choose the appropriate DNS Label if applicable (optional). choose Actions > Create Private Network. Choose an End Point Retention Policy Name (optional).Name of the BGP timers pol​ OSPF Timers .

3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network. choose the Tenant_Name. c. choose Tenants > ALL TENANTS. Choose the appropriate DNS Label if applicable (optional). 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network. g. Verify Private Network REST :: /api/node/class/fvCtx. select policy.xml CLI :: moquery -c fvCtx . e. 4 In the Work pane. Choose an End Point Retention Policy Name (optional). 2 In the Work pane. 5 Click Finish. choose the Tenant_Name. Choose a BGP Policy Name (optional). a. 2 In the Work pane. f. choose Actions > Delete. Choose an OSPF Policy Name (optional). 4 In the Work pane.Tenants 179 Modify Private Network 1 On the menu bar. Remove Private Network 1 On the menu bar. b. Choosesa Policy Enforcement (optional). d. Choose an EIGRP Policy Name (optional). choose Tenants > ALL TENANTS. Choose the appropriate Monitoring Policy if applicable (optional).

.

A Bridge Do​ main is a child of the Ten​ ant ob​ ject and must be linked to a Pri​ vate Net​ work. a sub​ net is con​ tained within a sin​ gle bridge do​ main. The bridge do​ main de​ fines the unique Layer 2 MAC ad​ dress space and a Layer 2 flood do​ main if flood​ ing is en​ abled. A bridge do​ main can have mul​ ti​ ple sub​ nets. .Tenants 181 Bridge Domain A Bridge Do​ main is the ab​ stract rep​ re​ sen​ ta​ tion of a Layer 2 for​ ward​ ing do​ main within the fab​ ric. Bridge do​ mains will span all switches in which as​ so​ ci​ ated EPG are con​ fig​ ured. Bridge Domain as part of the Tenant Application Profile The fol​ low​ ing image pro​ vides an ex​ am​ ple of a ten​ ant to show how bridge do​ mains are con​ tained in​ side of Pri​ vate Net​ works and how they are linked to EPGs and the other el​ e​ ments. Those sub​ nets will be spread across one or more bridge do​ mains con​ tained in the Pri​ vate Net​ work. that ad​ dress space can con​ sist of mul​ ti​ ple sub​ nets. How​ ever. Adding components to a Tenant .4. While a Pri​ vate Net​ work de​ fines a unique IP ad​ dress space.

al​ though it can act sim​ i​ lar to a VLAN. the bridge do​ main Acme-Ap​ pli​ ca​ tions-BD was as​ signed the PI VLAN ID 42 in the Leaf-1. Each Bridge Do​ main re​ ceives a VLAN ID in the leaf switches. Each PI VLAN is then linked to a VXLAN ID that will be used for for​ ward​ ing pur​ poses in​ side of the fab​ ric. can be trans​ lated lo​ cally as a VLAN with local sig​ nif​ i​ cance. under the Ten​ ant Acme. This VLAN con​ cept is dif​ fer​ ent from tra​ di​ tional net​ work​ ing and it is not used to for​ ward traf​ fic but as an iden​ ti​ fier. . on a leaf. each Bridge Do​ main will exist in a par​ tic​ u​ lar leaf if there is a con​ nected end​ point that be​ longs to that EPG. The VLAN ID used is also called Plat​ form In​ de​ pen​ dent VLAN or PI VLAN. From a prac​ ti​ cal per​ spec​ tive. This VLAN ID is sig​ nif​ i​ cant lo​ cally in the leaf switches and there​ fore it might be dif​ fer from one to other leaf switch. You in​ stead should think of it as a dis​ trib​ uted switch.182 Tenants End Point Group as part of the Tenant Application Profile It is im​ por​ tant to un​ der​ stand that a bridge do​ main is NOT a VLAN. In the fol​ low​ ing ex​ am​ ple. which.

This means that an SVI will be con​ fig​ ured under the VRF that rep​ re​ sents the pri​ vate . the de​ fault gate​ way for the end​ points will al​ ways be the first switch of the fab​ ric that is reached. Bridge do​ main in legacy mode is not rec​ om​ mended under ACI best prac​ tices. For this sit​ u​ a​ tion. In some sit​ u​ a​ tions where tra​ di​ tional net​ work​ ing is re​ quired. there is a fea​ ture called Bridge do​ main legacy mode. EPGs con​ tained in bridge do​ mains in legacy mode can​ not com​ mu​ ni​ cate with EPGs in ACI bridge do​ mains. This VLAN ID is dif​ fer​ ent from the Bridge Do​ main. There​ fore in ACI. also know as a per​ va​ sive gate​ way. sev​ eral VLANs will be used for EPs in​ side on one bridge do​ main. the leaf switches will be the de​ fault gate​ way for the EPGs using that sub​ net. each leaf will con​ fig​ ure the de​ fault gate​ way.Tenants 183 VLAN output from Leaf node EPGs are also as​ signed with a PI VLAN ID that is lo​ cally sig​ nif​ i​ cant in each leaf. it might be re​ quired to have only one en​ cap​ su​ la​ tion VLAN per Bridge do​ main across all the leaf switches and EPGs that ref​ er​ ence that Bridge Do​ main. If the EPGs have end​ points on mul​ ti​ ple leaves. In that way. When a Sub​ net is de​ fined in a Bridge Do​ main. for ex​ am​ ple dur​ ing some mi​ gra​ tion processes. For more de​ tails refer to the EP sec​ tion in this chap​ ter. It is thought to use tra​ di​ tional net​ work​ ing within ACI but it lim​ its the ACI fea​ tures that can be used.

The for​ ward​ ing method for un​ known layer 2 des​ ti​ na​ tions. Net​ work . If a Bridge Do​ main has sev​ eral sub​ nets. Every BD by de​ fault takes the fab​ ric wide de​ fault mac ad​ dress. mul​ ti​ cast traf​ fic is lim​ ited to the sub​ set of VLAN in​ ter​ faces on which the hosts re​ side.The as​ so​ ci​ ated layer 3 con​ text. uni​ cast rout​ ing will be per​ formed on the tar​ get IP ad​ dress.The name of the Layer 3 out​ side in​ ter​ face as​ so​ ci​ ated with this ob​ ject. ing pol​ icy name. For​ ward​ ing . As​ so​ ci​ ated L3 Outs . L3 Out for Route Pro​ file .A prop​ ing is dis​ abled. erty to spec​ ify whether ARP flood​ ing is en​ abled.The MAC ad​ dress of the bridge do​ main (BD) or switched vir​ tual in​ ter​ face (SVI). Un​ known Mul​ ti​ cast Flood​ ing .Op​ ti​ mize/Cus​ tom L2 Un​ known Uni​ cast . De​ fault is Proxy. there will be only one SVI per Bridge Do​ main but it will use sec​ ondary IP ad​ dresses. . Uni​ cast rout​ ing is en​ abled by de​ fault but can be dis​ abled by click​ ing the check box. Bridge Domain Configuration Parameters Name .The node for​ ward​ ing pa​ ra​ me​ ter for un​ known Mul​ ti​ cast des​ ti​ na​ tions.The name of the Bridge Do​ main. ward​ ing method based on pre​ de​ fined for​ ward​ ing cri​ Uni​ cast Rout​ ing . Con​ fig BD MAC Ad​ dress . Flood​ ing is dis​ abled by de​ fault but can be en​ abled by click​ ing the check box. By ex​ am​ in​ ing (snoop​ ing) IGMP IGMP Snoop Pol​ icy .184 Tenants net​ work that the Bridge Do​ main is linked to.The Layer 3 out​ side in​ ter​ face iden​ ti​ fier con​ trol​ ling con​ nec​ tiv​ ity to out​ side net​ works.The IGMP Snoop​ mem​ ber​ ship re​ port mes​ sages from in​ ter​ ested hosts. If flood​ ARP Flood​ ing .The for​ te​ ria (IP or MAC ad​ dress).

The net​ work vis​ ib ​il​ ity of the sub​ net. . Mon​ i​ tor​ ing Pol​ icy . to share with Layer 3 outbound. to only be used in that Tenant (will not be leaked). with the Public option configured. with the Private option configured. perform the following actions: a. 4 In the Work pane. c.Defines subnets under a bridge domain.Defines subnets under a bridge domain.The net​ work do​ main name label. Sub​ nets . Choose the Forwarding Semantics (optional).The default is Unspecified. The scope can be: • Shared . • Querier IP . DHCP La​ bels . The sub​ net is a por​ tion of a net​ work shar​ ing a par​ tic​ ul​ ar sub​ net ad​ dress. choose Tenants > ALL TENANTS. The default is Private.The mon​ it​ or​ ing pol​ icy name for the Ten​ ant se​ man​ tic scope (op​ tional). with the Shared option • Public . choose Actions > Create Bridge Domain. Create a new Bridge Domain 1 On the menu bar. • Private . to route leak to other Tenants within the Fabric. It will be con​ fig​ ured in the leaf nodes which have EPGs in that bridge do​ main. 2 In the Work pane.The con​ trol can be spe​ cific pro​ to​ cols ap​ plied to the sub​ net such as IGMP Snoop​ ing.Enables IGMP Snooping on the subnet. choose the Tenant_Name. 5 In the Create Bridge Domain dialog box.Defines subnets under an endpoint group. configured. Enter an Bridge Domain Name.Tenants 185 Route Pro​ file .The IP ad​ dress and mask of the de​ fault gate​ way. b.The as​ so​ ci​ ated route pro​ file name. The con​ trol can be: • Unspecified . Sub​ net Con​ trol . 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains. Choose the Network.

. Choose the DNS Label if applicable (optional). Remove a Bridge Domain 1 On the menu bar. h. e. Choose the Monitoring Policy if applicable (optional). g. Choose the Subnets (optional). 3 In the Navigation Pane choose Tenant_Name > Networking > Bridge Domain > Bridge Domain_Name. 4 In the Work pane. 4 In the Work pane. Choose the Forwarding Semantics (optional). choose Tenants > ALL TENANTS. Choose the Associated L3 Outs (optional). i. i. Choose the IGMP Snoop Policy (optional). Choose the Route Profile (optional). 6 Click Submit. e. b. f. choose the Policy tab and perform the following actions: a. choose Actions > Delete. Choose the Route Profile (optional). d. Choose the Associated L3 Outs (optional). h. c. g. Choose the L3 Out for Route Profile (optional). 5 Click Finish. Choose the L3 Out for Route Profile (optional). 2 In the Work pane. f. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains > Bridge_Domain_Name. Choose the Subnets (optional). j.186 Tenants d. choose the Tenant_Name. Choose the IGMP Snoop Policy (optional). Choose the Monitoring Policy if applicable (optional). Choose the Network. choose Tenants > ALL TENANTS. Choose the DNS Label if applicable (optional). choose the Tenant_Name. Modify a Bridge Domain 1 On the menu bar. 2 In the Work pane.

Tenants 187 Verify Bridge Domain REST :: /api/node/class/fvBD.xml CLI :: moquery -c fvBD .

.

This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages: • Ability to use a single private network for all internal and external fabric • No route leaking needed between EPGs in different VRFs • Single Layer 3 Outside can be used by all tenants connectivity Dis​ ad​ van​ tages: • Changes to routing will impact all tenants From a con​ tain​ ment and re​ la​ tion​ ship per​ spec​ tive. 2 In the Work pane.Tenants 189 Tenant Networking Use Cases Common Private Network for All Tenants This use case may be typ​ i​ cal for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate mul​ ti​ ple ten​ ants. this topol​ ogy looks as fol​ lows: Common Private Network for all Tenants To Con​ fig​ ure the com​ mon Ten​ ant pri​ vate net​ work: 1 On the menu bar. but place all within a sin​ gle pri​ vate net​ work in the fab​ ric. choose the common. choose Tenants > ALL TENANTS. .

c. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains. Now the net​ work ad​ min​ is​ tra​ tor will have to as​ so​ ci​ ate the com​ mon pri​ vate net​ work to the Ten​ ant by first cre​ at​ ing a bridge do​ main. b. Choosean End Point Retention Policy Name (optional). 1 On the menu bar. d. In the Gateway IP field enter the IP address for this subnet. . In the Scope field you have the option to select Private. Choosea BGP Policy Name (optional). choose the Tenant_Name. 5 Click Finish. d. Choose the appropriate Monitoring Policy if applicable (optional). 7 Click Finish. Choose network. 2 In the Work pane. For more information on what to select please reference the External Layer 3 section. choose policy. perform the following actions: a. e. 5 In the Create Bridge Domain dialog box. 4 In the Work pane.190 Tenants 3 In the Navigation pane choose common > Networking > Private Networks > default. click +. choose Actions > Create Bridge Domain. 6 Click OK. a. In the Subnets field. choose Tenants > ALL TENANTS. Choose an OSPF Policy Name (optional). 4 In the Work pane. e. The Ten​ ant has been cre​ ated. b. Note: By default the Private option is selected. Enter a Bridge Domain Name. Choose a Policy Enforcement (optional). f. c. Public and Shared. Choosethe appropriate DNS Label if applicable (optional).

0.Tenants 191 The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'default' moconfig commit # subnet cd '/aci/tenants/Cisco/networking/bridge-domains/Cisco/subnets' mocreate '172.1/24' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit # criterion cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs/EPG1/vm-attributescriteria' mocreate 'default' moconfig commit This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API .16.

One Pri​ vate Net​ work per Ten​ ant with In​ tra-EPG com​ mu​ ni​ ca​ tions Ad​ van​ tages: • Allow for maximum isolation between tenants • Ability to address hosts in tenants with overlapping IP addresses Dis​ ad​ van​ tages: • Increased complexity when needing to allow EPG communication between different tenants with dedicated VRF .1/24" preferred="no" scope="private"/> </fvBD> <fvAp name="App1"> <fvAEPg matchT="AtleastOne" name="EPG1"> <fvRsBd tnFvBDName="Cisco"/> </fvAEPg> </fvAp> <fvRsTenantMonPol tnMonEPGPolName=""/> </fvTenant> For many multi-ten​ ant en​ vi​ ron​ ments it is de​ sir​ able to allow each ten​ ant to man​ age and own their own ad​ dress space and not be con​ cerned with over​ laps be​ tween other ten​ ants.16.192 Tenants XML : Ten​ ant Cisco <fvTenant name="Cisco"> <fvBD arpFlood="no" multiDstPktAct="bd-flood" name="Cisco" unicastRoute="yes" unkMacUcastAct="proxy" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="default"/> <fvSubnet ctrl="nd" descr="" ip="172.0. This par​ tic​ ul​ ar use case demon​ strates how a pri​ vate net​ work can be as​ so​ ci​ ated with each ten​ ant.

perform the following actions: a. Click Next. Enter Private Network Name. Enter a Tenant Name. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. choose the Tenant_Name. 3 Click Finish. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. 1 On the menu bar. In the Create Private Network dialog box. . b. 4 5 In the Work pane. 2 In the Create Tenant dialog box. The Ten​ ant has been cre​ ated. perform the following actions: a. choose Actions > Create Private Network. Click Next.Tenants 193 The ob​ ject con​ tain​ ment for this par​ tic​ u​ lar setup can be de​ picted as shown below: Private Network per Tenant To cre​ ate the ten​ ant: 1 On the menu bar. 2 In the Work pane. b. choose Tenants > ADD TENANT. choose Tenants > ALL TENANTS.

194 Tenants c. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # subnet cd '/aci/tenants/Cisco/networking/bridge-domains/Cisco/subnets' mocreate '172. In the Gateway IP field enter the IP address for this subnet. In the Scope field you have the option to select Private. d. 6 Click OK. Enter Associated Bridge Domain Name. Public and Shared.1/24' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit . Note: By default the Private option is selected. For more information on what to select please reference the External Layer 3 section. In the Subnets field. f. e.0. 7 Click Finish.16. click +.

This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages: • Ability to have overlapping subnets within a single tenant Dis​ ad​ van​ tages: • EPGs residing in overlapping subnets cannot have policy applied between one another .0.1/24" name="" preferred="no" scope="private"/> </fvBD> <fvCtx knwMcastAct="permit" name="Cisco" pcEnfPref="enforced"/> <fvAp name="App1" prio="unspecified"> <fvAEPg name="EPG1"> <fvRsBd tnFvBDName="Cisco"/> </fvAEPg> </fvAp> </fvTenant> Multiple Private Networks with Intra-Tenant Communication An​ other use case that may be de​ sir​ able to sup​ port is the op​ tion to have a sin​ gle ten​ ant with mul​ ti​ ple pri​ vate net​ works. due to merg​ ers and ac​ qui​ si​ tions or other busi​ ness changes. This may be a re​ sult of need​ ing to pro​ vide multi-ten​ ancy at a net​ work level.Tenants 195 This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : Ten​ ant Cisco <fvTenant name="Cisco"> <fvBD arpFlood="no" multiDstPktAct="bd-flood" name="Cisco" unicastRoute="yes" unkMacUcastAct="proxy" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="Cisco"/> <fvSubnet ctrl="nd" ip="172.16. but not at a man​ age​ ment level. It may also be caused by need​ ing to sup​ port over​ lap​ ping sub​ nets within a sin​ gle ten​ ant.

Enter a Tenant Name. Click Next. choose the Tenant_Name. choose Tenants > ADD TENANT. The Ten​ ant has been cre​ ated. In the Name field enter a name for the bridge domain. b.196 Tenants The ob​ ject con​ tain​ ment for this par​ tic​ u​ lar setup can be de​ picted as shown below: Multiple Private Networks with Intra-Tenant communication To cre​ ate the ten​ ant: 1 On the menu bar. 2 In the Work pane. perform the following actions: a. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. 2 In the Create Tenant dialog box. 4 Click Finish. 3 Click Next. . perform the following actions: a. choose Actions > Create Private Network. In the Create Private Network dialog box. In the Name field enter a name for the Private Network. c. 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. choose Tenants > ALL TENANTS. 4 5 In the Work pane.

Click Next. In the Gateway IP field enter the IP address for this subnet. e. In the Gateway IP field enter the IP address for this subnet. e. In the Name field enter a name for the Application Profile. In the Name field enter a name for the second Private Network. In the Create Private Network dialog box. 1 On the menu bar. 6 Click Submit. 11 Click OK. . In the Subnets field click +. In the Scope field select Shared. In the Name field enter a name for the second bridge domain. f. 7 Click Finish. 8 In the Navigation pane choose Tenant_Name > Networking > Private Networks. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles. perform the following actions: a. you can move to the Ap​ pli​ ca​ tion Pro​ file. 2 In the Work pane. 6 Click OK. c. 9 10 In the Work pane. perform the following actions: a. d. In the Scope field select Shared. b. In the Subnets field click +. In the Create Application Profile dialog box. choose Actions > Create Private Network.Tenants 197 d. Now that the two pri​ vate net​ works and bridge do​ mains have been cre​ ated. f. 12 Click Finish. choose Tenants > ALL TENANTS. choose the Tenant_Name. 4 5 In the Action Tab select Create Application Profile.

Re​ peat these steps for the sec​ ond EPG. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Filters. In the Create Filter dialog box. 6 Click Finish. c. perform the following actions: a. In the Name field enter a name for the Filter. b. In the Ethertype select IP. 4 5 In the Work pane. choose the Tenant_Name. 2 In the Work pane. 4 5 In the Work pane. perform the following actions: a.198 Tenants To cre​ ate the two end​ point groups: 1 On the menu bar. In the IP Protocol column select ICMP. choose the Tenant_Name. you can re​ peat the process of defin​ ing . 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name. choose Actions > Create Filters. In the Bridge Domain field. choose Tenants > ALL TENANTS. 2 In the Work pane. choose Tenants > ALL TENANTS. you would have the knowl​ edge of the fil​ ters re​ quired to per​ mit traf​ fic across the two EPGs. In the Name column enter ICMP. e. d. The ten​ ant ad​ min​ is​ tra​ tor will have to now cre​ ate a con​ tract and fil​ ter be​ tween the two EPGs. 7 Click Submit. In the Name field enter a name for the End Point Group. As the ten​ ant ad​ min​ is​ tra​ tor. Click +. 6 Click Update. select the appropiate bridge domain. In the Create Application EPG dialog box. b. 1 On the menu bar. choose Actions > Create Application EPG. In the fil​ ter.

. 8 Click Submit. choose Actions > Create Contract. Now you have to de​ fine the con​ tract that will be con​ sumed and pro​ vided by the two EPGs. Each EPG that you have cre​ ated will then ei​ ther con​ sume or pro​ vide that con​ tract to es​ tab​ lish a re​ la​ tion​ ship be​ tween both EPGs. 2 In the Work pane. perform the following actions: a.Tenants 199 var​ io ​us dif​ fer​ ent net​ work pro​ to​ cols as re​ quired for your ap​ pli​ ca​ tions. choose Actions > Add Provided Contract. As​ sign the Con​ tract be​ tween the EPGs. Click Subjects field click +. In the Scope field select Global. 4 In the Work pane. f. Select the created contract . choose Tenants > ALL TENANTS. b. perform the following actions: a. c. 6 Click Update. d. Click Submit. a. b. In the Filter Chain field click +. A con​ tract is as​ signed to an EPG as ei​ ther a con​ sumed or a pro​ vided con​ tract. e. 6 7 In the Navigation pane under the second EPG select Contracts. In the Action Tab select Add Consume Contract. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contract. choose Tenants > ALL TENANTS. 5 In the Add Provided Contract dialog box. 2 In the Work pane. 4 5 In the Work pane. choose the Tenant_Name. In the Create Contract dialog box. 1 On the menu bar. choose the Tenant_Name. 7 Click OK. Select the Filter you just created. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Contracts. In the Name field enter a name for the Filter. Select the created contract. 1 On the menu bar. In the Name field enter a name for the Subject.

200 Tenants 8 Click Submit. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco1' cd 'Cisco1' moset network 'Cisco1' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco1' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' .

1/24' cd '172.16.Tenants 201 moconfig commit # fv-rscon cd '/aci/tenants/Cisco/application-profiles/App1/applicationepgs/EPG1/contracts/consumed-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs/EPG1/subnets' mocreate '172.16.1/24' cd '172.shared' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG2' cd 'EPG2' moset bridge-domain 'Cisco1' moconfig commit # fv-rsprov cd '/aci/tenants/Cisco/application-profiles/App/applicationepgs/EPG2/contracts/provided-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/CCO/application-epgs/EPG2/subnets' mocreate '172.1.2.shared' moconfig commit This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : Ten​ ant Cisco <fvTenant dn="uni/tn-Cisco" name="Cisco"> <vzBrCP name="ICMP" scope="tenant"> <vzSubj consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" .2.1.1:24' moset scope 'private.16.1:24' moset scope 'private.16.

1/24" scope="private.16.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD2"/> </fvAEPg> <fvAEPg matchT="AtleastOne" name="App"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-202/pathep-[eth1/2]"/> <fvSubnet ip="172.2.1/24" scope="private.1.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsProv matchT="AtleastOne" tnVzBrCPName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> Multiple Private Networks with Inter-Tenant Communication .202 Tenants revFltPorts="yes"> <vzRsSubjFiltAtt tnVzFilterName="icmp"/> </vzSubj> </vzBrCP> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvCtx knwMcastAct="permit" name="CiscoCtx2" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx2"/> </fvBD> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="Web"> <fvRsCons tnVzBrCPName="ICMP"/> <fvRsPathAtt encap="vlan-1201" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-201/pathep-[eth1/16]"/> <fvSubnet ip="172.16.

This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages • Each tenant container can be managed separately • Allows for maximum isolation between tenants Dis​ ad​ van​ tages • Tenant address space must be unique From a con​ tain​ ment and re​ la​ tion​ ship per​ spec​ tive. 2 In the Create Tenant dialog box perform the following actions: a. In the Name field enter a name for the first Tenant.Tenants 203 Multiple Private Networks with Inter-Tenant Communication This use case may be typ​ i​ cal for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate mul​ ti​ ple ten​ ants with the abil​ ity to sup​ port in​ ter-ten​ ant com​ mu​ ni​ ca​ tions. 3 Click Next. this topol​ ogy looks as fol​ lows: Multiple Private Networks with Inter-Tenant Communication To cre​ ate the ten​ ant: 1 In the GUI Navigate to Tenants > ADD TENANT. .

In the Create Application Profile dialog box. 2 In the Work pane. perform the following actions: a. choose Tenants > ALL TENANTS. choose Actions > Create Private Network. 4 5 In the Work pane. In the Name field enter a name for the Application Profile. 1 On the menu bar. 3 In the Navigation Pane choose Tenant_Name > Networking > Private Networks. . b. 2 In the Work pane. perform the following actions: a. d. To cre​ ate the ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. Click Next. 6 Click Submit. choose the Tenant_Name. c. 7 Click Finish. choose the Tenant_Name. 4 In the Work pane. In the Gateway IP field enter the IP address for this subnet. In the Subnets field click +. 3 In the Navigation Pane choose Tenant_Name > Networking > Application Profiles. In the Scope field select Shared. In the Name field enter a name for the bridge domain. choose Actions > Create Application Profile. f.204 Tenants 4 Click Finish. e. The ten​ ant has been cre​ ated. choose Tenants > ALL TENANTS. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. 6 Click OK. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). 5 In the Create Private Network dialog box. In the Name field enter a name for the Private Network.

. 4 Click Finish. 5 In the Create Private Network dialog box. 6 Click Finish. b. choose the Tenant_Name. choose the Tenant_Name. In the Subnets field click +. perform the following actions: a. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name 4 5 In the Work pane. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. choose Actions > Create Private Network. 2 In the Work pane. c. d. The ten​ ant has been cre​ ated. In the Name field enter a name for the End Point Group. 2 In the Work pane. In the Bridge Domain field.Tenants 205 To cre​ ate the end​ point group: 1 On the menu bar. In the Create Application EPG dialog box. 4 In the Work pane. In the Name field enter a name for the bridge domain. 2 In the Create Tenant dialog box perform the following actions: a. In the Name field enter a name for the Private Network. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. b. In the Gateway IP field enter the IP address for this subnet. e. choose Tenants > ALL TENANTS. 1 On the menu bar. 3 Click Next. Click Next. In the Name field enter a name for the first Tenant. choose Actions > Create Application EPG. perform the following actions: a. choose Tenants > ALL TENANTS. select the appropiate bridge domain. To cre​ ate the sec​ ond ten​ ant and ap​ pli​ ca​ tion pro​ file: 1 In the GUI Navigate to Tenants > ADD TENANT.

3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name 4 5 In the Work pane. 4 5 In the Work pane. 1 On the menu bar.206 Tenants f. To cre​ ate the end​ point group: 1 On the menu bar. choose the Tenant_Name. 7 Click Finish. . choose Tenants > ALL TENANTS. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles. choose the Tenant_Name. perform the following actions: a. 2 In the Work pane. select the appropiate bridge domain. choose Actions > Create Application Profile. b. In the Name field enter a name for the End Point Group. 2 In the Work pane. In the Scope field select Shared. choose Actions > Create Application EPG. 6 Click OK. In the Create Application EPG dialog box. In the Bridge Domain field. In the Create Application Profile dialog box. 2 In the Work pane. 6 Click Finish. In the Name field enter a name for the Application Profile. 6 Click Submit. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). choose the Tenant_Name. perform the following actions: a. The ten​ ant ad​ min​ is​ tra​ tor will have to now cre​ ate a con​ tract and fil​ ter be​ tween the two EPGs. choose Tenants > ALL TENANTS. choose Tenants > ALL TENANTS. To cre​ ate the ap​ pli​ ca​ tion pro​ file: 1 On the menu bar.

perform the following actions: a. In the IP Protocol column select ICMP. In the Name field enter a name for the Subject. 2 In the Work pane. choose Tenants > ALL TENANTS. Click Subjects field click +. choose Actions > Create Filters. 6 Click Update. In the Create Contract dialog box. Click +. In the Ethertype select IP. f. 8 Click Submit. 7 Click Submit. c. In the Name field enter a name for the filter. In the fil​ ter you can re​ peat the process of defin​ ing var​ io ​us dif​ fer​ ent net​ work pro​ to​ cols as re​ quired for your ap​ pli​ ca​ tions. . d. 5 In the Create Filter dialog box. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Contracts. d. Now you have to de​ fine the con​ tract that will be con​ sumed and pro​ vided by the two EPGs. e. choose the Tenant_Name. choose Actions > Create Contract. e. perform the following actions: a. In the Name field enter a name for the Filter. 6 Click Update. c. In the Filter Chain field click +. 4 5 In the Work pane. b. 7 Click OK. b. As the ten​ ant ad​ min​ is​ tra​ tor you would have the knowl​ edge of the fil​ ters re​ quired to per​ mit traf​ fic across the two EPGs. Select the filter you just created.Tenants 207 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Filters. In the Scope field choose Global. 1 On the menu bar. 4 In the Work pane. In the Name column enter ICMP.

choose Actions > Add Provided Contract. Each EPG that you have cre​ ated will then ei​ ther con​ sume or pro​ vide that con​ tract to es​ tab​ lish a re​ la​ tion​ ship be​ tween both EPGs. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : TEN​ ANT Cis​ co1 # tenant cd '/aci/tenants' mocreate 'Cisco1' moconfig commit # bridge-domain cd '/aci/tenants/Cisco1/networking/bridge-domains' mocreate 'Cisco1' cd 'Cisco1' moset network 'Cisco1' moconfig commit . b. choose Action > Add Consume Contract. 2 In the Work pane. Click Submit.208 Tenants As​ sign the Con​ tract be​ tween the EPGs. Select the created Contract. choose Tenants > ALL TENANTS. Select the created Contract. 8 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contracts. 10 Click Submit. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contracts. 5 In the Add Provided Contract dialog box. A con​ tract is as​ signed to an EPG as ei​ ther a con​ sumed or a pro​ vided con​ tract. 9 In Work pane. choose the Tenant_Name. 7 Select the second created Tenant. 1 On the menu bar. 6 In the GUI Navigate to Tenants > ALL TENANTS. a. 4 In the Work pane. perform the following actions: a.

1.shared' moconfig commit # contract cd '/aci/tenants/Cisco/security-policies/contracts' mocreate 'ICMP' cd 'ICMP' moset scope 'global' moconfig commit # contract-subject cd '/aci/tenants/Cisco/security-policies/contracts/ICMP/subjects' mocreate 'icmp' moconfig commit # vz-rssubjfiltatt cd '/aci/tenants/Cisco/security-policies/contracts/ICMP/subjects/icmp/common-filters' mocreate 'icmp' moconfig commit .1.Tenants 209 # private-network cd '/aci/tenants/Cisco1/networking/private-networks' mocreate 'Cisco1' moconfig commit # application-profile cd '/aci/tenants/Cisco1/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco1/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco1' moconfig commit # fv-rsprov cd '/aci/tenants/Cisco/application-profiles/CCO/applicationepgs/App/contracts/provided-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/CCO/application-epgs/App/subnets' mocreate '172.16.1:24' moset scope 'private.16.1/24' cd '172.

16.2.2.16.1:24' moset scope 'shared-subnet' moconfig commit # imported-contract cd '/aci/tenants/Cisco1/security-policies/imported-contracts' mocreate 'CiscoInterTenantICMP' cd 'CiscoInterTenantICMP' moset contract 'tenants/Cisco/security-policies/contracts/ICMP' moconfig commit .1/24' cd '172.210 Tenants CLI : TEN​ ANT Cis​ co2 # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco2/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit # fv-rsconsif cd '/aci/tenants/Cisco1/application-profiles/CCO/applicationepgs/Web/contracts/consumed-contract-interfaces' mocreate 'CiscoInterTenantICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco1/application-profiles/CCO/application-epgs/Web/subnets' mocreate '172.

Tenants 211 This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : TEN​ ANT Cis​ co1 <fvTenant dn="uni/tn-Cisco1" name="Cisco1"> <vzBrCP name="ICMP" scope="global"> <vzSubj consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" revFltPorts="yes"> <vzRsSubjFiltAtt tnVzFilterName="icmp"/> </vzSubj> </vzBrCP> <vzCPIf dn="uni/tn-Cisco1/cif-ICMP" name="ICMP"> <vzRsIf consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" revFltPorts="yes"> <vzRsSubjFiltAtt tDn="uni/tn-Cisco2/brc-default"/> </vzRsIf> </vzCPIf> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx2"/> </fvBD> <fvBD arpFlood="yes" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="EPG1"> .

1.1/24" scope="private.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsProv matchT="AtleastOne" tnVzBrCPName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> XML : TEN​ ANT Cis​ co2 <fvTenant dn="uni/tn-Cisco2" name="Cisco2"> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvBD arpFlood="yes" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="EPG2"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-201/pathep-[eth1/2]"/> <fvSubnet ip="172.1.shared"/> .1/24" scope="private.16.212 Tenants <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-202/pathep-[eth1/2]"/> <fvSubnet ip="172.16.

Tenants 213 <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsConsIf matchT="AtleastOne" tnVzBrCPIfName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> .

.

215 Working with Contracts .

.

Working with Contracts 217 Section Content • Contracts Contract Configuration Parameters Create/Modify/Remove Contracts Create Contracts Modify Contracts Remove Contracts Verify Contracts • Apply/Remove EPG Contracts Apply a Contract to an EPG Remove a Contract from EPG Verify Contract on EPG • Apply/Remove External Network Contracts Apply a Contract to an External Network Remove a Contract from an External Network Verify External Network Contracts • Apply/Remove Private Network Contracts Apply a Contract to a Private Network (vzAny) Remove a Contract From a Private Network (vzAny) Verify Private Network Contracts • Filters Filter Entry Configuration Parameters Create Filters Modify Filters Remove Filters Verify Filters .

Modify. or Delete Taboo Contracts Create Taboo Contracts Modify Taboo Contracts Delete Taboo Contracts Verify Taboo Contracts • Apply/Remove Taboo Contracts Apply Taboo Contract to an EPG Remove Taboo Contract from EPG Verify Taboo Contracts Applied to EPG • Inter-Tenant Contracts Configuration Parameters • Create/Modify/Remove Export Contracts Export Contract Modify Exported Contracts Remove Exported Contracts Verify Exported Contracts • Contracts Use Cases Inter-Tenant Contracts Tenant Cisco-1/EPG-1 Tenant Cisco-2/EPG-2 • Inter-Private Network Contracts Communication Tenant Cisco-1/EPG-1 Tenant Cisco-1/EPG-2 .218 Working with Contracts • Taboo Contracts Taboo Contract Configuration Parameters Create.

Working with Contracts 219 • Single Contract Bidirectional Reverse Filter • Single Contract Unidirectional with Multiple Filters • Multiple Contracts Uni-Directional Single Filter .

.

and service graph to perform Ethernet type. TCP flags and ports) matches based upon filters • Labels . Con​ tracts are com​ prised of the fol​ low​ ing items: • Subjects . The ex​ am​ ple below. There is no con​ tract re​ quired for in​ tra-EPG com​ mu​ ni​ ca​ tion: in​ tra-EPG com​ mu​ ni​ ca​ tion is al​ lowed by de​ fault. .Working with Contracts 221 Contracts Con​ tracts pro​ vide a way for the ACI ad​ min​ is​ tra​ tor to con​ trol traf​ fic flow within the ACI fab​ ric be​ tween EPGs. and the Ap​ pli​ ca​ tion EPG pro​ vides a con​ tract which the Data​ base EPG would con​ sume. protocol type. shows how con​ tracts would con​ trol traf​ fic flow be​ tween EPGs in a 3-tiered ap​ pli​ ca​ tion.A group of filters for a specific application or service • Filters . These con​ tracts are built using a provider-con​ sumer model where one EPG pro​ vides the ser​ vices it wants to offer and an​ other EPG con​ sumes them.Permit. copy. EPGs may acts a both a provider and con​ sumer of the same con​ tract eas​ ily by mak​ ing the con​ tract bidi​ rec​ tional.Used optionally to group objects such as subjects and EPGs for the purpose of further defining policy enforcement EPGs can only com​ mu​ ni​ cate with other EPGs based upon the con​ tract rules de​ fined. deny. log. mark. The Web EPG pro​ vides a con​ tract which is con​ sumed by the Ap​ pli​ ca​ tion EPG.Used to classify traffic based upon layer 2 to layer 4 attributes (such as • Actions . redirect.

.222 Working with Contracts Contract Policies Between End Point Groups Con​ tracts gov​ ern the fol​ low​ ing types of end​ point group com​ mu​ ni​ ca​ tions: • Between application EPGs • Between application EPGs and external networks • Between application EPGs and in-band management EPG.The scope of a ser​ vice con​ tract be​ tween two or more par​ tic​ i​ pat​ ing peer en​ ti​ ties. • Application-profile . same application profile.This contract will be applied for endpoint groups within the same tenant.This contract will be applied for endpoint groups associated with the same private network. for example if inband management is configured for the ACI fabric and certain EPGs are to be allowed to access it Contract Configuration Parameters When con​ fig​ ur​ ing con​ tracts you can de​ fine the fol​ low​ ing op​ tions: Con​ tract Scope . The states are: • Context .This contract will be applied for endpoint groups in the • Tenant .

Class 2 DSCP value. choose the Tenant_Name. 2 In the Work pane. You can as​ sign the same tag name to mul​ ti​ ple ob​ jects and you can as​ sign one or more tag names to an ob​ ject. Create/Modify/Remove Contracts Create Contracts 1 On the menu bar.The pri​ or​ ity level of the ser​ vice con​ tract. A tag al​ lows you to group mul​ ti​ ple ob​ jects by a de​ scrip​ tive name.This contract will be applied for endpoint groups throughout the fabric. The pri​ or​ ity level can be: • Unspecified • Level1 . 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts. The op​ tions are: • AtleastOne • AtmostOne • None • All The de​ fault is Atlea​ st​ O ​ne.Class 3 DSCP value. choose Tenants > ALL TENANTS. The de​ fault is Con​ text. Tags .Working with Contracts 223 • Global . • Level3 . The de​ fault is Un​ spec​ i​ fied. .The search key​ word or term that is as​ signed to the ap​ pli​ ca​ tion pro​ file.Class 1 Differentiated Services Code Point (DSCP) value. Match . • Level2 .The sub​ ject match cri​ te​ ria across con​ sumers. QoS Class .

Enter a Contract Subject Name. Choose a QoS Class (optional). 8 Click Submit. Choose a Contract Scope (optional). choose the Policy tab. to add a Contract Subject. c. Click + next to Filter Chain. 6 Click OK. 2 In the Work pane. Choose a Qos Class (optional). Click + next to the Subject field. Choose a Contract Scope (optional). In the Create Contract Subject dialog box. 2. Note: For information regarding filter creation. Click + in the Filter Chain field. perform the following actions: 1. perform the following actions: 1. b. i. choose the Tenant_Name. perform the following actions: a. In the Create Contract Subject dialog box. 7 Click OK. 4 In the Work pane. Modify Contracts 1 On the menu bar.​ For information regarding filter creation. Enter a Contract Name. Enter a Contract Subject Name. b. In the Create Contract dialog box. 5 Click Update. Remove Contracts . choose Tenants > ALL TENANTS. see the "Filters" section. a. d. 6 Click Update. Click + next to the Subject to add a Contract Subject. 7 Click Submit. c. see the "Filters" section. 2.224 Working with Contracts 4 5 In the Work pane. i. choose Actions > Create Contract. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name.

4 In the Work pane. Enter a Contract_Name. 4 In the Work pane. choose the Tenant_Name. 6 Click Submit. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. c. Choose a QOS policy (optional). Verify Contracts REST :: /api/node/class/vzBrCP. choose the Tenant_Name.Working with Contracts 225 Remove Contracts 1 On the menu bar. b. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts. Choose a Label (optional). choose Actions > Add Provided Contract or Actions > Add Consumed Contract. 2 In the Work pane. perform the following actions: a. choose Actions > Delete. choose Tenants > ALL TENANTS. Remove a Contract from EPG 1 On the menu bar. 2 In the Work pane.xml CLI :: moquery -c vzBrCP Apply/Remove EPG Contracts Apply a Contract to an EPG 1 On the menu bar. 5 In the Add Contract dialog box. choose Tenants > ALL TENANTS. Note: Choose the action depending on how the contract is to be deployed. choose Tenants > ALL TENANTS. .

3 In the Navigation pane choose Tenant_Name > Networking > External Routed Networks > Routed Outside_Name > Networks > External_Network_Instance_Profile. click + next to either Add Provided Contract or Add Consumed Contract. choose Actions > Delete. Choose a Match Criteria. c. choose Tenants > ALL TENANTS. 4 In the Work pane. 5 Click Update. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name . choose the Tenant_Name.226 Working with Contracts 2 In the Work pane. 4 In the Work pane. choose the Tenant_Name.xml CLI :: moquery -c fvRsProv Consumer ​ REST :: /api/node/class/fvRsCons. 2 In the Work pane. Note: Make a selection depending on how the contract is to be deployed. a. b.> Application EPGs > EPG_Name > Contracts > Contract_Name. . Choose a QOS Type. Choose a Contract_Name. Verify Contract on EPG Provider REST :: /api/node/class/fvRsProv.xml CLI :: moquery -c fvRsCons Apply/Remove External Network Contracts Apply a Contract to an External Network 1 On the menu bar.

if an ACI Ad​ min​ is​ tra​ tion has 100 EPGs that are all part of the same pri​ vate net​ work. . For in​ stance.xml CLI :: moquery -c fvRsProv Consumer REST :: /api/node/class/fvRsCons. This con​ cept is also re​ ferred as "vzAny" end​ point group​ . choose the Tenant_Name. con​ tracts can be ap​ plied di​ rectly to the pri​ vate net​ work. 2 In the Work pane. they can apply the con​ tracts to this one vzAny group under the pri​ vate net​ work. choose the Tenant_Name. 2 In the Work pane.xml CLI :: moquery -c fvRsCons Apply/Remove Private Network Contracts In order to apply con​ tracts to all end​ point groups within a pri​ vate net​ work. It eases con​ tract man​ age​ ment by al​ low​ ing the con​ tract con​ fig​ ur​ a​ tion for all end​ point groups within a pri​ vate net​ work from a sin​ gle lo​ ca​ tion as well as op​ ti​ miz​ ing hard​ ware re​ source con​ sump​ tion. choose Tenants > ALL TENANTS.Working with Contracts 227 Remove a Contract from an External Network 1 On the menu bar. Apply a Contract to a Private Network (vzAny) 1 On the menu bar. 4 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > External Routed Networks > Routed Outside_Name > Networks > External_Network_Instance_Profile. choose the Contract_Name and click x. rather than to each EPG. choose Tenants > ALL TENANTS. Verify External Network Contracts Provider REST :: /api/node/class/fvRsProv.

4 In the Work pane. Remove a Contract From a Private Network (​ vzAny) 1 On the menu bar. Click Update. choose Tenants > ALL TENANTS. Choose a QOS Type. Enter a Contract_Name. choose the Contract_Name and click x.228 Working with Contracts 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network_Name > EPG Collection for Context. Verify Private Network Contracts REST :: /api/node/class/vzBrCP. c. b. 4 In the Work pane. click + next to either Add Provided Contract or Add Consumed Contract. a. 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network_Name > EPG Collection for Context. Note: Make a selection depending on how the contract is to be deployed.xml CLI :: moquery -c vzBrCP . choose the Tenant_Name. Choose a Match Criteria.

IP Pro​ to​ col . The fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. an EPG provider dic​ tates the pro​ to​ cols and ports in both the in and out di​ rec​ tions. . TCP/IP header fields such as Layer 3 pro​ to​ col type. Filter Entry Configuration Parameters When con​ fig​ ur​ ing a Fil​ ter. See the Use Cases below for an ex​ am​ ple.The name of a fil​ ter entry. Fil​ ters are Layer 2 to Layer 4 fields.The Ether​ Type of the fil​ ter entry. Layer 4 ports. Con​ tract sub​ jects con​ tain as​ so​ ci​ a​ tions to the fil​ ters (and their di​ rec​ tions) that are ap​ plied be​ tween EPGs that pro​ duce and con​ sume the con​ tract. Arp Flag .The Ad​ dress Res​ ol​ u​ tion Pro​ to​ col flag for a fil​ ter entry. and so on. The fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. The Ether​ Types are: • ARP • FCOE • IP • MAC Security • MPLS Unicast • Trill • Unspecified The de​ fault is ARP. Ether​ Type .Working with Contracts 229 Filters A fil​ ter pol​ icy is a group of re​ solv​ able fil​ ter en​ tries. the fol​ low​ ing op​ tions can be de​ fined: Name . Ac​ cord​ ing to its re​ lated con​ tract. Each fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties.The IP pro​ to​ col for a fil​ ter entry.

The range is 0 to 0xffff.The start of the source port range. The range is 0 to 0xffff.The start of the source port range.The end of the source port range.The end of the des​ range is de​ ter​ mined by the server type. The start of the port range is de​ ter​ mined by the server type. The range is 0 to 0xffff. The port can be for the fol​ low​ ing server types: • Unspecified • ftpData • SMTP • DNS • HTTP • POP3 • HTTPS • RTSP Source Port: To . The start of the port range is de​ ter​ mined by the server type. The end of the port range is de​ ter​ mined by the server type.230 Working with Contracts Allow Frag​ ment . The port state set​ tings are: • Unspecified • ftpData . The port can be for the fol​ low​ ing server types: Source Port: From . The end of the port Des​ ti​ na​ tion Port: From . The port can be for the fol​ low​ ing server types: • Unspecified • ftpData • smtp • DNS • HTTP • POP3 • HTTPS • RTSP ti​ na​ tion port range. The range is 0 to 0xffff.

. The port range is de​ ter​ mined by the server type. Enter an Filter Name. sion rules for a fil​ ter entry. The fil​ ter entry is a com​ bi​ TCP Ses​ sion Rules . perform the following actions: a. The range is 0 to 0xffff.Working with Contracts 231 • SMTP • DNS • HTTP • POP3 • HTTPS • RTSP Des​ ti​ na​ tion Port: To . 5 In the Create Filter dialog box.The start of the des​ ti​ na​ tion port range. choose the Tenant_Name. choose Actions > Create Filter. Create Filters 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters.The TCP ses​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. choose Tenants > ALL TENANTS. The des​ ti​ na​ tion port can be set for the fol​ low​ ing server types: • unspecified • ftpData • smtp • dns • http • pop3 • https • rtsp The de​ fault is un​ spec​ i​ fied. 4 In the Work pane. b. 2 In the Work pane. Click + to add a Filter Entry.

Enter a Filter Entry Name. Modify Filters 1 On the menu bar. f. check the Allow Fragment check box (optional). In the drop-down list select the Source Port From (optional). 10. 8. 6 Click Submit. h. double click the filter entry. In the drop-down list select the TCP Session Rules (optional). 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters > Filter_Name. 7. b. choose Tenants > ALL TENANTS. click the Allow Fragment check box (optional). In the drop-down list select the TCP Session Rules (optional). e. In the drop-down list select the Destination Port To (optional). In the drop-down list select the Source Port From (optional). choose the Tenant_Name. a. 6 Click Update. If required. 5 Click Update. 7 Click Submit. 2 In the Work pane. In the drop-down list select the Destination Port From (optional). In the drop-down list select an IP Protocol (optional). If required. g. In the drop-down list select an IP Protocol (optional). 9. 4. In the drop-down list select an ARP Flag (optional). 2. In the drop-down list select the Source Port To (optional). 6. In the drop-down list select an Ethertype. In the drop-down list select the Destination Port To (optional). . c. 3. In the drop-down list select the Destination Port From (optional). 4 In the Work pane. i. In the drop-down list select an Ethertype (optional). In the Filter Entry dialog box. d.232 Working with Contracts i. perform the following actions: 1. 5. In the drop-down list select the Source Port To (optional). In the drop-down list select an ARP Flag (optional).

choose Actions > Delete.xml CLI :: moquery -c vzFilter .Working with Contracts 233 Remove Filters 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters > Filter_Name. choose the Tenant_Name. 2 In the Work pane. choose Tenants > ALL TENANTS. 4 In the Work pane. Verify Filters REST :: /api/node/class/vzFilter.

.

Taboos are a spe​ cial type of con​ tract that an ACI ad​ min​ is​ tra​ tor can use to deny spe​ cific traf​ fic that would oth​ er​ wise be al​ lowed by an​ other con​ tract. match​ ing a fil​ ter. La​ bels en​ able clas​ si​ fi​ ca​ tion of the ob​ jects which can and can​ not com​ mu​ ni​ cate with one an​ other (op​ tional). Taboo con​ tracts are not rec​ om​ mended as part of the ACI best prac​ tices but they can be used to tran​ si​ tion from tra​ di​ tional net​ work​ ing to ACI. Taboos can be used to drop traf​ fic match​ ing a pat​ tern (any EPG. a spe​ cific EPG. Di​ rec​ tive . and so forth). Modify. Taboo Contract Configuration Parameters When con​ fig​ ur​ ing Taboo Con​ tracts you can de​ fine the fol​ low​ ing op​ tions: Name . To im​ it​ ate the tra​ di​ tional net​ work​ ing con​ cepts. choose the Tenant_Name. .The fil​ ter di​ rec​ tives as​ signed to the taboo con​ tract. Create.The name of the con​ tract or con​ tract ob​ ject. Sub​ jects . Taboo rules are ap​ plied in the hard​ ware be​ fore the rules of reg​ ul​ ar con​ tracts are ap​ plied. or Delete Taboo Contracts Create Taboo Contracts 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts. an "al​ low-all-traf​ fic" con​ tract can be ap​ plied.The net​ work do​ main name label. 2 In the Work pane.Working with Contracts 235 Taboo Contracts There may be times when the ACI ad​ min​ is​ tra​ tor might need to deny traf​ fic that is al​ lowed by an​ other con​ tract. choose Tenants > ALL TENANTS. with taboo con​ tracts con​ fig​ ured to re​ strict cer​ tain types of traf​ fic.

choose Tenants > ALL TENANTS. In the Create Taboo Contract dialog box.236 Working with Contracts 4 5 In the Work pane. Click + to next to the Subject field. Delete Taboo Contracts 1 On the menu bar. 2 In the Work pane. perform the following actions: i. Choose Directives. Click + in the Filter Chain field. Enter a Filter Name. 4 In the Work pane. perform the following actions: a. Enter a Taboo Contract Name. 2 In the Work pane. b. Click + to next to the Subject field to add a Taboo Subject. ii. 6 Click Update. 1. choose Tenants > ALL TENANTS. . Enter a Filter Name. choose Action > Create Taboo Contract. choose policy. 5 Click Submit. 8 Click Submit. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts > Taboo_Contract_Name. choose the Tenant_Name. In the Create Taboo Contract Subject dialog box. i. 2. a. ii. Choose Directives. Enter a Taboo Contract Subject Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts > Taboo_Contract_Name. b. Modify Taboo Contracts 1 On the menu bar. 7 Click OK.

choose Tenants > ALL TENANTS. 6 Click Submit. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts. 5 In the Add Taboo Contract dialog box.Working with Contracts 237 4 In the Work pane. a. choose Tenants > ALL TENANTS. choose Action > Delete. choose Actions > Add Taboo Contract. 4 In the Work pane. 2 In the Work pane.xml CLI :: moquery -c vzTaboo Apply/Remove Taboo Contracts Apply Taboo Contract to an EPG 1 On the menu bar. Remove Taboo Contract from EPG 1 On the menu bar. 2 In the Work pane. choose the Tenant_Name. choose the Taboo Contract_Name > Actions > Delete. choose the Tenant_Name. 4 In the Work pane. . Verify Taboo Contracts REST :: /api/node/class/vzTaboo. Choose the Taboo Contract. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts.

238 Working with Contracts Verify Taboo Contracts Applied to EPG Provider REST :: /api/node/class/fvRsProv.xml CLI :: moquery -c fvRsProv C ​onsumer REST :: /api/node/class/fvRsCons.xml CLI :: moquery -c fvRsCons .

Ten​ ant . 4 In the Work pane. Some use case ex​ am​ ples show the com​ plete process in the next chap​ ter. choose Actions > Export Contract. choose the Tenant_Name. How​ ever. The con​ tract in essence is ex​ ported in the source ten​ ant and im​ ported into the tar​ get ten​ ant. .The name of the con​ Global Con​ tract .The Ten​ ant name of the tar​ geted Ex​ port con​ tract. the fol​ low​ ing op​ tions can be de​ fined: tract in​ ter​ face.Working with Contracts 239 Inter-Tenant Contracts There may be times when the ACI ad​ min​ is​ tra​ tor might need to allow traf​ fic be​ tween two ten​ ants. In​ ter​ face con​ tracts are a spe​ cial type of con​ tract that an ACI ad​ min​ is​ tra​ tor can use to allow spe​ cific traf​ fic through the use of a con​ tract ex​ port. choose Tenants > ALL TENANTS. 2 In the Work pane. Sim​ il​ ar to tra​ di​ tional con​ tracts. Create/Modify/Remove Export Contracts Export Contract 1 On the menu bar. b. perform the following actions: a. Configuration Parameters When Im​ port​ ing a Con​ tract.Name of a ser​ vice con​ tract to be shared be​ tween two or more par​ tic​ ip ​at​ ing peer en​ ti​ ties. 5 In the Export Contract dialog box. Name . 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts. Choose the Global Contract. Enter an Export Contract Name. in the tar​ get ten​ ant. the con​ tract is im​ ported as type con​ tract in​ ter​ face. the source EPG will be of type provider.

c. a. 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Imported Contracts > Contact_Name. 4 In the Work pane. choose Actions > Delete. choose the Tenant_Name. choose Tenants > ALL TENANTS. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name.xml CLI :: moquery -c vzCPif . Modify Exported Contracts 1 On the menu bar. Remove Exported Contracts 1 On the menu bar. choose the Tenant_Name. Enter the Tenant Name. Verify Exported Contracts REST :: /api/node/class/vzCPif. Enter the Tenant Name. choose policy.240 Working with Contracts c. b. Choose the Global Contract. choose Tenants > ALL TENANTS. 6 Click Finish. Enter an Export Contract Name. 5 Click Finish. 2 In the Work pane. 4 In the Work pane.

There are four com​ mon sce​ nar​ ios: 1 Inter-Tenant Contracts. must allow this traf​ fic across the whole fab​ ric. 2 Inter-Private Network Contracts. achiev​ ing bidi​ rec​ tional traf​ fic. Inter-Tenant Contracts ACME Inc. In ten​ ant Cisco-1 .. 4 Single Contract Unidirectional with multiple Filters. How these sce​ nar​ ios are im​ ple​ mented will de​ pend on the op​ er​ at​ ional model cho​ sen. makes use of shared ser​ vices such as DNS for name res​ o​ lu​ tion and Ac​ tive Di​ rec​ tory for user man​ age​ ment. Re​ view the con​ tracts sec​ tion on Con​ tract Scop​ ing for a more de​ tailed dis​ cus​ sion. That con​ tract will ap​ pear under the Im​ ported Con​ tract sec​ tion in the Se​ cu​ rity Poli​ cies of the des​ ti​ na​ tion ten​ ant. Note: A con​ tract con​ sump​ tion in​ ter​ face rep​ re​ sents one or more sub​ jects de​ fined under the con​ tract. tract In​ ter​ face will be used to as​ so​ ci​ ate an EPG from the des​ ti​ na​ tion A Con​ sumed Con​ ten​ ant with the im​ ported con​ tract. In the use case below. This is ac​ com​ plished by uti​ liz​ ing con​ tact in​ ter​ faces.Working with Contracts 241 Contracts Use Cases These use cases all as​ sume the ob​ jec​ tive is for a host in EPG-1 to talk to a host in EPG2. an end​ point group starts con​ sum​ ing all the sub​ jects rep​ re​ sented by the in​ ter​ face. 3 Single Contract Bidirectional forwarding with reverse filter. 5 Multiple Contracts Unidirectional with single Filter. Com​ mu​ ni​ ca​ tion be​ tween EPGs that be​ long to dif​ fer​ ent ten​ ants is only al​ lowed when they share the same con​ tract. it will need to be ex​ ported from the source ten​ ant to the ap​ pro​ pri​ ate des​ ti​ na​ tion ten​ ant. as with most com​ pa​ nies. By as​ so​ ci​ at​ ing to an in​ ter​ face. To use the same con​ tract. These ser​ vices will be used across most of their ten​ ants and so ACME Inc. and whether the sys​ tem is more fo​ cused on ob​ ject reuse or ten​ ant au​ ton​ omy. EPG-1 in ten​ ant Cisco-1 re​ quires com​ mu​ ni​ ca​ tion with EPG-2 in ten​ ant Cisco-2.

The user will then con​ firm the im​ ported con​ tract in ten​ ant Cisco-2 and se​ lect the con​ tract as con​ sumed. Tenant Cisco-2/EPG-2 1 Confirm the exported contract is listed under Imported Contracts.contract type provider. 3 Add the Interface Contract under EPG2 . 3 Add the Contract under EPG1 .subnet scope shared. Exporting Contracts Between Tenants Tenant Cisco-1/EPG-1 1 Create an Export Contract under security policies. To ad​ ver​ tise the routes from the source VRF to the in​ tended VRF. the user must cre​ ate the sub​ net within the EPG. 4 Create the host subnet under the bridge domain .subnet scope private/public. 4 Create the host subnet (default Gateway IP) under the bridge domain . . In ten​ ant Cisco-1 the user will ex​ port the in​ tended con​ tract and se​ lect provider to pro​ vide the con​ trast to EPG-2. 2 Create the host subnet (default Gateway IP) under EPG1 .contract type consumed. 2 Create the host subnet (default Gateway IP) under EPG2 .subnet scope private/public.242 Working with Contracts the user will ex​ port the in​ tended con​ tract in​ ter​ faces.subnet scope shared.

Tenant Cisco-1/EPG-1 1 Create the host subnet (default Gateway IP) under EPG1 . By cre​ at​ ing the sub​ net under the EPG and se​ lect​ ing shared. 2 Add the Contract under EPG1 .subnet scope shared.Working with Contracts 243 Inter-Private Network Contracts Communication In the use case below.contract type provider.contract scope Tenant. 2 Add the Contract under EPG2 . This is ac​ com​ plished by uti​ liz​ ing the sub​ net field within the EPG.contract type provider. Exporting Contracts Between Private Networks 1 Create the contract under Security Policies .subnet scope shared. This is . EPG-1 in VRF Cisco-1 re​ quires com​ mu​ ni​ ca​ tion with EPG-2 in VRF Cisco-2. Tenant Cisco-1/EPG-2 1 Create the host subnet (default Gateway IP) under EPG2 . Single Contract Bidirectional Reverse Filter This use case is use​ ful when im​ ple​ ment​ ing a con​ tract with the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions and with the op​ tion to apply the re​ verse fil​ ter. the route will be leaked to the VRF noted within the Ten​ ant scoped con​ tract.

In this ex​ am​ ple. i.e. This al​ lows the Web Client in EPG-2 to ac​ cess the Web Server in EPG-1. Single Filter with Reverse Filtering Ports Enabled Single Contract Unidirectional with Multiple Filters . Default Bi-directional Contract with Reverse Filter Re​ sult: A sin​ gle con​ tract with (1) Sub​ ject and (1) Fil​ ter with a sin​ gle provider and a sin​ gle con​ sumer.244 Working with Contracts the most com​ mon of the use cases and al​ lows for a sin​ gle sub​ ject/fil​ ter to be im​ ple​ mented with a sin​ gle Provider/Con​ sumer re​ la​ tion​ ship. EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of www and EPG-2 is con​ sum​ ing the con​ tract. Sample Contract using Single Bi-directional Subject. www. In the use case below. EPG-1 is pro​ vid​ ing a ser​ vice to EPG-2.

When uti​ liz​ ing a sin​ gle sub​ ject with​ out the use of “Apply Both Di​ rec​ tions. This al​ lows the Host in EPG-1 to ac​ cess the Host in EPG-2 via icmp. Single Contract.” the user must then con​ fig​ ure two fil​ ters. In the use case below. Sample Single Contract. In this ex​ am​ ple. EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of icmp and EPG-2 is con​ sum​ ing the con​ tract. one in each di​ rec​ tion. Single Unidirectional Subject. When se​ lect​ ing this op​ tion the user no longer has the op​ tion to se​ lect the re​ verse fil​ ter op​ tion.Working with Contracts 245 Single Contract Unidirectional with Multiple Filters This use case in​ volves im​ ple​ ment​ ing a con​ tract with​ out the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions. Single Unidirectional Subject. icmp. Multiple Filters Re​ sult: A sin​ gle con​ tract with (1) Sub​ ject (2) Fil​ ters and a sin​ gle provider and a sin​ gle con​ sumer. Multiple Filters .

This al​ lows the end-user the most gran​ u​ lar​ ity when de​ ploy​ ing con​ tracts. and with​ out the op​ tion to apply the re​ verse fil​ ter. That is. This al​ lows the Web Client in EPG-2 to ac​ cess the Web Server in EPG-1. Each con​ tract will have a sin​ gle provider and a sin​ gle con​ sumer ref​ er​ enc​ ing the same con​ tract. Single Filters Re​ sult: Two con​ tracts with (1) Sub​ ject (1) Fil​ ters. but is also the most com​ pre​ hen​ sive. In the use case below. . EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of www and EPG-2 is con​ sum​ ing the con​ tract. EPG-1 is pro​ vid​ ing a ser​ vice to EPG-2.246 Working with Contracts Multiple Contracts Uni-Directional Single Filter This use case is use​ ful when im​ ple​ ment​ ing a con​ tract with the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions. Multiple Contracts. The dif​ fer​ ence here is that the con​ tract is ex​ plic​ itly ap​ plied in BOTH di​ rec​ tions. Unidirectional Subjects.

Working with Contracts 247 Sample Contract with Single Bi-Directional Subjects and single Filter .

.

249 Layer 4 to Layer 7 Services .

.

Layer 4 to Layer 7 Services 251 Section Content • Understanding Layer 4 to Layer 7 Integration Service Insertion Design Principles Applying Service Graphs to EPG Communications Rendering Service Graphs Integration Support Function • Services Deployment Guide Reference Import Device Package Create a Device Modify a Device Create Layer 4 to Layer 7 Service Graph Template Apply a Service Graph Template to EPGs • Service Graph Monitoring Monitoring a Service Graph Instance Monitoring and Resolving Service Graph Faults Monitoring a Virtual Device • ASAv Sample Configuration Verify ASAv VM configuration Remove GW IP from Relevant Fabric Bridge Domains Create a Layer 4 to Layer 7 Device Create a Layer 4 to Layer 7 Service Graph Template Apply the Service Graph Template Associate the Web and App Bridge domains to the Interfaces Verifying service node configuration Display the Access List configuration pushed-down from APIC Display the Interface configuration pushed-down from APIC Display the current Interface names .

.

they knew they would need to in​ cor​ po​ rate some ser​ vices. which can cause change man​ age​ ment to be​ come chal​ leng​ ing. IDS/IPS. Some pos​ si​ ble Layer 4 to Layer 7 ser​ vices in​ clude: • Firewalls • Load balancers • Traffic inspection appliances • SSL offload functions • Application flow acceleration functions In​ te​ grat​ ing ser​ vices with Cisco ACI Ser​ vice Graphs will pro​ vide ACME with the fol​ low​ ing ben​ e​ fits: • Policy based configuration management . Through the rest of this chap​ ter. As an ex​ am​ ple. The main pur​ pose of data cen​ ter fab​ ric equip​ ment is fast and ef​ fi​ cient for​ ward​ ing of traf​ fic from ingress to the fab​ ric. or other types of higher-layer ser​ vice de​ vice. load bal​ ancers. In tra​ di​ tional in​ fra​ struc​ ture. but can be​ come in​ flex​ ib ​le and frag​ ile. These sta​ tic con​ fig​ ur​ a​ tion blocks build up over time to cre​ ate a sit​ ua ​t​ ion where the con​ fig​ ur​ a​ tion works. One of the key tech​ nol​ ogy in​ no​ va​ tions in Cisco's Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) is pol​ icy-based man​ age​ ment of ser​ vice in​ ser​ tion through ap​ pli​ ca​ tion of a Ser​ vice Graph. Use​ ful in​ fra​ struc​ ture im​ ple​ men​ ta​ tions also uti​ lize this fast fab​ ric in a smart way to also in​ te​ grate Layer 4 to Layer 7 ser​ vices. de​ signed their new ap​ pli​ ca​ tion. such as fire​ walls. fire​ walls might be placed di​ rectly in​ line as a "bump in the line" or might be an ad​ junct ser​ vice de​ vice near a gate​ way.Layer 4 to Layer 7 Services 253 Understanding Layer 4 to Layer 7 Integration When ACME Inc. Fire​ walls are typ​ ic ​ally con​ fig​ ured per de​ vice by build​ ing up blocks of sta​ tic con​ fig​ ur​ a​ tion. ser​ vice in​ ser​ tion would re​ quire some in​ line de​ vice place​ ment or redi​ rec​ tion in order to get traf​ fic to the ser​ vice de​ vices. we will dis​ cuss the high level overview of how the process works and how ACME will uti​ lize Ser​ vice Graphs for ef​ fi​ cient man​ age​ ment of Layer 4 to Layer 7 ser​ vices. be​ tween phys​ ic ​al and vir​ tual hosts within the fab​ ric and egress back out of the fab​ ric.

Ser​ vices ap​ pli​ ances can be phys​ ic ​al or vir​ tual and can be con​ nected to any leaf under man​ age​ ment of the ACI fab​ ric. Applying Service Graphs to EPG Communications To allow com​ mu​ ni​ ca​ tions be​ tween EPGs within an ACI fab​ ric. The Ser​ vice Graph then ties the con​ tract to the re​ solved ser​ vice de​ vice with the pol​ icy-based con​ fig​ ur​ a​ tion in place. a con​ tract must be put in to place. Where ap​ plic​ ab ​le. This con​ tract es​ sen​ tially con​ trols com​ mu​ ni​ ca​ tions flow be​ tween EPGs. . phys​ ic ​al ap​ pli​ ances can also be run in mul​ ti​ ple con​ text mode.254 Layer 4 to Layer 7 Services • Flexible configuration state abstraction through the ACI object model • Integrated configuration management using the APIC GUI. With vir​ tual ser​ vices ap​ pli​ ances. traf​ fic flow to/from ser​ vice ap​ pli​ ances is man​ aged very ef​ fi​ ciently. This con​ tract may take the form of a spe​ cific con​ sumer/provider re​ la​ tion​ ship de​ fined by spec​ if​ ied pro​ to​ col and port. and thus. cur​ rently only VMware Hy​ per​ vi​ sors and VLAN trans​ port modes are sup​ ported. There could also be an "allow all" con​ tract that al​ lows com​ pletely open com​ mu​ ni​ ca​ tions. al​ low​ ing multi-ten​ ant map​ ping of fab​ ric for​ ward​ ing and ten​ ant-spe​ cific ser​ vice con​ fig​ ur​ a​ tions. and can be ex​ tended to in​ clude ser​ vice in​ ser​ tion via at​ tach​ ment of a Ser​ vice Graph to a con​ tract. REST API or Python scripts. the ap​ pli​ ances do not need to be placed at any spe​ cific lo​ ca​ tion within the net​ work fab​ ric. all based on a consistent ACI object model • Complex topology modeling with logical flow stitching allowing abstracted links • Policy-based provisioning allowing rapid complex topology deployment • Configuration synchronization allowing dynamic workload provisioning and de- between multiple service devices provisioning without manual intervention • Application centric template-based configuration management and object • Infrastructure multi-tenancy within the fabric and the service devices reuse to shorten infrastructure implementation timelines Service Insertion Design Principles With the spine-leaf ar​ chi​ tec​ ture and holis​ tic fab​ ric man​ age​ ment as​ pects of ACI.

As it re​ lates to the Layer 4 to Layer 7 ser​ vices. and reused. ports. these ob​ jects ex​ press the in​ tent of use for that ob​ ject in re​ la​ tion to the ap​ pli​ ca​ tion. ob​ ject-groups. The APIC also sets up the net​ work for​ ward​ ing path to make sure the cor​ rect for​ ward​ ing ac​ tion is taken to get traf​ fic flow to the ser​ vice nodes for treat​ ment. such as Python or a RESTful API post through POSTman These pol​ icy ob​ jects can be cre​ ated. sub​ se​ quently reusing the tem​ plates is very straight​ for​ ward sim​ ply by re​ plac​ ing IP ad​ dresses. ex​ posed func​ tions. These de​ vice pack​ ages carry the de​ vice de​ scrip​ tion. reused and repack​ aged as nec​ es​ sary. ACME has a few cookie cut​ ter tem​ plates for fire​ wall and load-bal​ anc​ ing ser​ vices. The ren​ der​ ing in​ volves al​ lo​ ca​ tion of the nec​ es​ sary bridge do​ mains. the ser​ vice graph ob​ jects are then trans​ lated into spe​ cific de​ vice con​ fig​ ur​ a​ tions that gets pushed the ser​ vice nodes through a process called ren​ der​ ing. This ab​ stracted process of con​ fig​ ur​ a​ tion man​ age​ ment works like a pol​ icy tem​ plate where you can de​ fine the ex​ pected be​ hav​ ior. . then link two groups and sub​ ject their re​ la​ tion​ ship to that pol​ icy. Though the ini​ tial de​ f​ in ​i​ t​ ion of these tem​ plates can be po​ ten​ tially cum​ ber​ some. the ACI fab​ ric has full un​ der​ stand​ ing of what a de​ vice can do. This pol​ icy can be copied. cre​ ation of the VLAN on these de​ vices to cre​ ate the path for the func​ tions. When the de​ vice pack​ age is im​ ported. Integration Support Function In the ACI model.Layer 4 to Layer 7 Services 255 Rendering Service Graphs The ACI al​ lows users to de​ fine a pol​ icy using the fol​ low​ ing ways: • APIC GUI • API with a programmatic tool. con​ fig​ ur​ a​ tion of IP ad​ dresses on the fire​ wall and load bal​ ancer in​ ter​ faces. how it con​ nects to the fab​ ric. and other val​ ues. and per​ for​ mance of all the work nec​ es​ sary to make sure that the path be​ tween EPGs is the path de​ fined in the ser​ vice graph. As is the case with many cus​ tomers. ma​ nip​ ul​ ated. When an ap​ pli​ ca​ tion pro​ file is de​ ployed and end​ points are at​ tached to the leaf switches. and have con​ fig​ ur​ a​ tion script con​ tent. com​ mu​ ni​ ca​ tions with the ser​ vice de​ vices is sup​ ported by im​ port​ ing de​ vice pack​ ages.

​ cisco.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ unified-fabric/​ aci_​ ecosystem. Some of the De​ vice Pack​ ages can be down​ loaded at: http://​ www. An up to date list​ ing of part​ ners that lever​ age the API are avail​ able at: http://​ www.256 Layer 4 to Layer 7 Services how to build path for​ ward​ ing to bring traf​ fic into and get traf​ fic back from the de​ vice.​ html . and how to trans​ late pol​ icy in​ tent to a de​ vice-spe​ cific con​ fig​ ur​ a​ tion.​ html De​ vice pack​ ages from mul​ ti​ ple ven​ dors that lever​ age the rich open API that ACI pro​ vides are avail​ able from ven​ dors such as Cit​ rix and F5 at the time of writ​ ing of this book.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centricinfrastructure/​ solution-overview-c22-732445. This de​ vice pack​ age is a ven​ dor-de​ vel​ oped pack​ age that is read​ ily avail​ able from the orig​ in ​al ven​ dor or from Cisco on the soft​ ware down​ load page.

Layer 4 to Layer 7 Services 257 Services Deployment Guide Reference The di​ a​ gram below shows a high level overview of the Layer 4 to Layer 7 ser​ vices work​ flow when at​ tempt​ ing to in​ te​ grate a de​ vice. Create a Device Once the de​ vice pack​ age is im​ ported.​ html. ACME will im​ port the ven​ dor/model-spe​ cific de​ vice pack​ ages. the de​ vices will be added through a process of cre​ at​ ing a log​ i​ cal de​ vice clus​ ter and cre​ at​ ing a re​ la​ tion​ ship be​ tween this log​ i​ cal de​ vice and the phys​ i​ cal ap​ pli​ ance. The key sec​ tions of the De​ ploy​ ment Guide are listed below: Import Device Package To begin the process of Ser​ vice Node in​ te​ gra​ tion.​ cisco. but are very sim​ i​ lar. as shown in this sec​ tion of the De​ ploy​ ment Guide. The con​ fig​ u​ ra​ tion steps dif​ fer slightly for phys​ i​ cal de​ vices and vir​ tual de​ vices. This can be done with a phys​ i​ cal or VM de​ vice. see the Cisco APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide: http://​ www.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy. . L4-L7 Workflow For in​ for​ ma​ tion about de​ ploy​ ing Layer 4 to Layer 7 ser​ vices.

258 Layer 4 to Layer 7 Services Modify a Device You can mod​ ify a de​ vice's con​ fig​ ur​ a​ tion through the GUI as de​ scribed in this sec​ tion of the De​ ploy​ ment Guide. . Apply a Service Graph Template to EPGs Once the ap​ pli​ ca​ tion End​ point Groups (EPGs) have been cre​ ated. the process to apply a ser​ vice graph tem​ plate to EPGs can be found in this sec​ tion of the De​ ploy​ ment Guide. Create Layer 4 to Layer 7 Service Graph Template This sec​ tion of the De​ ploy​ ment Guide ex​ plains how to cre​ ate a ser​ vice graph.

choose the tenant. Func​ tions of a Graph in​ stance. For fur​ ther de​ tails of the pos​ si​ ble states and other rel​ e​ vant states. in​ clud​ ing: • Monitoring a Service Graph Instance • Monitoring and Resolving Service Graph Faults • Monitoring a Virtual Device Monitoring a Service Graph Instance Once a ser​ vice graph is con​ fig​ ured and as​ so​ ci​ ated with a con​ tract that is at​ tached to an EPG. To mon​ it​ or a ser​ vice graph in​ stance: 1 On the menu bar. there are some pri​ mary mon​ it​ or​ ing as​ pects that should be con​ sid​ ered: State of the Ser​ vice Graph. choose Tenant_Name > L4-L7 Services > Deployed Graph Instances. and the current state of the graph policy. see the Cisco APIC vices De​ ploy​ ment Guide at: http://​ www. the re​ quire​ ment of ob​ serv​ ing the Ser​ vice Graph be​ comes an op​ er​ at​ ional im​ per​ at​ ive. the associated contracts. 3 In the Navigation pane. To sup​ port these ef​ forts.​ cisco. re​ sources al​ lo​ cated to a func​ tion and pa​ ra​ me​ ters spec​ if​ ied for a func​ tion. choose Tenants > ALL TENANTS. including a list of the deployed service graphs. A state of "Applied" means the graph has been applied and is active in the fabric and the service device.​ com/​ c/​ en/​ us/​ td/​ Layer 4 to Layer 7 Ser​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy/​ b_​ L4L7_​ Deploy_​ chapter_​ 01010. The Work pane displays information about the deployed graph instances. there are a few tech​ niques that can be em​ ployed.Layer 4 to Layer 7 Services 259 Service Graph Monitoring Once ACME Inc. 2 In the Work pane. has de​ ployed Ser​ vice Graphs for ap​ pli​ ca​ tion ser​ vice in​ ser​ tion.​ html#​ task_​ F2BFF7545D9142EFB208C10F5DFBB1B4 .

2 In the Work pane. 3 In the Navigation pane. The Work pane displays the faults that are related to the active service graph. 3 In the Navigation pane. choose the tenant. 2 In the Work pane. To mon​ it​ or a vir​ tual de​ vice: 1 On the menu bar. the func​ tions in use and the func​ tion pa​ ra​ me​ ters passed to the de​ vices. which VLANs are con​ fig​ ured for a de​ vice clus​ ter.​ html#​ concept_​ 307C0CA3EB57469EAF7EF87AAE5A240F Monitoring a Virtual Device After you con​ fig​ ure a ser​ vice graph and at​ tach the graph to an end​ point group (EPG) and a con​ tract. see Ta​ bles 1 and 2 in the Cisco APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide at: http://​ www. choose the tenant. 4 In the Work pane. and the health of the de​ vices in a de​ vice clus​ ter. .​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy/​ b_​ L4L7_​ Deploy_​ chapter_​ 01010. such as faults and health scores. To un​ der​ stand the faults listed and pos​ si​ ble res​ o​ lu​ tions. choose Tenant_Name > L4-L7 Services > Virtual Devices. the sta​ tis​ tics from the de​ vices. choose Tenant_Name > L4-L7 Services > Deployed Graph Instances. choose Tenants > ALL TENANTS.260 Layer 4 to Layer 7 Services Monitoring and Resolving Service Graph Faults To mon​ it​ or a ser​ vice graph's faults: 1 On the menu bar. choose the Faults tab. choose Tenants > ALL TENANTS. The Work pane displays information about the virtual devices. you can mon​ it​ or the vir​ tual de​ vices as​ so​ ci​ ated with the ser​ vice graphs of a ten​ ant.​ cisco. Mon​ it​ or​ ing the vir​ tual de​ vices tells you what de​ vice clus​ ters are as​ so​ ci​ ated.

5 Verify connectivity from the APIC to the management 0/0 interface of the ASAv. SSH to the APIC cluster IP address b. Issue the following command: # ping {ASAv management 0/0 interface} .Layer 4 to Layer 7 Services 261 ASAv Sample Configuration One of the ser​ vice nodes that ACME Inc. will in​ te​ grate is an ASAv de​ ployed in sin​ glenode. a. As a routed node. the ASAv will be​ come the de​ fault gate​ way for the hosts in the 2 EPGs that are con​ nected by the con​ tract that this ser​ vice graph gets as​ so​ ci​ ated to. routed FW mode. This ex​ am​ ple de​ tails the process they fol​ lowed to con​ fig​ ure the ASAv fire​ wall vir​ tual ser​ vice ap​ pli​ ance as a sin​ gle node in routed mode. 3 Issue the following command: # show ip int brief 4 Verify that the ASAv has a correct IPv4 address on the management 0/0 interface. 2 Enter enable mode. The high level steps are: • Create Logical and Concrete device clusters • Define a firewall ruleset/graph • Deploy the graph between 2 EPGs Verify ASAv VM configuration The first set of steps per​ formed is to ver​ ify ASAv VM con​ fig​ ur​ a​ tion and up​ load the ASA de​ vice pack​ age (or ver​ ify if it has al​ ready been done) 1 Log in to the ASAv VM.

4 In the Work pane. 2 In the Work pane. Choose Import Device Package. click Flood. choose Tenants > ALL TENANTS. c.10: icmp_seq=1 ttl=64 time=0. For the L2 Unknown Unicast radio buttons. then perform the following actions: a. Remove GW IP from Relevant Fabric Bridge Domains Once the ASAv pack​ age and VM are ver​ if​ ied. To re​ move the rout​ ing de​ fault gate​ way func​ tion on the EPG1 and EPG2 bridge do​ mains: 1 On the menu bar. 6 In the APIC GUI. Uncheck the Unicast Routing check box. Create a Layer 4 to Layer 7 Device . choose Tenant_Name > Networking > Bridge Domains > BD-EPG1. choose the tenant. Re​ peat this process for the Bridge Do​ mains of the af​ fected EPGs.16. 8 If an ASA device package is not listed. Click Submit. on the menu bar.050 ms If the re​ sponse is dif​ fer​ ent. the next step is to re​ move SVI/GW IP ad​ dresses from the fab​ ric Bridge Do​ mains so the Layer 3 routed fire​ wall can be​ come the de​ fault gate​ way. perform the following actions: a. Follow the prompts to upload the device package. choose Tenant > Tenant_Name. 3 In the Navigation pane. b. b. The return response should be similar to the following example: 64 bytes from 172. c.262 Layer 4 to Layer 7 Services c. d. then there is likely some sort of con​ nec​ tiv​ ity issue. For the L3 Unknown Multicast Flooding radio buttons.10. 7 In the Navigation Pane expand Tenant_Name > L4-L7 Services > Packages > L4-L7 Service Device Types. Ad​ dress the con​ nec​ tiv​ ity prob​ lems be​ fore mov​ ing on. Right click on the Device Types. click Flood.

In the Feature and Parameters section. choose Tenants > ALL TENANTS.4j as needed. g. In the Profile drop-down list. 7 Verify the external and internal interfaces IPv4 addresses. i. Click SUBMIT. choose the tenant. Enter a meaningful name. In the Create L4-L7 Services Function Profile Group dialog box. 3 In the Navigation Pane choose Tenant_Name > L4-L7 Services. h. e.Layer 4 to Layer 7 Services 263 Create a Layer 4 to Layer 7 Device Per​ form the fol​ low​ ing steps to add log​ ic ​al and con​ crete de​ vice clus​ ters to a ten​ ant: 1 On the menu bar. choose WebServiceProfileGroup or WebPolicyForRoutedMode. Expand Interface Related Configuration for externalIf.n.b. choose the All Parameters tab. ii. In the Profile Group drop-down list. choose the tenant. k.d/m. choose Create Function Profile Group. 5 In the Create a L4-L7 Function Profile dialog box. perform the following actions: i.o. such as "TXX-FP-Group". perform the following actions: a. Repeat steps 4f .p format j. such as "TXX-ASAvFP". Expand Interface Specific Configuration. d. click Create a L4-L7 Function Profile. choose Tenants > ALL TENANTS. Enter: ipv4 address in a. 2 In the Work pane. b. . 4 In the Work pane. f. Click Update. You will configure IP addressing under Interface Related Configuration for both external and internal interfaces (externalIf and interalIf). Enter a name that is relevant to the tenant and fuction. c. Double-click IPv4 Address.c. The fol​ low​ ing steps will cre​ ate a Layer 4 to Layer 7 de​ vice: 1 On the menu bar. 2 In the Work pane. 6 Choose L4-7 Services > Function Profiles > ALL PARAMETERS.

Management Port: https iii. VM: Tenant ASAv Controller (in the dropdown box) iv. 3 In the Navigation Pane choose Tenant_Name > L4-L7 Services > L4-L7 Devices. Management IP address: ASAv IP address ii. Device Package: CISCO-ASA-1. Connectivity VMM Domain: Txx-vCenter f. 2 In the Work pane.264 Layer 4 to Layer 7 Services 3 In the Navigation pane. Click NEXT. Function Type: Goto e. Enter a meaningful name: Txx-ASAv-Cluster b. Click FINISH. Virtual Interfaces: Create two entries. choose Tenant_Name > L4-L7 Services. APIC to Device: Out of Band g. 5 Select TXX-ASAv-Cluster_Device_1 to see a graphic view of the concrete device. click Create a L4-L7 virtual device. Name: GigabitEthernet0/0 vNIC: Network Adapter 2 Direction: provider 2. Ver​ ify the Log​ ic ​al and Con​ crete De​ vice Clus​ ters have been con​ fig​ ured: 1 On the menu bar. click + twice. choose the tenant. vii. Under Device 1.1 Model: ASAv c. specify the following values: i. 4 Expand both TXX-ASAv-Cluster and TXX-ASAv-Cluster_Device_1 to view the logical and physical interfaces. 5 In the Create L4-L7 Devices dialog box. vi. choose Tenants > ALL TENANTS. . Click UPDATE after each entry. enter interface values accordingly: 1. 4 In the Work pane. Mode: Single Node d. Credentials: {uid/pwd} h. Name: GigabitEthernet0/1 vNIC: Network Adapter 3 Direction: consumer v. perform the following actions: a.

6 You can verify that the template was created successfully by expanding Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates > Txx-ASAvL3-Routed-Template > Function Node . 5 In the Apply L4-L7 Service Graph Template to EPGs dialog box. perform the following actions: a. d. In the Profile drop-down list. perform the following actions: a. 2 In the Work pane. choose the tenant. choose Tenants > ALL TENANTS. choose EPG2. choose EPG1. choose Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates. choose Single Node .1/Firewall. choose Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates. 3 In the Navigation pane. 4 Right click Txx-ASAv-L3-Routed-Template and choose Apply L4-L7 Service Graph Template. Create a Layer 4 to Layer 7 Service Graph Template 1 On the menu bar.Firewall in Routed Mode.Firewall. 4 In the Work pane. which is the functional profile you created previously. choose Tenants > ALL TENANTS. 3 In the Navigation pane. Click Submit. In the Name field. choose TXX-ASAv-FP. b. . e. 2 In the Work pane. b. In the Provider EPG / External Network drop-down list. enter “TXX-ASAv-L3-Routed-Template”. choose CISCO-ASA-1. choose Actions > Create a L4-L7 Service Graph Template. Apply the Service Graph Template 1 On the menu bar.Layer 4 to Layer 7 Services 265 Note This com​ pletes the Log​ ic ​al and Con​ crete de​ vice cre​ ation. c. In the Consumer EPG / External Network drop-down list. choose the tenant. 5 In the Create L4-L7 Devices dialog box. In the Type drop-down list. In the Device Function drop-down list.

Double click Device Config > Interface Related Configuration externalIf > Access Group > Inbound Access List. g. In the Contract Name field. In the L4-L7 Devices drop-down list. choose Txx-ASAv-Cluster. 4 Assign the Bridge Domain to BD-EPG1. k. 3 Select External. Click Finish. q. j. This inbound access list is not desirable for the lab traffic-flow. d. e. You can see that the Consumer and Provider EPGs are associated with the EPG1 and EPG2 server EPGs. 6 Choose TXX-EPG1-to-EPG2-TXX-ASAv-L3-Router-Template-TXXProduction. The Devices Selection Polices folder and Deployed Graph Instances folder are now populated. l. In the Value drop-down list. Check the No Filter (Allow All Traffic) check box. m. p. . Click Update.266 Layer 4 to Layer 7 Services c. For the Contract radio buttons. Double click Device Config > Interface Related Configuration internalIf > Access Group > Inbound Access List. If the mask is missing. n. o. Associate the Web and App Bridge domains to the Interfaces 1 Expand Devices Selection Policies. Click Next. In the Features and Parameters section. In the Service Graph Template drop-down list. Click Reset. the configuration will not push to the ASA. enter “TXX-EPG1-to-EPG2”. h. choose Txx-ASAv-L3Routed-Template. f. i. This unassigns the inbound access list from the internal interface. click Create A New One. Expand Device Config > Interface Related Configuration externalIf > Interface Related Configuration to verify the IP address assignment. Expand Device Config > Interface Related Configuration internalIf > Interface Specific Configuration to verify the IP address assignment with the mask. choose the All Parameters tab. 2 Expand TXX-EPG1-to-EPG2-TXX-ASAv-L3-Router-Template-Firewall. choose access-list-inbound.

8 Expand TXX-ASAv-L3-Router-Template. 14 Filters: common/default. Issue the fol​ low​ ing com​ mand: # show run | grep access This will show the ac​ cess list con​ fig​ ur​ a​ tion and you can re​ late that to the con​ fig​ ur​ a​ tion that was done in the APIC. 7 Expand Layer 4 to Layer 7 Service Graph Templates. Display the Access List configuration pushed-down from APIC SSH to the ASAv ser​ vice node for ac​ cess list val​ id ​a​ tion. The ASAv will now have IP ad​ dresses asigned in the Ser​ vice Graph Pro​ file. 10 Filters: common/default. 11 CTX Terms: TXX-ASAv-L3Routed Template/T1/Out. 16 Click SUBMIT. .Layer 4 to Layer 7 Services 267 5 Repeat the process to assign the Bridge Domain to BD-EPG2. 6 Click SUBMIT. 12 Click SUBMIT. 9 Expand Function Node – Firewall > choose external. This can be ver​ if​ ied by going into the ASAv VM con​ sole and issue the fol​ low​ ing com​ mand: # show ip int brief Verifying service node configuration Once the De​ vice has been in​ te​ grated and the Ser​ vice Graph has been con​ fig​ ured and as​ so​ ci​ ated. 15 CTX Terms: TXX-ASAv-L3Routed Template/T1/In. 13 Repeat the process for internal. the re​ sult​ ing con​ fig​ ur​ a​ tion pushed to the ser​ vice node can be ver​ if​ ied by sim​ ple show com​ mands on the real ser​ vice node de​ vice.

Issue the fol​ low​ ing com​ mand: # show nameif This will show the in​ ter​ face names pushed from the APIC and will show the re​ lated in​ ter​ face names to the log​ ic ​al in​ ter​ face names that were con​ fig​ ured in the APIC above. . Issue the fol​ low​ ing com​ mand: # show run interface This will show the in​ ter​ face con​ fig​ ur​ a​ tion where the IP ad​ dress con​ fig​ ur​ a​ tion was pushed from the APIC.268 Layer 4 to Layer 7 Services Display the Interface configuration pushed-down from APIC SSH to the ASAv ser​ vice node for in​ ter​ face con​ fig​ ur​ a​ tion val​ id ​a​ tion. Display the current Interface names SSH to the ASAv ser​ vice node for in​ ter​ face name con​ fig​ ur​ a​ tion val​ id ​a​ tion.

269 Health Scores .

.

Health Scores 271 Section Content • Understanding Health Scores • Understanding Faults • How Are Health Scores Calculated • Health Score Use Cases Using Health Scores for Proactive Monitoring Using Health Scores for Reactive Monitoring .

.

and cor​ re​ lated by the APIC in real time and then pre​ sented in an un​ der​ stand​ able for​ mat. all health scores are in​ stan​ ti​ ated from the health​ Inst class and can be ex​ tracted through the API. can be rep​ re​ sented in a sin​ gle con​ sol​ id ​ated health score. The ob​ ject model at the heart of ACI is in​ her​ ent to the in​ fra​ struc​ ture. It is worth not​ ing that while pro​ vid​ ing such an​ swers may be easy as it re​ lates to an in​ de​ pen​ dent de​ vice or link. which pro​ vides Health Scores that make in​ for​ ma​ tion on sta​ tus. or any sub​ set of the sys​ tem. per​ for​ mance. To man​ ua ​lly col​ lect and cor​ re​ late in​ for​ ma​ tion would have pre​ vi​ ously been a long and te​ dious task. and there​ fore the cur​ rent sta​ tus of all of the ob​ jects in​ clud​ ing links.Health Scores 273 Understanding Health Scores ACME's Op​ er​ at​ ions team has been chal​ lenged on a reg​ ul​ ar basis to an​ swer basic ques​ tions re​ gard​ ing the cur​ rent sta​ tus. and avail​ abil​ ity of the sys​ tem they are re​ spon​ si​ ble for op​ er​ at​ ing. their re​ la​ tion​ ships. The vis​ ib ​il​ ity pro​ vided by health scores give the op​ er​ at​ or a quick at-a-glance as​ sess​ ment of the cur​ rent sta​ tus of the en​ tire sys​ tem. as well as the real time sta​ tus of their uti​ liza​ tion. per​ for​ mance. This vis​ i-​ bil​ ity has a num​ ber of prac​ ti​ cal use cases. de​ vices. but with health scores. To ad​ dress these chal​ lenges they can now uti​ lize the Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI). . and de​ scribe the re​ la​ tion​ ship be​ tween the var​ io ​us de​ vices and links in at​ tempt to pro​ vide a cor​ re​ la​ tion. this in​ for​ ma​ tion by it​ self is of lit​ tle to no value with​ out ad​ di​ tional data on its ef​ fect on the over​ all health of the net​ work. and how var​ io ​us faults im​ pact the cal​ cu​ la​ tion of the health score. which can be found from the Dash​ board or Pol​ icy tabs of the ob​ ject from the GUI. Tra​ di​ tional net​ work mon​ it​ or​ ing and man​ age​ ment sys​ tems at​ tempt to pro​ vide a model of in​ fra​ struc​ ture that has been pro​ vi​ sioned. data through​ out the fab​ ric is col​ lected. and in this chap​ ter we will clas​ sify these use cases as re​ ac​ tive and proac​ tive. and avail​ abil​ ity read​ ily avail​ able. Most ob​ jects in the model will have an as​ so​ ci​ ated health score. com​ puted. ACI also pro​ vides the flex​ ib ​il​ ity to mon​ it​ or some as​ pects of how the health scores are cal​ cu​ lated. Ad​ di​ tion​ ally.

if you nav​ i​ gate to the ap​ pli​ ca​ tion pro​ file it has a Health tab. ACI health scores pro​ vide a quick check as to whether an issue being re​ ported is con​ firmed in a degra​ da​ tion of the health score. ap​ pli​ ca​ tions. .274 Health Scores Tenant Health Score In a re​ ac​ tive ca​ pac​ ity. the root cause of the issue can be found by ex​ plor​ ing the faults and how these get rolled up in the larger model. Health scores also pro​ vide a real-time cor​ re​ la​ tion in the event of a fail​ ure sce​ nario. and EPGs are im​ pacted by that fail​ ure. im​ me​ di​ ately pro​ vid​ ing feed​ back as to which ten​ ants. If so. In this tab is a tree that will show the var​ i​ ous ob​ jects in a tree form to re​ veal faults. As a ex​ am​ ple.

In​ stead the health score should be seen as a met​ ric that will change over time.Health Scores 275 Object with a fault Proac​ tively. Links. and end​ points have fail​ ures. with the goal of in​ creas​ ing the av​ er​ age health score of a given set of com​ po​ nents over time. how​ ever. . equip​ ment. and other ca​ pac​ ity plan​ ning ex​ er​ cises. Ide​ ally. ACI health scores can help iden​ tify po​ ten​ tial bot​ tle​ necks in terms of hard​ ware re​ sources. Op​ er​ a​ tions teams also stand a bet​ ter chance of iden​ ti​ fy​ ing is​ sues be​ fore they im​ pact cus​ tomers or users. this is not al​ ways re​ al​ is​ tic given the dy​ namic na​ ture of data cen​ ter en​ vi​ ron​ ments. the health of all ap​ pli​ ca​ tion and in​ fra​ struc​ ture com​ po​ nents would al​ ways be at 100%. band​ width uti​ liza​ tion.

.

In this model. Faults. 2 Telemetry device . in​ clud​ ing faults. managed and applied. Fault Lifecycle . state​ ful. Virtual Switches. It maintains a comprehensive. All ob​ jects within ACI can be queried. up-to-date run-time representation of the administrative or configured state. and per​ sis​ tent MO. events. integrated Layer 4 to Layer 7 devices) in an ACI fabric report faults.All devices (Fabric Switches. a fault is rep​ re​ sented as a mu​ ta​ ble. and sta​ tis​ tics in the ACI fab​ ric are rep​ re​ sented as a col​ lec​ tion of Man​ aged Ob​ jects (MOs) within the over​ all ACI Ob​ ject Model/Man​ age​ ment In​ for​ ma​ tion Tree (MIT). events and statistics to the APIC.Health Scores 277 Understanding Faults From a man​ age​ ment point of view we look at the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) from two per​ spec​ tives: 1 Policy Controller .Where all fabric configuration is created.

they have a par​ ent. The Fault “code” is an al​ phanu​ mer​ ic ​al string in the form FXXX. only one fault is raised while a counter for the re​ cur​ rence of that fault will be in​ cre​ mented. There can be at most one fault with a given code under an MO. and so on.xml?query-target=self&rspsubtree-include=faults As you can see. such as a com​ po​ nent fail​ ure or an alarm. For more in​ for​ ma​ tion about fault codes. Man​ The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the health score for a ten​ ant named "3tier​ app": https://hostname/api/node/mo/uni/tn-3tierapp. and deleted by the sys​ tem as spe​ cific con​ di​ tions are de​ tected. In most cases.278 Health Scores When a spe​ cific con​ di​ tion oc​ curs. with prop​ erty fil​ ters. if the same con​ di​ tion is de​ tected mul​ ti​ ple times for the same af​ fected ob​ ject. es​ ca​ lated. a fault MO is au​ to​ mat​ ic ​ally cre​ ated. To re​ move a fault. A fault MO re​ mains in the sys​ tem until the fault con​ di​ tion is cleared. the fault con​ di​ tions are de​ fined by the fault rules of the par​ ent ob​ ject class.xml?query-target=self&rsp-subtreeinclude=stats The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the faults for a leaf node: https://hostname/api/node/mo/topology/pod-1/node-103. a DN. RN. MOs can be queried by class and DN. see the Cisco APIC Faults. If the same con​ di​ tion is de​ tected mul​ ti​ ple times while the cor​ re​ spond​ ing fault MO is ac​ tive. In other words. pag​ i​ na​ tion.xml?query-target=self&rsp-subtreeinclude=health The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the sta​ tis​ tics for a ten​ ant named "3tier​ app": https://hostname/api/node/mo/uni/tn-3tierapp. Fault MOs ap​ pear as reg​ ul​ ar MOs in MIT. Events. whether by con​ fig​ ur​ a​ - . and Sys​ tem Mes​ sages age​ ment Guide. no ad​ di​ tional in​ stances of the fault MO are cre​ ated. the sys​ tem cre​ ates a fault MO as a child ob​ ject to the MO that is pri​ mar​ ily as​ so​ ci​ ated with the fault. and so on. de-es​ ca​ lated. For a fault ob​ ject class. the con​ di​ tion rais​ ing the fault must be cleared.

An ex​ cep​ tion to this is if the fault is in the cleared or re​ tained state.Health Scores 279 tion. Sever​ ity pro​ vides an in​ di​ ca​ tion of the es​ ti​ mated im​ pact of the con​ di​ tion on the ca​ pa​ bil​ ity of the sys​ tem or com​ po​ nent to pro​ vide ser​ vice. you can set fault thresh​ olds on sta​ tis​ ti​ cal mea​ sure​ ments such as health scores. some of which are user configurable For ex​ am​ ple. . in which case the fault can be deleted by the user by ac​ knowl​ edg​ ing it. or tem​ per​ at​ ures. data traf​ fic. Pos​ si​ ble val​ ues are: • Warning (possibly no impact) • Minor • Major • Critical (system or component completely unusable) The cre​ ation of a fault MO can be trig​ gered by in​ ter​ nal processes such as: • Finite state machine (FSM) transitions or detected component failures • Conditions specified by various fault policies. or a change in the run time state of the fab​ ric.

.

it will only be weighted by the end points that are in​ cluded in that ten​ ant. as well as an over​ all health score for the over​ all sys​ tem. di​ vided by the total num​ ber of learned end points. mul​ ti​ plied by the spine co​ ef​ fi​ cient which is de​ rived from the num​ ber of spines and their health scores. ten​ ants. All health scores are cal​ cu​ lated using the num​ ber and im​ por​ tance of faults that apply to it.Health Scores 281 How Health Scores Are Calculated Health scores exist for sys​ tems and pods. Crit​ i​ cal faults might have a . In other words: Health Score calculation Ten​ ant health scores are sim​ i​ lar. man​ aged ob​ jects (such as switches and ports). but con​ tain health scores of log​ i​ cal com​ po​ nents within that ten​ ant. Each fault is weighted de​ pend​ ing on the level of im​ por​ tance. Sys​ tem and pod health scores are a weighted av​ er​ age of the leaf health scores. You can see how all of these scores are ag​ gre​ gated by look​ ing at how man​ aged ob​ ject scores are cal​ cu​ lated. For ex​ am​ ple. which is di​ rectly by the faults they have as​ so​ ci​ ated with them.

Luck​ ily there is re​ ally no need to un​ der​ stand the cal​ cu​ la​ tions of the health scores to use them ef​ fec​ tively. see the Trou​ bleshoot​ ing Cisco Ap​ pli​ ca​ tion tric In​ fra​ struc​ ture book: http://​ datacenter. but a ten​ ant admin would only be able to see the health scores that per​ tain to the ten​ ants to which they have ac​ cess. but there should be a basic un​ der​ stand​ ing of whether faults should have high. In this case the fab​ ric ad​ min​ is​ tra​ tor may have to start trou​ bleshoot​ ing. such as fire​ walls. Faults that have been iden​ ti​ fied as not im​ pact​ ing might even be re​ as​ signed a per​ cent​ age value of 0% so that it does not af​ fect the health score com​ pu​ ta​ tion. a fab​ ric admin will be able to see all health scores. it is pos​ si​ ble to change these val​ ues to bet​ ter match your en​ vi​ ron​ ment.282 Health Scores high fault level at 100%. load bal​ ancers. pod. Keep in mind. along with faults within our VMM do​ mains will all roll up into our ten​ ant. while warn​ ings might have a low fault level at only 20%. low. For more in​ for​ ma​ tion on how to use faults. or “none” fault lev​ els. but it is pos​ si​ ble a fault may be oc​ cur​ ring that is af​ fect​ ing more than that one ten​ ant. and in​ tru​ sion pre​ ven​ tion/de​ tec​ tion sys​ tems.​ Cen​ . Though faults in ACI come with de​ fault val​ ues. medium. These.​ io/​ aci-troubleshooting-book/. For ex​ am​ ple. and over​ all sys​ tem health scores. be​ cause of the role-based ac​ cess con​ trol.​ github. not all ad​ min​ is​ tra​ tors will be able to see all of the health scores. The ten​ ant and fab​ ric ad​ mins may also see health scores of any layer four through seven de​ vices. In most cases. the ten​ ant admin should be able to drill into the health scores that are vis​ ib ​le to them.

If you see that one of the leaf switches is at 100% (green for good) one week. health scores can be used to di​ ag​ nose prob​ lems with the ACI fab​ ric. it is pos​ si​ ble the links are over​ sub​ scribed and so it can be time to ei​ ther move some of of the work​ load to an​ other leaf or maybe to add more band​ width by con​ nect​ ing more ca​ bles. VMware ad​ min​ is​ tra​ tors. Upon no​ ti​ fi​ ca​ tion that a health score has been de​ graded. As shown in the above ex​ am​ ples. or maybe even op​ ti​ miz​ ing the rules may be needed to make traf​ fic flow more ef​ fi​ cient. . and so on. Using Health Scores for Reactive Monitoring Re​ ac​ tively. Once the root cause faults have been iden​ ti​ fied. or fire​ wall. there is time to re​ solve the issue be​ fore any bot​ tle​ necks on the net​ work are no​ tice​ able. This would pro​ vide mon​ i-​ tor​ ing of the en​ vi​ ron​ ment across the net​ work that has not pre​ vi​ ously been avail​ able and which is not able to be re​ trieved by any other means. In these cases adding an​ other load bal​ ancer. and the next week the leaf is show​ ing a warn​ ing. and their as​ so​ ci​ ated faults. Other ways health scores can be used to proac​ tively mon​ it​ or your ACI en​ vi​ ron​ ment are by giv​ ing vis​ ib ​il​ ity of cer​ tain com​ po​ nents to other groups. they are es​ sen​ tially base​ lines to which you can make com​ par​ isons later. ACI health scores will allow them to start pre​ vent​ ing is​ sues. Health scores not only act as in​ di​ ca​ tors of faults. you can drill down to see what changed. This pro​ vides the abil​ ity to “dou​ ble-click to root cause”. it is pos​ si​ ble to send these no​ ti​ fi​ ca​ tions to ap​ pli​ ca​ tion own​ ers. Data​ base Ad​ min​ is​ tra​ tor. In this sce​ nario. the fault it​ self will con​ tain in​ for​ ma​ tion about pos​ si​ ble re​ me​ di​ at​ ion steps. Since it is still only a warn​ ing. this baselin​ ing method can be used as a ca​ pac​ ity plan​ ning tool.Health Scores 283 Health Score Use Cases Using Health Scores for Proactive Monitoring While ACME ad​ min​ is​ tra​ tors have tra​ di​ tion​ ally spent a lot of time re​ act​ ing to is​ sues on the net​ work. Most ob​ jects will have a Health tab which can be used to ex​ plore the re​ la​ tion​ ship be​ tween ob​ jects. Since you can ex​ port the scores and faults. The same sce​ nario can ob​ served with a load bal​ ancer or fire​ wall that is get​ ting over​ loaded. an op​ er​ at​ or can use the GUI to eas​ ily nav​ ig ​ate the re​ la​ tion​ ships and faults that are con​ tribut​ ing to that health score.

.

285 Monitoring .

.

Infrastructure Monitoring APICs CPU utilization and Memory Disk Utilization Physical and Bond Interface Statistics APIC Fan Status Temperature Status Power Supply Status ​Monitoring Leaf Switches Monitoring Switch CPU Utilization Monitoring Switch Memory Utilization Monitoring File System Health Monitoring CoPP (Control Plane Policing) Statistics Physical Interface Statistics and Link State Module Status Switch Fan Status .Tenant and Fabric Policies Stats Collection Policies Stats Export Policies Diagnostics Policies Call Home/SNMP/syslog Event Severity and Fault Severity Assignments Fault Lifecycle Policies TCAM Policy Usage Create TCAM Policy Monitor TCAM Prefix Usage Health Score Evaluation Policy Communication Policy • Proactive Monitoring .Monitoring 287 Section Content • Proactive Monitoring .

288 Monitoring Power Supply Status LLDP Neighbor Status GOLD Diagnostic Results • Proactive Monitoring Use Cases Monitoring Workload Bandwidth EPG Level Statistics • Reactive Monitoring Reactive Monitoring Tools Switch Port Analyzer (SPAN) Traceroute Atomic Counters Traffic Map Enhanced Troubleshooting Wizard IPing Audit Logs • Reactive Monitoring Use Cases ​Loss of Connectivity to Endpoint Users Report that an Application Running in the Fabric is Slow .

can be used to drill into any of the com​ po​ nents and pro​ vides the abil​ ity to click on a Stats tab to dis​ play on-de​ mand sta​ tis​ tics. There are four dif​ fer​ ent scopes for sta​ tis​ tics gath​ er​ ing: Com​ mon or Fab​ ric Wide. For ex​ am​ ple. When plan​ ning to move an ap​ pli​ ca​ tion from a legacy net​ work to the ACI in​ fra​ struc​ ture. How​ ever. The next step is to con​ fig​ ure a pol​ icy for trend analy​ sis. the human error and ef​ fort is min​ im ​al. Since sta​ tis​ tics are gath​ ered au​ to​ mat​ ic ​ally and poli​ cies are used and can be re-used in other places. this will save net​ work ad​ min​ is​ tra​ tors both time and frus​ tra​ tion. How​ ever. A Fab​ ric Wide pol​ icy would be cre​ ated as a de​ fault pol​ icy to be ap​ plied to all ten​ ants. SNMP and a third party tool may have been used to mon​ it​ or the CPU of switches or band​ width uti​ liza​ tion on ports. or often for​ got to add a new de​ vice to their Net​ work Mon​ it​ or​ ing Sys​ tem (NMS). Sta​ tis​ tics gath​ er​ ing has been a some​ what man​ ual and even re​ source in​ ten​ sive process for ACME in the past. or Ac​ cess. it has still been nec​ es​ sary to man​ ua ​lly spec​ ify which de​ vices are to be mon​ it​ ored and how they should be mon​ it​ ored.Monitoring 289 Proactive Monitoring Tenant and Fabric Policies Proac​ tive mon​ it​ or​ ing is a very im​ por​ tant piece of the net​ work ad​ min​ is​ tra​ tor's job. Ten​ ant. as well as to trou​ bleshoot or pre​ dict any is​ sues that may be aris​ ing. CLI. but is often ne​ glected be​ cause putting out fires in the net​ work usu​ ally takes pri​ or​ ity. Fab​ ric. whether ac​ cessed through the GUI. but more im​ por​ tantly it en​ ables the setup of poli​ cies to keep per​ sis​ tent data to an​ al​ yze trends in the en​ vi​ ron​ ment. since the APIC makes it in​ cred​ ib ​ly easy to gather sta​ tis​ tics and per​ form analy​ ses. it is sen​ si​ ble to start by test​ ing be​ fore going straight to pro​ duc​ tion. the ten​ - . Add test VMs to port groups on ei​ ther a DVS or AVS as​ so​ ci​ ated with the APIC. Even when they have used tools to gather data on layer one through seven de​ vices. to over​ ride that pol​ icy for a par​ tic​ ul​ ar ten​ ant. At this point the APIC is al​ ready gath​ er​ ing sta​ tis​ tics for the VMM do​ main and the phys​ ic ​al de​ vices. and pro​ vides the abil​ ity to proac​ tively mon​ it​ or your en​ tire en​ vi​ ron​ ment with​ out all of the has​ sle of main​ tain​ ing a third party mon​ it​ or​ ing tool. ACI pro​ vides an APIC which will do all of the sta​ tis​ tics gath​ er​ ing. but they strug​ gled with en​ ter​ ing cor​ rect SNMP in​ for​ ma​ tion on each de​ vice. This could also be in a test​ ing ten​ ant which is com​ pletely sep​ ar​ ate from the pro​ duc​ tion en​ vi​ ron​ ment. and add phys​ ic ​al test servers to VPCs on the leaf switches. or API. The APIC.

It is sim​ il​ arly pos​ si​ ble to change the poli​ cies for spe​ cific Mon​ it​ or​ ing Ob​ jects. but be re​ tained for 2 hours. b. and syslog • Event Severity Assignment Policies • Fault Lifecycle Policies Stats Collection Policies Click​ ing on Stats Col​ lec​ tion Poli​ cies will dis​ play the de​ fault re​ ten​ tion pe​ ri​ ods and admin states (En​ abled/Dis​ abled) for ALL Mon​ it​ ored Ob​ jects. to have it poll a com​ po​ nent every 5 min​ utes. In the fol​ low​ ing test​ ing ex​ am​ ple. 4 In the Work pane. Most likely the de​ faults will be kept. In the Name field enter a name for the Monitoring Policy. 6 In the Navigation pane. Create Tenant Monitor policy To cre​ ate a ten​ ant mon​ it​ or​ ing pol​ icy: 1 On the menu bar. Click Submit. For ex​ am​ ple. choose Actions > Create Monitoring Policies. 3 In the Navigation pane. A mon​ it​ or​ ing ob​ ject tells the APIC which com​ po​ nents to gather . SNMP. but a dou​ ble click on them will change the admin state or re​ ten​ tion pe​ ri​ ods. a Ten​ ant pol​ icy is cre​ ated to gather sta​ tis​ tics. cus​ tomers.290 Monitoring ant pol​ icy will over​ ride the Fab​ ric pol​ icy. choose the Tenant_Name. it will pro​ vide a real world ex​ am​ ple of how the ap​ pli​ ca​ tion will be​ have in a pro​ duc​ tion en​ vi​ ron​ ment. 5 In the Create Monitoring Policies dialog box. choose Tenants > ALL TENANTS. choose Tenant_Name > Monitoring Policies. Even if this ten​ ant is shared with other ap​ pli​ ca​ tions. just click on the pol​ icy that spec​ if​ ies a 5 minute gran​ ul​ ar​ ity and change the re​ ten​ tion pe​ riod to 2 hours. perform the following actions: a. test cases. 2 In the Work pane. choose Tenant_Name > Monitoring Policies > Policy_Name to display the following information: • Stats Collection Policies • Stats Export Policies • Diagnostics Policies • Callhome.

2 In the Work pane. Click on the pull down menu to se​ lect a mon​ it​ or​ ing ob​ ject and add a re​ ten​ tion pol​ icy to it. b. or explicitly select enabled or disabled. Stats Export Policies . Click Submit. For this ex​ am​ ple.Monitoring 291 sta​ tis​ tics about. choose the Tenant_Name. changes might be made to Mon​ it​ or​ ing Ob​ ject poli​ cies for Ten​ ant. c. e.​ RSOInfraBD) Mon​ it​ or​ ing Ob​ ject. Select the granularity with which it is to poll. 3 In the Navigation pane choose Tenant_Name > Monitoring Policies > Monitoring Policy_Name > Stats Collection Policies a. Click Update. To add mon​ it​ or​ ing ob​ jects: 1 On the menu bar. There are sev​ eral op​ tions and this will all de​ pend on what is im​ por​ tant to mon​ it​ or in the en​ vi​ ron​ ment. use the Bridge Do​ main (infra. To add a pol​ icy to a Mon​ it​ or​ ing Ob​ ject: 1 On the menu bar. choose the Tenant_Name. 3 In the Navigation pane choose Monitoring Policies > Monitoring Policy_Name > Stats Collection Policies. Click + to add the policy. perform the following actions: a. in the Stats Collection Policy dialog box. and/or Taboo Con​ tract. d. 2 In the Work pane. choose Tenants > ALL TENANTS. The retention policy may either be inherited or explicitly specified as enabled or disabled as well. Select the Monitoring Object. b. and remove any checkmarks next to Monitoring Objects to be left out. f. Leaf Port. 4 In the Work pane. Put a checkmark next to the Monitoring Objects to be included. Click on the Pencil icon to edit the Monitoring Objects. to change the in​ for​ ma​ tion gath​ ered for Bridge Do​ mains. choose Tenants > ALL TENANTS. Leave the state as inherited to stick with the defaults as set for ALL. For ex​ am​ ple. c. VXLAN Pool.

choose Tenants > ALL TENANTS. Choose to compress it using GZIP. Use the Stats Ex​ port Poli​ cies op​ tion in the left nav​ ig ​a​ tion pane. or leave it uncompressed. f. or se​ lect spe​ cific mon​ it​ or​ ing ob​ jects and spec​ ify where this in​ for​ ma​ tion will be saved. c. To cre​ ate a Stats Ex​ port Pol​ icy: 1 On the menu bar. Choose either JSON or XML as the format. Much like the Stats Col​ lec​ tion Poli​ cies. in the Stats Export Policy dialog box. or it may be dictated by the tool used to read it. 4 In the Work pane. b. perform the following actions: a. Click + under Export Destinations to specify a server where this information is to be collected.Boot-Up di​ ag​ nos​ tics or On​ go​ ing di​ ag​ nos​ tics. Next to the Mon​ it​ or​ ing Ob​ ject is the Pen​ cil but​ ton which en​ ables se​ lec​ tion of the mon​ it​ or​ ing ob​ jects to be con​ fig​ ured with di​ ag​ nos​ tics poli​ cies.292 Monitoring Stats Export Policies It is de​ sir​ able to col​ lect these on​ go​ ing sta​ tis​ tics as well as to see how this data be​ haves over time. There's really no difference other than personal preference. e. . Click + to add the policy. This is a re​ ally slick fea​ ture that al​ lows the setup of di​ ag​ nos​ tics test for the Mon​ it​ or​ ing Ob​ jects that were spec​ if​ ied in the Stats Col​ lec​ tion Poli​ cies. d. There are two dif​ fer​ ent kind of poli​ cies for con​ fig​ ur​ a​ tion . 5 Click Submit. g. 2 In the Work pane. Another wizard will pop up to enable specification of the protocols and credentials used to connect to this server. Now define the Stats Export Policy in the wizard. choose the Tenant_Name. it is pos​ si​ ble to cre​ ate a pol​ icy for ALL mon​ it​ or​ ing ob​ jects. lo​ cated under the mon​ it​ or​ ing pol​ icy. 3 In the Navigation pane choose Tenant_Name > Monitoring Policies > Monitoring Policy_Name > Stats Export Policies. Select ALL or a specific monitoring object from the drop-down list. Click Ok. Diagnostics Policies Next are the di​ ag​ nos​ tics poli​ cies in the nav​ ig ​a​ tion pane on the left.

Monitoring 293 To con​ fig​ ure di​ ag​ nos​ tic poli​ cies: 1 On the menu bar. CPU. SNMP or sys​ log poli​ cies can also be used with cur​ rent no​ ti​ fi​ ca​ tion sys​ tems. select either Boot-Up or Ongoing. i. . full tests. a. ii. Click Submit. The Call Home/SNMP/sys​ log pol​ icy will allow alert​ ing to be con​ fig​ ured in a flex​ ib ​le man​ ner. In the wizard give it a name and select the admin state. 3 In the Work pane. Internal Connectivity. 2 In the Navigation pane choose Tenant_Name > Monitoring Policies > default > Diagnostics Policies. Double-click on each to obtain the option of specifying no tests. Dif​ fer​ ent log​ ging lev​ els may be se​ lected for no​ ti​ fi​ ca​ tions and alert lev​ els spec​ if​ ied for Mon​ it​ or​ ing Ob​ jects from which alerts are to be re​ ceived. iii. b. Select one of the Monitoring Objects. and Ongoing will run the tests as often as specified within the wizard. Peripherals. and System Memory. perform the following actions: Note: Click on the Pencil Icon and put checks next to the Monitoring Objects which diagnostics tests are to be added to. or recommended tests. iv. Click + to add an Object. in the Diagnostic Policies dialog box. This al​ lows ad​ min​ is​ tra​ tors to re​ solve is​ sues be​ fore they turn into out​ ages. Boot-Up runs the tests while the devices are booting. Call Home/SNMP/syslog There are a few dif​ fer​ ent ways to setup no​ ti​ fi​ ca​ tion or alert poli​ cies. v. There are five different diagnostics tests available: ASIC. choose Fabric > Fabric Policies. Cisco Call Home is a fea​ ture in many Cisco prod​ ucts that will pro​ vide email or webbased no​ ti​ fi​ ca​ tion alerts in sev​ eral dif​ fer​ ent for​ mats for crit​ ic ​al events. The di​ ag​ nos​ tics found here can be use​ ful in find​ ing failed com​ po​ nents be​ fore they cause major is​ sues within your en​ vi​ ron​ ment.

which will dictate the fault codes for which you are changing the fault severity. perform the following actions: a. Note: Squelched gives it a weight of 0%. Warning. It re​ mains in the "clear​ ing time" state for the amount of time spec​ if​ ied in the "clear​ ing in​ ter​ val. 4 In the Work pane. d. 1 On the menu bar. Select the severity: Cleared. Critical." Lastly it moves on to the "re​ tain​ ing" state and does not get re​ moved until the end of the "re​ tain​ ing in​ ter​ val. the de​ fault sever​ ity as​ sign​ ments for Events and Faults will be kept. The Event Sever​ ity As​ sign​ ment Poli​ cies are con​ fig​ ured in the same way. Minor. Major. 5 Click Update. in the Fault Severity Assignment Policies dialog box. if only crit​ ic ​al faults are being no​ ti​ fied. After the fault clears it's in a state called "raised clear​ ing. 3 In the Navigation pane. Once a fault is de​ tected it is in the "soak​ ing" state. meaning it does not affect health scores. Click + to add an Object. Select the particular fault code for which severity is to be changed. choose Tenant_Name > Monitoring Policies > Monitoring_Policy > Fault Lifecycle Policies. b." It is only in this state briefly and moves on to the "clear​ ing time" state. After a cer​ tain amount of time. choose the Tenant_Name. Inherit. but there is a major fault you'd also like to be no​ ti​ fied about im​ me​ di​ ately. Fault Lifecycle Policies Fault Life​ cy​ cle is the term Cisco uses to de​ scribe the life of a fault. Select a Monitoring Object. Info. Most likely. 2 In the Work pane. Squelched. you can change the sever​ ity for that par​ tic​ ul​ ar fault code. For ex​ am​ ple. re​ ferred to as the "soak​ ing in​ ter​ val" it will move on to the "raised" state. c. but there are ex​ am​ ples where an ACI ad​ min​ is​ tra​ tor may de​ cide the event or fault is more or less se​ vere than the de​ fault value. "Raised" means the fault is still pre​ sent after the soak​ ing in​ ter​ val. choose Tenants > ALL TENANTS.294 Monitoring Event Severity and Fault Severity Assignments Event and fault sever​ it​ ies can be changed for events raised by Mon​ it​ or​ ing Ob​ jects." .

. choose Tenant_Name > Monitoring Policies > Monitoring_Policy > Fault Lifecycle Policies. where n is the num​ ber of sources. To see this click on Fab​ ric > In​ ven​ tory > Pod1 and then se​ lect the Op​ er​ a​ tional tab in the work pane. the Retention Interval is 3600 seconds. and Soaking Interval (all in seconds).Monitoring 295 To change Fault Life​ cy​ cle In​ ter​ vals: 1 On the menu bar. Click +. sources and des​ ti​ na​ tions be​ come one entry for a given EPG. Retention Interval. c. TCAM Policy Usage The phys​ ic ​al ternary con​ tent-ad​ dress​ able mem​ ory (TCAM) in which pol​ icy is stored for en​ force​ ment is an ex​ pen​ sive com​ po​ nent of switch hard​ ware and there​ fore tends to lower pol​ icy scale or raise hard​ ware costs. ACME will have other poli​ cies to con​ fig​ ure in the fab​ ric as out​ lined in the fol​ low​ ing sec​ tions. There is a a sys​ tem wide view of avail​ able TCAM re​ sources. perform the following actions: a. 4 In the Work pane. choose Tenants > ALL TENANTS. Note: The default for the Clearing Interval is 120 seconds. This pol​ icy size can be ex​ pressed as n*m*f. Within the Cisco ACI fab​ ric. At this point there will be a fully work​ ing ten​ ant mon​ it​ or​ ing pol​ icy. b. and you will see a table sum​ ma​ riz​ ing ca​ pac​ i-​ tiy for all nodes. which will dictate the fault codes for which you are changing the default intervals. 3 In the Navigation pane. Select a Monitoring Object. and the Soaking Interval is 120 seconds. TCAM is a fab​ ric re​ source that should be mon​ it​ ored. 2 In the Work pane. Specify times for the Clearing Interval. and f is the num​ ber of pol​ icy fil​ ters. pol​ icy is ap​ plied based on the EPG rather than the end​ point it​ self. choose the Tenant_Name. Within the Cisco ACI fab​ ric. in the Fault Lifecycle Policies dialog box. which re​ duces the num​ ber of total en​ tries re​ quired. m is the num​ ber of des​ ti​ na​ tions.

If your en​ vi​ ron​ ment has a high rate of change. The ar​ chi​ tec​ ture/de​ sign team should ar​ tic​ u​ late what the as​ sump​ tions were for TCAM uti​ liza​ tion. choose Monitor Policies > default > Stats Collection Policies. the de​ fault mon​ i​ tor​ ing poli​ cies will alert you to a re​ source short​ age and lower over​ all fab​ ric health score.​ com/​ datacenter/​ FabricResourceCalculation As a gen​ eral rule. or you an​ tic​ i​ pate the pos​ si​ bil​ ity of con​ sis​ tently being over​ sub​ scribed. . you may want to set dif​ fer​ ent thresh​ olds. choose Fabric > Fabric Policies. 2 In the Navigation pane.296 Monitoring Switch TCAM capacity dashboard TCAM is a crit​ i​ cal sys​ tem re​ source in an ACI fab​ ric and should be mon​ i​ tored for uti​ liza​ tion. There is a Fab​ ric Re​ source Cal​ cu​ la​ tion tool on Github that will help with es​ ti​ ma​ tion of nor​ mal op​ er​ at​ ing pa​ ra​ me​ ters: https://​ github. Create TCAM Policy Monitor 1 On the menu bar.

In the Penalty of fault severity warning dropdown menu. 2 In the Navigation pane. 2 In the Navigation pane. d. In the Penalty of fault severity minor dropdown menu. Select the Stats Type Policy Entry. In the Thresholds For Collection 5 Minute window. perform the following actions: a. perform the following actions: a. 1 On the menu bar. select the blue pencil icon next to policy CAM entries usage current value. Health Score Evaluation Policy 1 On the menu bar.Monitoring 297 3 In the Work pane. b. select the desired %. . TCAM Prefix Usage This pro​ ce​ dure man​ ages a TCAM Pre​ fix Usage. in the Properties dialog box. perform the following actions: a. c. in the Stats Collection Policies dialog box. select the desired %. select the desired %. b. c. 3 In the Work pane. Click + under Config Thresholds.Entity). 3 In the Work pane. In the Penalty of fault severity critical dropdown menu. choose Monitor Policies > default > Stats Collection Policies. c. choose Fabric > Fabric Policies. In the Penalty of fault severity major dropdown menu. choose Monitor Policies > Common Policies > Health Score Evaluation Policy > Health Score Evaluation Policy.Entity). In the Thresholds For Collection 5 Minute window. Select the Monitoring Object Equipment Capacity Entity (eqptcapacity. Select the Monitoring Object Equipment Capacity Entity (eqptcapacity. choose Fabric > Fabric Policies. d. select the desired %. Select the Stats TypeLayer3 Entry. Click + under Config Thresholds. select the blue pencil icon next to policy CAM entries usage current value. d. b. in the Stats Collection Policies dialog box.

298 Monitoring 4 Click Submit. choose Actions > Create Communication Policy. perform the following actions: a. d. From the HTTP Admin State dropdown menu select the desired state. From the Telnet Port dropdown menu select the desired port. . choose Fabric > Fabric Policies. expand Pod Policies > Policies > Communication. Select the desired HTTPS redirect state. i. Select the desired HTTP redirect state. From the HTTPS Admin State dropdown menu select the desired state. g. From the HTTP Port dropdown menu select the desired port. From the HTTPS Port dropdown menu select the desired port. e. Enter Communication Policy Name. 2 In the Navigation pane. From the SSH Admin State dropdown menu select the desired state. 5 Click Submit. h. f. c. Communication Policy 1 On the menu bar. b. 4 In the Create Communication Policy dialog box. 3 In the Work pane. j. From the Telnet Admin State dropdown menu select the desired state.

Infrastructure While health scores pro​ vide a com​ pre​ hen​ sive view of the health sta​ tus of var​ io ​us ob​ jects. This sec​ tion of the book at​ tempts to cover some key per​ for​ mance in​ di​ ca​ tors that should be mon​ it​ ored. pro​ ce​ dures for mon​ it​ or​ ing them. and ref​ er​ ences are pro​ vided to ways of ob​ tain​ ing the data when pos​ si​ ble.​ html It is im​ por​ tant to note that this list changes as newer soft​ ware ver​ sions are made avail​ able. . A full list of MIBs sup​ ported on the 1. ACME will also need to set up poli​ cies to mon​ it​ or spe​ cific re​ sources. GUI. SNMP.​ cisco. The screen​ shot shows where this in​ for​ ma​ tion is vis​ ib ​le. In this ex​ am​ ple. sug​ gest rec​ om​ mended thresh​ olds for trig​ ger​ ing alerts or alarms from Net​ work Mon​ it​ or​ ing Sys​ tems (NMS). the health of the APICs and the health of the clus​ ter it​ self are dis​ played right at the dash​ board. When log​ ging into the sys​ tem dash​ board. two out of the three APICs are in a sub-op​ ti​ mal state and APIC 1 is also ex​ pe​ ri​ enc​ ing is​ sues. and when pos​ si​ ble. Op​ er​ at​ ors can use a wide array of tools in​ clud​ ing Sys​ log. and fab​ ric and ten​ ant poli​ cies show us a nar​ rower view of fab​ ric and ten​ ant health.Monitoring 299 Proactive Monitoring . Monitoring APICs CPU utilization and Memory GUI The eas​ ie ​st way to quickly ver​ ify the health of the con​ trollers is the APIC. REST API calls and CLI.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.x is avail​ able at: http://​ www. There are sev​ eral meth​ ods to ob​ tain this data. and more vari​ ables may be ex​ posed via SNMP MIBs in the fu​ ture.

pro​ cEn​ tity is a con​ tainer of processes in the sys​ tem.xml? . A more de​ tailed drill​ down is avail​ able by click​ ing on Sys​ tem > Con​ trollers.CPU uti​ liza​ tion maxMemAl​ loc .The max​ i​ mum amount of avail​ able mem​ ory for the sys​ tem Sam​ ple Usage: This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: http[s]://apic_ip/api/node/class/procEntity.The max​ i​ mum mem​ ory al​ lo​ cated for the sys​ tem mem​ Free . The pro​ cEn​ tity ob​ jects con​ tain the fol​ low​ ing use​ ful prop​ er​ ties: cpuPct . REST API Con​ trollers pro​ vide in​ for​ ma​ tion re​ gard​ ing the cur​ rent sta​ tus of CPU and mem​ ory uti​ liza​ tion by cre​ at​ ing in​ stances of the pro​ cEn​ tity class. This ob​ ject holds de​ tailed in​ for​ ma​ tion about var​ i​ ous processes run​ ning on the APIC.300 Monitoring System health dashboard The nor​ mal state for these is to have them all green in a "fully fit" state im​ ply​ ing the APICs are syn​ chro​ nized with each other.

74 svc_ifc_scripth 20 0 834m 419m 94m S 1.3 20:35.7 0. Cpu(s): Mem: 0.2 17:41.2 16:28. 4.2 17:08.29 1 running.2 17:13.2 38:11.37 svc_ifc_observe 32128 ifc 20 0 639m 315m 69m S 1. 0.27.6 20:03.3 0.0%st 1952656k cached PID USER PR NI 32102 root 20 0 556m 205m 85m S 3.19.0%wa. 0k used.0 0.0%ni. 0 stopped.92 svc_ifc_topomgr 32105 root 20 0 659m 258m 85m S 1. 409540k buffers TIME+ COMMAND Disk Utilization GUI There are sev​ eral disks and file sys​ tems pre​ sent on the APICs.bin 1291 root VIRT 0k free. 0.0 0.0%si.3%us. 32132 ifc 20 0 657m 252m 71m S 1.11:41:51 up 16:50.3%id. Swap: 4 users. 7473180k used.bi 32121 ifc 20 0 631m 286m 86m S 2. Tasks: 354 total. 131954932k total. 4.24 nginx.3 0.04 svc_ifc_applian 32120 ifc 20 0 660m 343m 86m S 2.7 0.35 svc_ifc_bootmgr 32113 ifc 20 0 1083m 721m 69m S 1. 99. 0k total. .34 svc_ifc_reader. The disk uti​ liza​ tion can be viewed by click​ ing on Sys​ tem > Con​ trollers > Apic-X > Stor​ age The work pane dis​ plays the uti​ liza​ tion of all par​ ti​ tions in the sys​ tem.7 0.4%sy. 353 sleeping. 0.0%hi.Monitoring 301 CLI The Linux Top util​ ity also comes built into the APIC con​ trollers and can be used for trou​ bleshoot​ ing and/or ver​ if​ i​ ca​ tion.7 0.73 svc_ifc_dbgr. 0 zombie 0. load average: 4. 124481752k free. The GUI pro​ vides ready ac​ cess to disk space uti​ liza​ tion of all par​ ti​ tions on the sys​ tem and can be used for mon​ i​ tor​ ing this in​ for​ ma​ tion. 0. RES SHR S %CPU %MEM 0. user@apic1:~> top top .3 27:58.

xml? Physical and Bond Interface Statistics APICs use a bonded in​ ter​ face that is typ​ ic ​ally dual-homed to two leaves for con​ nec​ tiv​ ity to the ACI fab​ ric and have the abil​ ity to use a bonded in​ ter​ face that can be dual homed to the out-of-band man​ age​ ment net​ work. REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: http[s]://apic-ip/api/node/class/eqptStorage. . Bond1 is the bond in​ ter​ face used to con​ nect to the out-of-band seg​ ment (to con​ nect to an OOB seg​ ment that al​ lows setup of the APIC it​ self). Bond0 is the bond in​ ter​ face used to con​ nect to the fab​ ric it​ self (to con​ nect to leaves that con​ nect into the fab​ ric).302 Monitoring CLI user@apic1:~> df Filesystem 1K-blocks /dev/dm-1 41282880 10518960 Used Available Use% Mounted on 28666872 tmpfs 4194304 56456 4137848 tmpfs 65977464 964 65976500 1% /tmp 10518960 28666872 27% /data 13860672 25327104 36% /firmware 27% / 2% /dev/shm /dev/mapper/vg_ifc0-data 41282880 /dev/mapper/vg_ifc0-firmware 41284928 /dev/mapper/vg_ifc0-data2 583149656 1281104 552246280 1% /data2 * Note that not all file sys​ tems are vis​ ib ​le from the CLI as some re​ quire root ac​ cess to reach the mount points. The GUI should be used as a sin​ gle source of truth for file sys​ tem uti​ liza​ tion.

json?querytarget=subtree&target-subtree-class=l3EncRtdIf APIC Fan Status The fol​ low​ ing sec​ tion de​ scribes method​ olo​ gies to re​ trieve the sta​ tus of the fan trays on the APICs. REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-1/sys. nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > In​ ter​ faces CLI Both "if​ con​ fig" and the "ip link" CLI com​ mands can be used to ver​ ify link state. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. The CLI also pro​ vides in​ for​ ma​ tion on de​ tailed in​ ter​ face sta​ tis​ tics such as RX and TX coun​ ters.Monitoring 303 The bond in​ ter​ faces rely on un​ der​ ly​ ing phys​ ic ​al in​ ter​ faces and it is im​ por​ tant to note that the GUI pro​ vides link in​ for​ ma​ tion for both the phys​ ic ​al and log​ ic ​al bond in​ ter​ faces.json?querytarget=subtree&target-subtree-class=eqptFan CLI . nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > Equip​ ment-Fans REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-1.

The CIMC port is the in​ te​ grated lights-out man​ age​ ment port that can be used to re​ cover an APIC in the event of a cat​ as​ trophic fail​ ure. Min. admin@172.--------.304 Monitoring CLI The Fan sta​ tus for the APICs can be mon​ it​ ored using the CLI on the CIMC port of the APIC. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs.16.-------. Max.179's password: C220-FCH1807V02V# scope sensor C220-FCH1807V02V /sensor # show fan Name Sensor Reading Units Status ----------.179' (RSA) to the list of known hosts. user@apic1:~> ssh -l admin 172.------. To ob​ tain this. Warning Warning Failure Failure ----.16.176.--------- FAN1_TACH1 Normal 7490 RPM 1712 N/A 1284 N/A FAN1_TACH2 Normal 7490 RPM 1712 N/A 1284 N/A FAN2_TACH1 Normal 7490 RPM 1712 N/A 1284 N/A FAN2_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN3_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A FAN3_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN4_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A FAN4_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN5_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A Temperature Status To mon​ it​ or the tem​ per​ at​ ure state of the var​ io ​us sen​ sors avail​ able on the APICs use the fol​ low​ ing steps.-------- Min. Max.16.179 Warning: Permanently added '172. login to the CIMC using the cre​ den​ tials used for set​ ting up the CIMC (may not be the same as the cre​ den​ tials used for APIC).176. nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > Equip​ ment-Sen​ sors REST API .--------. If this has not been setup pre​ vi​ ously. the de​ fault user​ name is admin and the de​ fault pass​ word is pass​ word.176.

--------.0 C N/A 65.0 C N/A 65. Warning Warning Failure Failure ------.0 N/A 70.0 N/A 70.0 C N/A 60.0 FP_TEMP_SENSOR Normal 37.0 N/A 85.0 Power Supply Status To mon​ it​ or the tem​ per​ at​ ure state of the var​ io ​us sen​ sors avail​ able on the APICs use the fol​ low​ ing steps.0 C N/A 65.0 P2_TEMP_SENS Normal 50.0 C N/A 65.-------.0 N/A 85.0 C N/A 60.0 C N/A 60.0 C N/A 80.0 C N/A 85.0 C N/A 60.0 DDR3_P2_H1_TEMP Normal 41.0 PSU1_TEMP Normal 37.0 PCH_TEMP_SENS Normal 51.0 N/A 86. Max.0 RISER1_OUTLETTMP Normal 50.0 N/A 70.--------- P1_TEMP_SENS Normal 49.0 DDR3_P2_G1_TEMP Normal 42.json?querytarget=subtree&target-subtree-class=eqptSensor CLI C220-FCH1807V02V /sensor # show temperature C220-FCH1807V02V /sensor # show temperature Name Sensor Reading Units Status -----------------.0 C N/A 65.0 N/A 65.0 N/A 85.0 DDR3_P1_A1_TEMP Normal 42.0 DDR3_P1_B1_TEMP Normal 43.5 C N/A 81. Min.5 C N/A 81.0 N/A 70.0 C N/A 60.0 N/A 86.0 RISER2_OUTLETTMP Normal 41.0 C N/A 65.0 N/A 90.0 VICP81E_0_TMP3 Normal 56. .0 DDR3_P1_D1_TEMP Normal 44.0 N/A 85.-------- Min.0 N/A 85.0 N/A 70.Monitoring 305 REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}//api/node/mo/topology/pod-1/node-1.0 N/A 85.0 RISER1_INLET_TMP Normal 45.0 RISER2_INLET_TMP Normal 41.0 DDR3_P2_F1_TEMP Normal 43.0 C N/A 65.0 DDR3_P2_E1_TEMP Normal 43.------.-------.0 DDR3_P1_C1_TEMP Normal 44.0 N/A 85.0 C N/A 60.0 N/A 85.0 N/A 85.0 C N/A 65. Max.

The leaf switches pro​ vide first hop con​ nec​ tiv​ ity to any​ thing that at​ taches to the fab​ ric. Min.-------.306 Monitoring CLI C220-FCH1807V02V /sensor # show psu Name Sensor Reading Units Status -----------------. Max. Warning Warning Failure Failure ------.0 N/A 86.0 POWER_USAGE Normal 160 Watts N/A N/A N/A 800 PSU1_POUT Normal 136 Watts N/A 624 N/A 648 PSU1_PIN Normal 160 Watts N/A 720 N/A 744 PSU1_STATUS Normal present PSU2_STATUS Normal absent PSU1_PWRGD Normal good PSU1_AC_OK Normal good Monitoring Leaf Switches Leaf switches in the ACI fab​ ric typ​ ic ​ally equate to the Nexus 9300 fam​ ily of switches (with the ex​ cep​ tion of the Nexus 9336 switch). all ex​ ter​ nal con​ nec​ tiv​ ity is pro​ vided through a set of leaf switches that pro​ vide high den​ sity 10gig or 40gig con​ nec​ tiv​ ity to el​ e​ ments out​ side the fab​ ric. Max. in the ACI fab​ ric. Note that un​ like tra​ di​ tional two or three tier de​ signs where the "WAN" layer at​ taches to ei​ ther the dis​ tri​ bu ​t​ ion/ag​ gre​ ga​ tion layer.--------.5 C N/A 81.-------- -------- Min.-------. To ac​ cess the dash​ board to mon​ it​ or switches nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-* .--------- P1_TEMP_SENS Normal 49.

Monitoring Switch CPU Utilization There are sev​ eral meth​ ods to poll CPU uti​ liza​ tion and trend it over dif​ fer​ ent pe​ ri​ ods of time. and the sub tabs on the right hand side (topol​ ogy/gen​ eral/stats/health/faults/trou​ bleshoot​ ing/his​ tory) can be used to quickly drill down into the var​ i​ ous prop​ er​ ties of the switch to un​ der​ stand how the switch is de​ ployed from a hard​ ware con​ fig​ u​ ra​ tion stand​ point. The fol​ low​ ing sec​ tions de​ scribe a few of the meth​ ods avail​ able.Monitoring 307 Leaf switch monitoring dashboard No​ tice that the dash​ board for this switch de​ faults to pre​ sent​ ing the health score of the switch at a node level on the dash​ board. This class up​ dates every 10 sec​ onds. re​ me​ di​ ate faults on the switch. REST API Spine and Leaf switches CPU uti​ liza​ tion can be mon​ i​ tored using the fol​ low​ ing classes. . based on the de​ sired timescale and gran​ u​ lar​ ity. proc:SysCPU5min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 5 minute sam​ pling in​ ter​ val. and trou​ bleshoot the switch from a hard​ ware per​ spec​ tive.

proc:SysCPU1w A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 week sam​ pling in​ ter​ val.3% svc_ifc_observe .xml? CLI Leaf-1# show proc cpu sort PID Runtime(ms) Invoked uSecs 1Sec Process ----- ----------- -------- ----- ------ ----------- 4012 69510 493837 140 1.308 Monitoring proc:SysCPU15min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 15 minute sam​ pling in​ ter​ val. This class up​ dates every hour.3% python 4292 3841 134758 28 0. proc:SysCPU1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 hour sam​ pling in​ ter​ val. http[s]://apic_ip//api/node/class/procSysCPU1d. proc:SysCPU1mo A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 month sam​ pling in​ ter​ val. proc:SysCPU1qtr A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 quar​ ter sam​ pling in​ ter​ val. This class up​ dates every day. This class up​ dates every 15 min​ utes.4% nginx 4067 1911 206 9278 0. proc:SysCPU1year A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 year sam​ pling in​ ter​ val.3% t2usd_tor 4065 7239 27609 262 1. This class up​ dates every day.4% svc_ifc_policye 4302 1904 1862 1022 0. This class up​ dates every day.8% svc_ifc_opflexe 4391 2355 4423 532 0. ACME would like to see the av​ er​ age CPU uti​ liza​ tion of all of the fab​ ric switches over the last day. This class up​ dates every day. This class up​ dates every 5 min​ utes. re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 proc:SysCPU1d A class that rep​ day sam​ pling in​ ter​ val.

.3% svc_ifc_confele 4123 1407 251 5606 0.1.1% ospf 13606 435 211 2065 0.0% policy_mgr 4301 1860 19152 97 0..0% dc3_sensor 4299 8559 8225 1040 0.0% bgp 4296 6263 7413 844 0.....3% svc_ifc_dbgrele 4846 119693 36527 3276 0....4. 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) # = average CPU% 5 0 5 ...1% pfmclnt 4864 2361 2812 839 0.3..5..0% plog 4866 2792 3269 854 0.2...4.0% mcecm In order to ob​ tain a his​ tor​ ic ​al view of CPU uti​ liza​ tion from the CLI it may be nec​ es​ sary to jump into an al​ ter​ na​ tive shell from the switch bash prompt. This shell is called vsh (or v-shell)........2% stats_manager 3923 15406 2645 5824 0..Monitoring 309 4311 1811 1018 1779 0.. Leaf-1# show processes cpu history 1 1 33 1 746554661885833055376572545534667663554785033943645665335644 100 90 80 70 60 50 40 # 30 ## 20 ## 10 # ### ####### ######## # ## ##### ## #### # # #### ## 0..0% snmpd 4297 6667 4542 1467 0.5.3% svc_ifc_eventmg 4310 1802 689 2616 0..0% isis 5025 1611 1743 924 0....5.2.3.....1% ospfv3 4865 2402 2717 884 0....1.

.5......3....5..3...5..3..7.... 0 5 0 5 0 5 0 5 CPU% per hour (last 72 hours) * = maximum CPU% # = average CPU% 0 5 0 5 0 ..6......1.6.....5.1. 0 5 0 5 0 5 0 5 0 5 CPU% per minute (last 60 minutes) * = maximum CPU% # = average CPU% 1 440 030 100 * 90 * 80 * 70 * 60 * 50 * 40 *** 30 *** 20 *** 10 ### 0........4...4..2..2...5....2.5........2...1.......3.....310 Monitoring 32 13311134111111111311111111131 11111113 1111 11231 1111111 749513800432206328353370732175609342000769025791144192680117 100 90 80 70 60 50 40 * * 30 * ** 20 ** **** * ** ** * * * ** * * * * * *** ** * ** ** ** * 10 ############################################################ 0......1.4............4.

re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ proc:Sys​ Mem1mo A class that rep​ ory in a 1 month sam​ pling in​ ter​ val. ACME would like to mon​ it​ or mem​ ory over the last day. proc:Sys​ Mem15min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 15 minute sam​ pling in​ ter​ val. proc:Sys​ Mem5min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 5 minute sam​ pling in​ ter​ val. REST API Spine and Leaf switches mem​ ory uti​ liza​ tion can be mon​ it​ ored using the fol​ low​ ing classes. This class up​ dates every 15 min​ utes. proc:Sys​ Mem1qtr A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 quar​ ter sam​ pling in​ ter​ val. This class up​ dates every hour.Monitoring 311 Monitoring Switch Memory Utilization There are sev​ eral meth​ ods to poll mem​ ory uti​ liza​ tion and trend it over dif​ fer​ ent pe​ ri​ ods of time. This class up​ dates every day. proc:Sys​ Mem1d A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 day sam​ pling in​ ter​ val. proc:Sys​ Mem1year A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 year sam​ pling in​ ter​ val. This class up​ dates every 10 sec​ onds. This class up​ dates every day. This class up​ dates every day. This class up​ dates every day.xml? . proc:Sys​ Mem1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 hour sam​ pling in​ ter​ val. based on the de​ sired timescale and gran​ ul​ ar​ ity. The fol​ low​ ing sec​ tions de​ scribe a few of the meth​ ods avail​ able. and would use the fol​ low​ ing REST call: http[s]://apic_ip/api/node/class/procSysMem1d. proc:Sys​ Mem1w A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 week sam​ pling in​ ter​ val. This class up​ dates every 5 min​ utes.

Memory usage: 5 minutes: 1. the CLI is only way to mon​ it​ or the uti​ liza​ tion of the file sys​ tem on the leaves. 7347052K free Current memory status: OK SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.​ html Monitoring File System Health CLI Cur​ rently. The crit​ ic ​al vol​ umes to keep track of in terms of uti​ liza​ tion are /volatile.4% idle 9054020K used. 15 minutes: 0.14 2.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mibsupport.​ cisco. See http://​ www. 16401072K total.21 Processes : 513 total.312 Monitoring CLI Leaf-1# show system resources Load average: 1 minute: 1.x.80 93. 2 running CPU states : 4. boot​ flash and logflash Leaf-1# df df: `/nxos/tmp': No such file or directory df: `/var/home': No such file or directory df: `/var/tmp': No such file or directory df: `/nginx': No such file or directory df: `/debugfs': No such file or directory df: `/recovery': No such file or directory df: `/cfg0': No such file or directory df: `/cfg1': No such file or directory df: `/logflash/INXOS_SYSMGR/startup-cfg': No such file or directory df: `/mnt/plog': No such file or directory Filesystem 1K-blocks Used Available Use% Mounted on .1% user.5% kernel. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive. It is nor​ mal to see a higher % uti​ liza​ tion on some of the mount points in the file sys​ tem hi​ er​ ar​ chy.

old.Monitoring 313 rootfs 512000 1064 510936 1% / rootfs 512000 1064 510936 1% / none 512000 1064 510936 1% /isan none 512000 1064 510936 1% /var none 51200 2288 48912 5% /etc none 51200 108 51092 1% /var/log none 3145728 336664 2809064 11% /dev/shm 512000 0 512000 0% /volatile 7782036 1080636 6306088 15% /bootflash none /dev/sda4 /dev/sda5 60485 5356 52006 10% /mnt/cfg/0 /dev/sda6 60485 5356 52006 10% /mnt/cfg/1 /dev/sda3 120979 13349 101384 /dev/sda7 512000 1064 510936 /dev/sda9 15748508 591216 14357292 4% /mnt/ifc/cfg /dev/sda8 15748508 991204 13957304 7% /mnt/ifc/log /dev/sda8 15748508 991204 13957304 7% /var/log/dme/oldlog /dev/sda9 15748508 591216 14357292 716800 665728 51072 93% /mnt/ifc/cfg/mgmt/opt/controller/sbin 93% /controller/sbin rootfs rootfs /dev/sda8 rootfs /dev/sda4 rootfs /dev/sda9 rootfs rootfs 716800 665728 51072 15748508 991204 13957304 716800 665728 51072 7782036 1080636 6306088 12% /mnt/pss 1% /logflash 4% /controller 7% /data/techsupport 93% /bin 15% /bootflash 716800 665728 51072 15748508 591216 14357292 93% /data/challenge.plugin 716800 665728 51072 93% /controller/sbin 93% /dev 4% /controller 716800 665728 51072 none 3145728 336664 2809064 none 51200 2288 48912 none 2097152 682360 1414792 33% /isan/plugin/0/isan/utils none 2097152 682360 1414792 33% /isan/plugin/0/lc/isan/utils none 2097152 682360 1414792 33% /isan/plugin/0/lc/isan/lib none 2097152 682360 1414792 33% /isan/plugin/0/isan/lib none 2097152 682360 1414792 33% /isan/lib none 2097152 682360 1414792 33% /isan/plugin/0/lib none 2097152 682360 1414792 33% /isan/utils rootfs 716800 665728 51072 93% /lc/isan/utils rootfs 716800 665728 51072 93% /lib rootfs 716800 665728 51072 93% /mnt/cfg 60485 5356 52006 10% /mnt/cfg/0 /dev/sda5 11% /dev/shm 5% /etc .

314 Monitoring /dev/sda6 60485 5356 52006 10% /mnt/cfg/1 716800 665728 51072 93% /mnt/ifc 15748508 591216 14357292 716800 665728 51072 /dev/sda8 15748508 991204 13957304 /dev/sda3 120979 13349 101384 rootfs /dev/sda9 rootfs rootfs /dev/sda8 none rootfs none 4% /mnt/ifc/cfg 93% /mnt/ifc/cfg/mgmt/opt/controller/sbin 7% /mnt/ifc/log 12% /mnt/pss 716800 665728 51072 15748508 991204 13957304 93% /sbin 1572864 39444 1533420 3% /tmp 716800 665728 51072 93% /usr 7% /data/techsupport 51200 108 51092 15748508 991204 13957304 51200 108 51092 rootfs 716800 665728 51072 93% /var/log/dme rootfs 716800 665728 51072 93% /var/log/dme/nginx rootfs 716800 665728 51072 93% /usr/share/vim /dev/sda7 11811760 375608 10836140 4% /var/log/dme/log /dev/sda8 15748508 991204 13957304 7% /var/log/dme/oldlog none 512000 1064 510936 1% /var/run/mgmt/log none 512000 1064 510936 1% /var/run/utmp 11811760 375608 10836140 40960 8 40952 11811760 375608 10836140 rootfs 716800 665728 51072 none 512000 0 512000 /dev/sda8 none /dev/sda7 none /dev/sda7 1% /var/log 7% /var/log/dme/oldlog 1% /var/log/messages 4% /var/sysmgr 1% /var/sysmgr/startup-cfg 4% /logflash/core 93% /usb 0% /volatile Monitoring CoPP (Control Plane Policing) Statistics CLI CoPP is en​ abled by de​ fault and the pa​ ra​ me​ ters can​ not be changed at this time. To show the CoPP pol​ icy that is pro​ gramed by the sys​ tem use the fol​ low​ ing CLI com​ mand: Leaf-1# show copp policy COPP Class COPP proto COPP Rate COPP Burst . CoPP sta​ tis​ tics are avail​ able through the CLI.

Monitoring 315 mcp mcp 500 500 ifc ifc 5000 5000 igmp igmp 1500 1500 nd nd 1000 1000 cdp cdp 1000 1000 pim pim 500 500 dhcp dhcp 1360 340 lacp lacp 1000 1000 ospf ospf 2000 2000 arp arp 1360 340 lldp lldp 1000 1000 acllog acllog 500 500 stp stp 1000 1000 eigrp eigrp 2000 2000 coop coop 5000 5000 traceroute traceroute 500 500 isis isis 1500 5000 icmp icmp 500 500 bgp bgp 5000 5000 To show drops against spe​ cific CoPP classes. use the fol​ low​ ing CLI com​ mand: Leaf-1# show copp policy stats COPP Class COPP proto COPP Rate COPP Burst AllowPkts AllowBytes DropPkts DropBytes mcp mcp 500 500 0 0 0 0 ifc ifc 5000 5000 195072 161613961 0 0 igmp igmp 1500 1500 3 192 0 0 nd nd 1000 1000 6 564 0 0 cdp cdp 1000 1000 494 140543 0 0 pim pim 500 500 0 0 0 0 dhcp dhcp 1360 340 4 1400 0 0 lacp lacp 1000 1000 0 0 0 0 ospf ospf 2000 2000 0 0 0 0 arp arp 1360 340 1284 90068 0 0 lldp lldp 1000 1000 5029 1717208 0 0 acllog acllog 500 500 0 0 0 0 stp stp 1000 1000 0 0 0 0 eigrp eigrp 2000 2000 0 0 0 0 coop coop 5000 5000 4722 470546 0 0 .

316 Monitoring traceroute traceroute 500 500 0 0 0 0 isis isis 1500 5000 17141 2167565 0 0 icmp icmp 500 500 0 0 0 0 bgp bgp 5000 5000 864 73410 0 0 Physical Interface Statistics and Link State GUI To ac​ cess in​ ter​ face link state in​ for​ ma​ tion. routed in​ ter​ faces. the oper state col​ umn dis​ plays the op​ er​ a​ tional state of the link. nav​ i​ gate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > In​ ter​ faces > Phys​ i​ cal In​ ter​ faces. and then click on the Stats tab in the work pane on the right-hand side. nav​ i​ gate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > In​ ter​ faces > Phys​ i​ cal in​ ter​ faces > Eth X/Y. To ac​ cess in​ ter​ face sta​ tis​ tics. vir​ tual port-chan​ nels. in the APIC GUI. loop​ backs. etc. In the work pane. Physical interface throughput statistics Note that click​ ing on the check icon en​ ables you to se​ lect ad​ di​ tional sta​ tis​ tics that can be graphed sim​ i​ lar to the fol​ low​ ing screen​ shot. Note that there are other tabs avail​ able in the work pane that ref​ er​ ence other types of in​ ter​ faces like port-chan​ nels. . in the APIC GUI.

5 minute rate. input/out​ put / du​ plex. etc.).g. in​ for​ ma​ tion pro​ vided in pre​ ced​ ing sec​ tions and the tools de​ scribed therein An ex​ am​ ple of the base API call for phys​ i​ cal in​ ter​ face sta​ tis​ tics is: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/phys-[eth1/1]. As a pointer.Monitoring 317 Granular physical interface statistics REST API For cus​ tomers that pre​ fer the REST API in​ ter​ face to poll for in​ ter​ face sta​ tis​ tics. It is ex​ pected that the reader has a good un​ der​ stand​ ing of the ob​ ject model and is able to nav​ i​ gate through the model to ob​ tain the in​ for​ ma​ tion de​ siered using the ex​ am​ ple below.json . 30 sec​ ond rates. mul​ ti​ cast pack​ ets. uni​ cast pack​ ets. There are sev​ eral such coun​ ters that are avail​ able (e. sev​ eral ob​ jects are avail​ able. RX/TX. as the chil​ dren can be de​ rived from it. the par​ ent man​ aged ob​ ject is pro​ vided below.

such as ingress and egress bytes. the op​ er​ at​ or could see chil​ dren of the in​ ter​ face ob​ ject. DLY 1 usec reliability 255/255. BW 1000000 Kbit. Other sup​ ported com​ mands in​ clude "show in​ ter​ face port-chan​ nel x/y" Leaf-1# show int e1/1 Ethernet1/1 is up admin state is up. media type is 1G Beacon is turned off Auto-Negotiation is turned on Input flow-control is off. One of the child ob​ jects in​ cludes the fol​ low​ ing: topology/pod-1/node-101/sys/phys-[eth1/1]/dbgEtherStats&#10 CLI The show in​ ter​ face eth x/y com​ mand can be used to mon​ it​ or in​ ter​ faces from the CLI.8771 (bia 7c69.f60f. 1000 Mb/s. output flow-control is off Auto-mdix is turned off Rate mode is dedicated Switchport monitor is off EtherType is 0x8100 EEE (efficient-ethernet) : n/a Last link flapped 04:19:13 Last clearing of "show interface" counters never 1 interface resets 30 seconds input rate 169328 bits/sec.json Vi​ sore al​ lows the op​ er​ at​ or to dig deeper into the hi​ er​ ar​ chi​ cal tree.318 Monitoring For ex​ am​ ple.8771) MTU 9000 bytes. address: 7c69. 97 packets/sec 30 seconds output rate 424528 bits/sec. the ACME op​ er​ at​ or could issue the fol​ low​ ing API call: /topology/pod-1/node-101/sys/phys-[eth1/1].f60f. Dedicated Interface Hardware: 1000/10000/auto Ethernet. 115 packets/sec . rxload 1/255 Encapsulation ARPA. to de​ ter​ mine the total ingress bytes on Leaf 101 port Eth1/1. txload 1/255. From the prior com​ mand. medium is broadcast Port mode is trunk full-duplex.

the NFE (Net​ work For​ ward​ ing En​ gine ASIC) which pro​ vide the front panel ports.​ cisco. there are two data plane com​ po​ nents.x. they have a su​ per​ vi​ sor com​ po​ nent which refers to the CPU com​ plex. 134 pps. which pro​ vides up​ link con​ nec​ tiv​ ity to the spines. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for in​ ter​ faces See: http://​ www.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. 114 pps RX 2474537 unicast packets 8434 multicast packets 2482973 input packets 0 jumbo packets 0 runts 0 storm suppression bytes 0 giants 0 input error 0 watchdog 2 broadcast packets 1686129815 bytes 0 CRC 0 no buffer 0 short frame 0 bad etype drop 0 input with dribble 0 overrun 0 underrun 0 bad proto drop 0 ignored 0 if down drop 712 input discard 0 Rx pause TX 1673907 unicast packets 1674489 output packets 575 multicast packets 7 broadcast packets 455539518 bytes 0 jumbo packets 0 output error 0 collision 0 deferred 0 lost carrier 0 no carrier 0 babble 0 late collision 0 output discard 0 Tx pause SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. . output rate 365544 bps. The fol​ low​ ing meth​ ods can be used to de​ ter​ mine the sta​ tus of the mod​ ules in the switch.​ html Module Status Even though the leaves are con​ sid​ ered fixed switches. From a for​ ward​ ing per​ spec​ tive.Monitoring 319 Load-Interval #2: 5 minute (300 seconds) input rate 644416 bps. and the ALE or ALE2 (Ap​ pli​ ca​ tion Leaf En​ gine ASIC) de​ pend​ ing on the gen​ er​ at​ ion of switch hard​ ware. viz.

in the APIC GUI. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/supslot-1/sup https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/lcslot-1/lc CLI The show mod​ ule com​ mand can be used to ob​ tain the sta​ tus of the base mod​ ule and the up​ link mod​ ule.1(0.152) 0. Leaf-1# show module Mod Ports Module-Type --- ----- ----------------------------------.320 Monitoring GUI To ac​ cess mod​ ule sta​ tus for the NFE and the CPU com​ plex. REST API The fol​ low​ ing REST API call(s) can be used to mon​ it​ or the state of the su​ per​ vi​ sor and the mod​ ule.-----------------. To ac​ cess mod​ ule sta​ tus for the ALE/ALE2. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Mod​ ule > Line mod​ ules and the sta​ tus of the mod​ ule is dis​ played in the work pane.---------- Model Status 1 48 1/10G Ethernet Module N9K-C9396PX active GEM 12 40G Ethernet Expansion Module N9K-M12PQ ok Mod Sw Hw --- -------------- ------ 1 11.2050 Mod MAC-Address(es) Serial-Num --- -------------------------------------- ---------- 1 7c-69-f6-0f-87-71 to 7c-69-f6-0f-87-ad SAL17267Z9U . in the APIC GUI. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Mod​ ule > Su​ per​ vi​ sor mod​ ules and the sta​ tus of the mod​ ule is dis​ played in the work pane.

​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.x. GUI To ac​ cess fan sta​ tus for the leaf switch. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Fan Tray and the sta​ tus of the mod​ ules is dis​ played in the work pane.​ cisco. in the APIC GUI. REST API The fol​ low​ ing REST API call(s) and their child ob​ jects can be used to mon​ it​ or the state of the fans on a leaf switch (note that there are 3 slots on this par​ tic​ ul​ ar switch). See: http://​ www. Leaf-1# show environment fan Fan: .​ html Switch Fan Status The fol​ low​ ing sec​ tion de​ scribes method​ olo​ gies to re​ trieve the sta​ tus of the fan trays on the leaf switches. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for mod​ ules. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-1 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-2 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-3 CLI The fol​ low​ ing CLI's can be used to mon​ it​ or the state of the fans on a leaf switch.Monitoring 321 Mod Online Diag Status --- ------------------ 1 pass SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.

x.​ cisco.322 Monitoring -----------------------------------------------------Fan Model Hw Status -----------------------------------------------------Fan1(sys_fan1) N9K-C9300-FAN1-B -- ok Fan2(sys_fan2) N9K-C9300-FAN1-B -- ok Fan3(sys_fan3) N9K-C9300-FAN1-B -- ok Fan_in_PS1 -- -- unknown Fan_in_PS2 -- -- ok Fan Speed: Zone 1: 0x5f Fan Air Filter : Absent SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.​ html). the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for fan trays (http://​ www. in the APIC GUI.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. Power Supply Status The fol​ low​ ing sec​ tions de​ scribe method​ olo​ gies to re​ trieve the sta​ tus of the power sup​ plies on the leaf switches GUI To ac​ cess power sup​ ply sta​ tus for the leaf switch. REST API The fol​ low​ ing REST API call(s) and their child ob​ jects can be used to mon​ it​ or the state of the fans on a leaf switch (note that there are 3 slots on this par​ tic​ ul​ ar switch). nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Power Sup​ ply Units and the sta​ tus of the mod​ ules is dis​ played in the work pane. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/psuslot-1 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/psuslot-2 .

the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for power sup​ plies. .Per module power not available Power Usage Summary: -------------------Power Supply redundancy mode (configured) Non-Redundant(combined) Power Supply redundancy mode (operational) Non-Redundant(combined) Total Power Capacity (based on configured mode) 648 W Total Power of all Inputs (cumulative) 648 W Total Power Output (actual draw) 168 W Total Power Allocated (budget) N/A Total Power Available for additional modules N/A SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.0 Volts Power Supply Model Actual Total Output Capacity (Watts ) (Watts ) Status ------- ------------------- ----------- ----------- 1 UCSC-PSU-650W 0 W 648 W shut 2 UCSC-PSU-650W 168 W 648 W ok Actual Power Draw Allocated Module Model -------------- Status (Watts ) (Watts ) ----------- ----------- 168 W 456 W Powered-Up -------- ------------------- 1 N9K-C9396PX -------------- fan1 N9K-C9300-FAN1-B N/A N/A Powered-Up fan2 N9K-C9300-FAN1-B N/A N/A Powered-Up fan3 N9K-C9300-FAN1-B N/A N/A Powered-Up N/A .x.Monitoring 323 CLI The fol​ low​ ing CLI com​ mands can be used to mon​ it​ or the state of the fans on a leaf switch: Leaf-1# show environment power Power Supply: Voltage: 12.

(T) Telephone. (B) Bridge.​ cisco. (S) Station.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. (C) DOCSIS Cable Device (W) WLAN Access Point. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Pro​ to​ cols > LLDP > Neigh​ bors > eth x/y A full list​ ing of all LLDP neigh​ bors on the in​ ter​ face can be ob​ tained in the work pane.​ html LLDP Neighbor Status The APIC pro​ vides a sin​ gle pane of glass to query and de​ ter​ mine all LLDP neigh​ bors in a fab​ ric. (P) Repeater. To ob​ tain a list of LLDP neigh​ bors on an in​ ter​ face. REST API The fol​ low​ ing rest API call can be used to ob​ tain the same in​ for​ ma​ tion: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/lldp/inst/if-[eth1/1] CLI Leaf-1# show lldp neighbors Capability codes: (R) Router. (O) Other Device ID Local Intf Hold-time Capability Port ID apic2 Eth1/1 120 apic3 Eth1/4 120 4c:4e:35:09:77:2f 5548-2 Eth1/7 120 B Eth1/3 Spine-1 Eth1/49 120 BR Eth4/1 Spine-2 Eth1/50 120 BR Eth4/1 Spine-3 Eth1/51 120 BR Eth1/3 Spine-4 Eth1/52 120 BR Eth1/3 Spine-5 Eth1/53 120 BR Eth1/5 90:e2:ba:4b:fa:d4 . In the above work​ flow click​ ing on Neigh​ bors (in​ stead of eth x/y) gives you a list of all LLDP neigh​ bors on the switch.324 Monitoring See: http://​ www.

r . the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for LLDP.IGMP.Trans-Bridge. B . D . nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod1 > Leaf-X > Pro​ to​ cols > CDP > Neigh​ bors > eth x/y A full list​ ing of all CDP neigh​ bors on the in​ ter​ face can be ob​ tained in the work pane. s .​ html APIC pro​ vides a sin​ gle pane of glass to query and de​ ter​ mine all CDP neigh​ bors in a fab​ ric. V . T .Source-Route-Bridge S .Switch.Repeater.Remotely-Managed-Device.Supports-STP-Dispute Device-ID Local Intrfce Hldtme Capability Platform Port ID .Host.Monitoring 325 Spine-6 Eth1/54 120 BR Eth1/5 Total entries displayed: 9 SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. See: http://​ www.x. I . REST API The fol​ low​ ing rest API call can be used to ob​ tain the same in​ for​ ma​ tion: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/cdp/inst/if-[eth1/1] CLI Leaf-1# show cdp neighbors Capability Codes: R .​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.VoIP-Phone. CDP Neigh​ bor Sta​ tus: GUI To ob​ tain a list of CDP neigh​ bors on an in​ ter​ face. In the above work​ flow click​ ing on Neigh​ bors (in​ stead of eth x/y) gives you a list of all LLDP neigh​ bors on the switch. H .​ cisco.Router.

​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. Then click trou​ bleshoot​ ing in the work pane.​ html GOLD Diagnostic Results GOLD is cov​ ered in greater de​ tail in the Hard​ ware Re​ place​ ment sec​ tion. See: http://​ www. as well as the abil​ ity to run on de​ mand di​ ag​ nos​ tics to iso​ late po​ ten​ tial hard​ ware at fault.​ cisco. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for CDP.x. Then click Trou​ bleshoot​ ing in the work pane. . To view the same for mod​ ules. click on Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-1 > Chas​ sis > Line Mod​ ules > Slot-x.326 Monitoring Services-UCS-A(SSI15450J63) Eth1/5 129 S I s UCS-FI-6248UP Eth1/17 Eth1/7 123 S I s N5K-C5548UP Eth1/3 5548-2(SSI154300VL) SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. GOLD di​ ag​ nos​ tics pro​ vide an easy and quick way for op​ er​ at​ ions teams to con​ firm that bootup and non-dis​ rup​ tive tests that run dur​ ing nor​ mal op​ er​ at​ ions have ex​ e​ cuted prop​ erly. GUI To view GOLD Di​ ag​ nos​ tic test re​ sults in the GUI for the Su​ per​ vi​ sors. click on Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-1 > Chas​ sis > Su​ per​ vi​ sor Mod​ ules > Slot-1.

31) obfl-acc-----------------------> . . U = Untested. 26) ge-eeprom----------------------> . . . 25) act2-acc-----------------------> . 23) mem-health---------------------> . F = Fail. A = Abort. . 41) pcie-bus-----------------------> . . . . 22) cpu-cache----------------------> . 32) nvram-cksum--------------------> . . . 34) asic-scratch-------------------> .Monitoring 327 CLI Leaf-1# show diagnostic result module all Current bootup diagnostic level: bypass Module 1: 1/10G Ethernet Module (Active) Test results: (. I = Incomplete. = Pass. . E = Error disabled) 1) bios-mem-----------------------> . 30) cons-dev-----------------------> . 4) nsa-mem------------------------> . . 24) ssd-acc------------------------> . . 2) mgmtp-lb-----------------------> . 29) usb-bus------------------------> . 40) rtc-test-----------------------> . 33) fpga-reg-chk-------------------> . 6) fabp-prbs: Port 1 2 3 4 5 6 7 8 9 10 11 12 ----------------------------------------.

.

The fol​ low​ ing will pro​ vide an ex​ am​ ple of how sta​ tis​ tics poli​ cies and thresh​ olds can be used to alert the op​ er​ at​ ors when ad​ di​ tional band​ width is re​ quired to a server. First. Select 5 minutes from the granularity column. perform the following actions: a. Click on the edit button next to Monitoring Object. From the stats type drop-down list. This en​ ables them to an​ swer ques​ tions about whether a server needs to be up​ graded from 10G to 40G. 4 In the Create Monitoring Policy dialog box. ACME will con​ fig​ ure a stats col​ lec​ tion pol​ icy to col​ lect the av​ er​ age uti​ liza​ tion of a link over the de​ sired 5 minute in​ ter​ val. 2 In the Navigation pane. To ac​ com​ plish these tasks re​ quires con​ fig​ ur​ ing a mon​ it​ or​ ing pol​ icy with two dis​ tinct types of poli​ cies. e. f. choose L1 Physical Interface Configuration. choose Fabric > Access Policies. Click the edit button next to Stats Type and select Ingress and Egress. or mul​ ti​ ple in​ ter​ faces need to be bonded to pro​ vide more band​ width to the server. choose Ingress. choose Monitoring Policies. ACME's server team has de​ ter​ mined that they would like to mon​ it​ or link uti​ liza​ tion over a 5 minute pe​ riod. Next they will con​ fig​ ure a thresh​ old pol​ icy to gen​ er​ ate a fault when the link uti​ liza​ tion ex​ ceeds var​ io ​us thresh​ olds ac​ cord​ ing to the fol​ low​ ing table. and when the av​ er​ age uti​ liza​ tion is above 80%. g. choose Actions > Create Monitoring Policy. Click +. Expand the newly created monitoring policy and select Stats Collection Policy. they would like to raise a fault for the af​ fected in​ ter​ face. Cre​ ate an in​ ter​ face mon​ it​ or​ ing pol​ icy. and select L1 Physical Interface Configuration. d. and click on update. c.Monitoring 329 Proactive Monitoring Use Cases Monitoring Workload Bandwidth ACME would like to proac​ tively mon​ it​ or con​ nec​ tions to servers to de​ ter​ mine whether ad​ e​ quate band​ width is avail​ able to a given work​ load. . From the Monitoring Object drop-down list. b. 1 On the menu bar. 3 In the Work pane.

3 Click on Ingress Link utilization average value. select the newly created monitoring policy. such as the ag​ gre​ gate amount of traf​ fic to a spe​ cific tier. choose Fabric > Access Policies. so we will flag this as a warn​ ing. we will as​ so​ ci​ ate the newly cre​ ated pol​ icy with an in​ ter​ face pol​ icy group that rep​ re​ sents the in​ ter​ faces we to mon​ it​ or with this pol​ icy. and look for de​ vi​ at​ ions above or below that nor​ mal value. Note: In this ex​ am​ ple. we will mon​ it​ or the amount of traf​ fic to the web tier of a given ap​ pli​ ca​ tion. 4 Enter 0 for the normal value and click the Rising radio button. we will be rais​ ing a warn​ ing when the uti​ liza​ tion goes above 80. Fi​ nally. The set col​ umn spec​ if​ ies the level at which the fault will be raised. 80% uti​ liza​ tion does not nec​ es​ sar​ ily mean that the ap​ pli​ ca​ tion per​ for​ mance is de​ graded. these poli​ cies can be fur​ ther cus​ tomized to pro​ vide a nor​ mal value. EPG Level Statistics The ap​ pli​ ca​ tion owner would like to be able to mon​ it​ or net​ work-re​ lated in​ for​ ma​ tion for their ap​ pli​ ca​ tion. choose Interface Policies > Policy Groups. Re​ peat these steps for the egress sta​ tis​ tics as well. For ex​ am​ ple. b. In the Work pane. 2 In the Thresholds for Collection 5 minute window. From the Monitoring Policy drop down menu. we will apply the pol​ icy to the UCS-10G-PG 1 2 On the menu bar. Select UCS-10G-PG. a. Ad​ di​ tional lev​ els/sever​ it​ ies can also be spec​ if​ ied if de​ sired. For our ex​ am​ ple. In . In this sce​ nario. click +. As an ex​ am​ ple. and the reset col​ umn will spec​ ify the the level at which the fault will be cleared. Next we will con​ fig​ ure a thresh​ old which will alert us if the uti​ liza​ tion ex​ ceeds 80% and as​ sign an de​ fault fault sever​ ity to it.330 Monitoring We have now cre​ ated a pol​ icy which will mon​ it​ or as​ so​ ci​ ated in​ ter​ faces and ingress traf​ fic rate at 5 minute in​ ter​ vals. 1 Click + in the Config Thresholds column. we are in​ ter​ ested in an ab​ solute per​ cent​ age. and clear the warn​ ing when the uti​ liza​ tion falls below 75.

Monitoring 331

this ex​
am​
ple, the de​
fault mon​
i​
tor​
ing poli​
cies are ap​
pro​
pri​
ate, and they are sim​
ply ex​
tract​
ing them from the sys​
tem to be con​
sumed ex​
ter​
nally. This in​
for​
ma​
tion is use​
ful in
sce​
nar​
ios such as a new re​
lease being pushed, and to make sure that no traf​
fic anom​
alies are cre​
ated after the push.
This can be ac​
com​
plished by nav​
i​
gat​
ing to the EPG, and se​
lect​
ing the Stats tab:

EPG-level throughput statistics
Ad​
di​
tion​
ally, this in​
for​
ma​
tion can be gath​
ered from the API:
http[s]://apic_ip/api/node/mo/uni/tn-mynewproject/ap-app1/epg-web-epg.xml?querytarget=self&rsp-subtree-include=stats

Monitoring 333

Reactive Monitoring
It is cru​
cial that the ACME op​
er​
a​
tional staff are able to react to any in​
di​
ca​
tion of some​
thing going wrong. If there is a no​
ti​
fi​
ca​
tion that some​
thing has gone wrong, such as a
fault no​
ti​
fi​
ca​
tion, a low health score, or a ticket/re​
port that end-user func​
tion​
al​
ity has
been im​
pacted, knowl​
edge of the avail​
able mon​
i​
tor​
ing tools is im​
por​
tant for the iden​
ti​
fi​
ca​
tion and col​
lec​
tion of ev​
i​
dence. This ev​
i​
dence can then be used to iden​
tify and an​
a​
lyze the root cause of the prob​
lem be​
fore tak​
ing cor​
rec​
tive ac​
tion. For more in​
for​
ma​
tion
re​
gard​
ing faults and health scores please refer to those spe​
cific sec​
tions within this book.
A deep dive into the processes of trou​
bleshoot​
ing is out of the scope of this book.
bleshoot​
ing Cisco Ap​
pli​
ca​
tion Cen​
tric In​
fra​
struc​
ture: An​
a​
lyt​
i​
cal
Please refer to "Trou​
lem solv​
ing ap​
plied to the pol​
icy dri​
ven data cen​
ter" avail​
able at: http://​
datacenter.​
prob​
github.​
io/​
aci-troubleshooting-book/​

Tenant—Troubleshoot Policies
Within the APIC GUI, under each Ten​
ant you can find a Trou​
bleshoot Pol​
icy sec​
tion.
This sec​
tion will allow con​
fig​
ur​
a​
tion of poli​
cies that are spe​
cific to one ten​
ant, and the
mon​
it​
or​
ing of traf​
fic and test con​
nec​
tiv​
ity be​
tween end​
points.
As seen in the image above, the fol​
low​
ing trou​
bleshoot​
ing poli​
cies can be con​
fig​
ured:

SPAN (Switched Port ANalyzer)—Configuration of SPAN and ERSPAN sources
and destinations to be used in external monitoring of Tenant traffic flows

Endpoint-To-Endpoint Traceroute—Configuration of a path validation tool for
verifying validity of communications between Tenant endpoints in an ACI fabric

Atomic Counters—Configuration of a set of customizable counters to collect
and report on information between a definable set of objects. As shown in the
image below, a policy can be configured to collect statistics between EPs,
between EPGs, between EPGs and specific IPs and other special objects, such
as Any or External ​
traffic flows

Fabric—Troubleshoot Policies
For trou​
bleshoot​
ing within the en​
tire fab​
ric, there are the fol​
low​
ing tools and poli​
cies:

334 Monitoring

SPAN (Switched Port Analyzer)—Configuration of SPAN and ERSPAN sources

On-demand Diagnostics—Configuration of a policy for collection of diagnostic

and destinations to be used in external monitoring of fabric traffic flows
information that can be executed at a point in time and which will return a set
of valuable output for investigation

Leaf Nodes Traceroute—Configuration of a path validation tool for verifying

Traffic Map—At-a-glance hotspot map of node-to-node traffic flow in an ACI

validity of communications between ACI fabric nodes
fabric

Enhanced Troubleshooting Wizard
From ver​
sion 1.1, APIC pro​
vides a trou​
bleshoot​
ing graphic tool to find rel​
e​
vant faults
and sta​
tis​
tics, re​
cent changes, and run con​
nec​
tiv​
ity tests in a sim​
ple man​
ner.
It also gives the op​
tion to gen​
er​
ate a re​
port to save the re​
sults so it can be used as a
ref​
er​
ence.

Other Tools

iPing—A troubleshooting tool in the ACI fabric that can be used to verify
reachability of a device connected to the fabric utilizing the fabric as the
pervasive source

Audit Logs—Audit logs are continually collected on all actions taken in an ACI
fabric and can give a quick indication of which user took which actions at what
time

Reactive Monitoring Use Cases
At the end of this chap​
ter, we are going to de​
scribe two sit​
ua
​t​
ions where ACME ran
into is​
sues and how they made use of the tools pre​
vi​
ously de​
scribed.
1

No access to application: An end-user calls and reports that they can no longer
access a web application running within the fabric.

2

Users report that an application running in the fabric is slow or there is a report
of slowness to the web application running within the fabric.

Monitoring 335

Reactive Monitoring Tools
Switch Port Analyzer (SPAN)
SPAN is gen​
er​
ally used in two ways:

Proactively as part of a third party or offline analysis requirement.
Security tools (IDS/IPS, Data Loss Prevention)
Call Recording

Troubleshooting application and networking issues within the fabric.

It may be help​
ful to per​
form a cap​
ture of some par​
tic​
ul​
ar traf​
fic to see what is going on
within the stream of data. Look​
ing through traf​
fic flows will allow in​
ves​
ti​
ga​
tion of
packet and pro​
to​
col level de​
tails, such as traf​
fic re​
sets, mis​
be​
hav​
ing pro​
to​
cols, im​
proper host re​
quests or re​
sponses, or node level com​
mu​
ni​
ca​
tions. This will pro​
vide deeper in​
sight into how de​
vices are using the net​
work than sim​
ple traf​
fic flow and
fab​
ric con​
fig​
ur​
a​
tion re​
view.
Switched Port AN​
al​
yzer, or SPAN, is a stan​
dard fea​
ture that al​
lows copy and repli​
ca​
tion
of traf​
fic to a net​
work an​
al​
yzer for fur​
ther de​
cod​
ing and in​
ves​
ti​
ga​
tion. It can be used to
copy traf​
fic from one or more ports, VLANs, or end​
point groups (EPGs).
The SPAN fea​
ture process is non-dis​
rup​
tive to any con​
nected de​
vices and is fa​
cil​
it​
ated
in hard​
ware, which pre​
vents any un​
nec​
es​
sary CPU load.
SPAN ses​
sions can be con​
fig​
ured to mon​
it​
or traf​
fic re​
ceived by the source (ingress
traf​
fic), traf​
fic trans​
mit​
ted from the source (egress traf​
fic), or both. By de​
fault, SPAN
mon​
it​
ors all traf​
fic, but fil​
ters can be con​
fig​
ured to mon​
it​
or only se​
lected traf​
fic.

Multinode SPAN
APIC traf​
fic mon​
it​
or​
ing poli​
cies can en​
force SPAN at the ap​
pro​
pri​
ate places to copy
traf​
fic from mem​
bers of each End Point Group wher​
ever they are con​
nected. If a mem​
ber moves, APIC au​
to​
mat​
ic
​ally pushes the pol​
icy to the new leaf switch. For ex​
am​
ple,
when a VMo​
tion event re​
lo​
cates an End​
point to a new leaf switch, the SPAN fea​
ture
con​
fig​
ur​
a​
tion au​
to​
mat​
ic
​ally ad​
justs.

336 Monitoring

SPAN Guidelines and Restrictions

Use SPAN for troubleshooting. SPAN traffic competes with user traffic for
switch resources. To minimize the load, configure SPAN to copy only the
specific traffic that you want to analyze.

An l3extLIfP Layer 3 subinterface cannot be configured as a SPAN source. The
entire port must be used for monitoring traffic from external sources.

Tenant and access SPAN use the encapsulated remote extension of SPAN
(ERSPAN) type I, while fabric SPAN uses ERSPAN type II. For information
regarding ERSPAN headers, refer to the IETF Internet Draft at this URL: https:/​
/​
tools.ietf.org/​
html/​
draft-foschiano-erspan-00.

See the Verified Scalability Guide for Cisco ACI document for SPAN-related
limits, such as the maximum number of active SPAN sessions.

Configuring a SPAN Session
This pro​
ce​
dure shows how to con​
fig​
ure a SPAN pol​
icy to for​
ward repli​
cated source
pack​
ets to a re​
mote traf​
fic an​
al​
yzer.
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the tenant.

3

In the Navigation pane, choose Tenant_Name > Troubleshooting Policies >
SPAN > SPAN Destination Groups.

4

In the Work pane, choose Actions > Create SPAN Destination Group.

5

In the Create SPAN Destination Group dialog box, perform the following
actions:
a. In the Name field, enter a name for the SPAN destination group.
b. In the Destination EPG dropdowns, select the destination Tenant,
Application Profile, and EPG.
c. Enter the Destination IP.
d. Enter the Source IP Prefix.
e. Optionally, modify the other fields as needed.
f. Click OK.
g. If needed, add additional destinations.
h. Click Submit.

Monitoring 337

6

Under SPAN, right-click SPAN Source Groups and choose Create SPAN Source
Group.

7

In the Create SPAN Source Group dialog box, perform the following actions:
a. In the Name field, enter a name for the SPAN source group.
b. From the Destination Group drop-down list, choose the SPAN destination
group that you configured previously.
c. In the Create Sources table, click the + icon to open the Create Sources
dialog box.
d. In the Name field, enter a name for the source.
e. In the Direction field, choose the radio button based on whether you want to
replicate and forward packets that are incoming to the source, outgoing from
the source, or both incoming and outgoing.
f. From the Source EPG drop-down list, choose the EPG (identified by
Tenant/ApplicationProfile/EPG) whose packets will be replicated and
forwarded to the SPAN destination. Click OK to save the SPAN source.
g. Click Submit to save the SPAN source group.

Traceroute
Tracer​
oute is a use​
ful fea​
ture in tra​
di​
tional net​
work​
ing. In ACI this fea​
ture is im​
ple​
mented tak​
ing into ac​
count the way the fab​
ric works.
Tracer​
oute sup​
ports a va​
ri​
ety of modes, in​
clud​
ing end​
point-to-end​
point, and leaf-toleaf (tun​
nel end​
point, or TEP to TEP). It dis​
cov​
ers all paths across the fab​
ric, dis​
cov​
ers
point of exits for ex​
ter​
nal end​
points, and helps to de​
tect if any path is blocked.
A tracer​
oute that is ini​
ti​
ated from the ten​
ant end​
points shows the de​
fault gate​
way as
an in​
ter​
me​
di​
ate hop that ap​
pears at the ingress leaf switch.
Note: If tracer​
oute is done from the OS of a con​
nected server or VM, it will show the
hops for the leaves and spines as un​
known, and will keep record​
ing the in​
for​
ma​
tion
after the packet gets out of the fab​
ric. For more pre​
cise in​
for​
ma​
tion, please use tracer​
oute from the APIC (GUI or CLI)

338 Monitoring

Traceroute Guidelines and Restrictions

When the traceroute source or destination is an endpoint, the endpoint must
be dynamic and not static. Unlike a dynamic endpoint (fv:CEp), a static endpoint
(fv:StCEp) does not have a child object (fv:RsCEpToPathEp) that is required for
traceroute.

See the Verified Scalability Guide for Cisco ACI document for traceroute-related
limits.

Traceroute results will display the IP address of the remote node and interface
which it came it came in on. (Browse on the APIC GUI to Fabric | Inventory |
Fabric Management to view the IP address information for correlation.)

Traceroute cannot be used for endpoints that reside in an external EPG.

Performing a Traceroute Between Endpoints
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the tenant.

3

In the Navigation pane, choose Tenant_Name > Troubleshooting Policies >
Endpoint-to-Endpoint Traceroute Policies.

4

In the Work pane, choose Actions > Create Endpoint-to-Endpoint Traceroute
Policy.

5

In the Create Endpoint-to-Endpoint Traceroute Policy dialog box, perform the
following actions:
a. In the Name field, enter a name for the traceroute policy.
b. In the Source End Points table, click the + icon to edit the traceroute source.
c. From the Source MAC drop-down list, choose or enter the MAC address of
the source endpoint and click Update.
d. In the Destination End Points table, click the + icon to edit the traceroute
destination.
e. From the Destination MAC drop-down list, choose or enter the MAC address
of the destination endpoint and click Update.
f. In the State field, click the Start radio button.
g. Click Submit to launch the traceroute.

6

In the Navigation pane or the Traceroute Policies table, click the traceroute
policy. The traceroute policy is displayed in the Work pane.

Monitoring 339

7

In the Work pane, click the Operational tab, click the Source End Points tab,
and click the Results tab.

8

In the Traceroute Results table, verify the path or paths that were used in the trace.
a. More than one path might have been traversed from the source node to the
destination node.
b. For readability, increase the width of one or more columns, such as the Name
column.

Atomic Counters
Atomic Coun​
ters are use​
ful for trou​
bleshoot​
ing con​
nec​
tiv​
ity be​
tween end​
points, EPGs,
or an ap​
pli​
ca​
tion within the fab​
ric. A user re​
port​
ing ap​
pli​
ca​
tion may be ex​
pe​
ri​
enc​
ing
slow​
ness, or atomic coun​
ters may be needed for mon​
it​
or​
ing any traf​
fic loss be​
tween
two end​
points. One ca​
pa​
bil​
ity pro​
vided by atomic coun​
ters is the abil​
ity to place a
trou​
ble ticket into a proac​
tive mon​
it​
or​
ing mode, for ex​
am​
ple when the prob​
lem is in​
ter​
mit​
tent, and not nec​
es​
sar​
ily hap​
pen​
ing at the time the op​
er​
at​
or is ac​
tively work​
ing
the ticket.
Atomic coun​
ters can help de​
tect packet loss in the fab​
ric and allow the quick iso​
la​
tion of the
source of con​
nec​
tiv​
ity is​
sues. Atomic coun​
ters re​
quire NTP to be en​
abled on the fab​
ric.
Leaf-to-leaf (TEP to TEP) atomic coun​
ters can pro​
vide the fol​
low​
ing:

Counts of drops, ad​
mits, and ex​
cess pack​
ets

Short-term data col​
lec​
tion such as the last 30 sec​
onds, and long-term data col​
lec​
tion such as 5 min​
utes, 15 min​
utes, or more

A break​
down of per-spine traf​
fic (avail​
able when the num​
ber of TEPs, leaf or

On​
go​
ing mon​
it​
or​
ing

VPC, is less than 64)

Leaf-to-leaf (TEP to TEP) atomic coun​
ters are cu​
mu​
la​
tive and can​
not be cleared. How​
ever, be​
cause 30 sec​
ond atomic coun​
ters reset at 30 sec​
ond in​
ter​
vals, they can be used
to iso​
late in​
ter​
mit​
tent or re​
cur​
ring prob​
lems.
Ten​
ant atomic coun​
ters can pro​
vide the fol​
low​
ing:

Application-specific counters for traffic across the fabric, including drops,
admits, and excess packets

Choose or enter the identifying information for the traffic destination. Note that a single target endpoint could have multiple IP addresses associated with it. They do not take into ac​ count drops or error coun​ ters in a hard​ ware level. c. Configuring Atomic Counters 1 On the menu bar. or IP address).*** Dropped pack​ ets are cal​ cu​ lated when there are less pack​ ets re​ ceived by the des​ ti​ na​ tion than trans​ mit​ ted by the source. Traffic can be measured between a combination of endpoints. Choose or enter the identifying information for the traffic source. 2 In the Work pane. external interfaces. endpoint groups. . In the Add Policy dialog box. perform the following actions: a. choose the tenant. endpoint group. enter a name for the policy. 3 In the Navigation pane. choose Actions > Add Policy_Name Policy. choose Tenants > ALL TENANTS. external interface.340 Monitoring • Modes include the following: • Endpoint to endpoint MAC address. Ex​ cess pack​ ets are cal​ cu​ lated when there are more pack​ ets re​ ceived by the des​ ti​ na​ tion than trans​ mit​ ted by the source. In the Name field. The required identifying information differs depending on the type of source (endpoint. 5 6 In the Work pane. and IP addresses. choose Tenant_Name > Troubleshooting Policies > Atomic Counter Policy. • EPG to EPG with optional drill down • EPG to endpoint • EPG to * (any) • Endpoint to external IP address ***Atomic coun​ ters track the amount pack​ ets of be​ tween the two end​ points and use this as a mea​ sure​ ment. b. 4 Choose a policy (traffic topology). or endpoint to endpoint IP address.

Optional: (Optional) In the Filters table. choose Fabric > Fabric Policies. click + to specify filtering of the traffic to be counted. specify filtering by the IP protocol number (TCP=6. Traf​ fic maps make use of atomic coun​ ters to con​ tin​ u​ ously mon​ i​ tor and dis​ play traf​ fic be​ tween leaf switches to help with quick de​ bug​ ging and iso​ la​ tion of ap​ pli​ ca​ tion con​ nec​ tiv​ ity is​ sues. . The policy configuration is displayed in the Work pane. choose the Operational tab and choose the Traffic subtab to view the atomic counter statistics. Click Submit to save the atomic counter policy. Traffic Map Low per​ for​ mance and con​ ges​ tion can be iden​ ti​ fied by use of Traf​ fic maps. 7 In the Navigation pane. for example) and by source and destination IP port numbers. e. 2 In the Navigation pane. In the resulting Create Atomic Counter Filter dialog box. Configuring Traffic Map 1 On the menu bar. under the selected topology. choose Troubleshooting Policies > Traffic Map. choose the new atomic counter policy.Monitoring 341 d. 8 In the Work pane.

Show any related recent changes in the path between EPs. and click search. click the wrench icon to launch the enhanced troubleshooting wizard. 1 In the menu bar. In the Source field. c. In the Name field. d. f. highlighting the affected component. Show the faults in the path between the selected EPs. Run traceroute between EPs. you can click the following items for more information: a. the En​ hanced Trou​ bleshoot​ ing Wiz​ ard may be used to quickly iden​ tify con​ nec​ tiv​ ity is​ sues. c. b. click the check box.342 Monitoring 3 Set the drop-down menu options to view the source Leaf to destination Leaf traffic paths. In the Destination field. 2 In the Create SPAN Destination Group dialog box. received. 3 In the next screen. enter a name for the session. Select the Time Window duration for the debug. e. b. dropped and excess • All spines and a specific spine switch The per​ cent​ age is shown in rel​ at​ ive terms to all traf​ fic by Source or re​ ceived by Des​ ti​ na​ tion. . enter the MAC or IP address for the source endpoint. The En​ hanced Trou​ bleshoot​ ing Wiz​ ard pro​ vides a sin​ gle lo​ ca​ tion that in​ cludes sev​ eral com​ monly used tools and out​ puts re​ quired for trou​ bleshoot​ ing end point con​ nec​ tiv​ ity. • Last interval and cumulative • Sent. Show relevant statistics for those EPs. Enhanced Troubleshooting Wizard When trou​ bleshoot​ ing con​ nec​ tiv​ ity be​ tween end​ points within the fab​ ric. enter the MAC or IP address for the destination endpoint. 4 Clicking on a cell opens a table with all data for all trails and links. Optional: Generate a Report to download the results. d. Click Start. Optional: If it is an external IP address. perform the following actions: a. and click search.

154): 56 data bytes 64 bytes from 10.59.59.59. IPing IPing is used to test and val​ i​ date con​ nec​ tiv​ ity within the from leaf node to end​ points within the fab​ ric. Configure SPAN between EPs g.154 (10.0.245 ms .154: icmp_seq=1 ttl=55 time=0.0.59.154: icmp_seq=0 ttl=55 time=0.0.254 ms 64 bytes from 10.Monitoring 343 e.256 ms 64 bytes from 10.0.59.154: icmp_seq=2 ttl=55 time=0. IPing is a trou​ bleshoot​ ing tool for net​ work users sim​ i​ lar to the PING com​ mand.0. Show the configured contracts between EPs.59. f. Using IPing iping [ -V vrf ] [ -c count ] [ -i wait ] [ -p pattern ] [ -s packetsize ] [ -t timeout ] host Syntax Description Examples pod1-leaf1# iping -V overlay-1 10.0.154 PING 10. tak​ ing into ac​ count the pri​ vate net​ work. Configure Atomic Counters between EPs.

choose the Events subtab to view the event log. Audit logs are records of who made a change.154: icmp_seq=4 ttl=55 time=0. 7 Double-click a log entry to view additional details about the event.0. 5 packets received. 6 Under the History tab. choose Common. 2 In the Work pane. 3 In the Navigation pane.59. This pro​ ce​ dure shows how to view ten​ ant events as an ex​ am​ ple.344 Monitoring 64 bytes from 10.0. choose the Audit Log subtab to view the audit log. the audit log can be viewed.154: icmp_seq=3 ttl=55 time=0.256 ms Audit Logs At times it may be re​ quired to view changes which have taken place in the fab​ ric. 0. choose Common. 5 Under the History tab.23/0. or data pulled for an audit re​ quire​ ment. Viewing Audit Logs Pro​ ce​ dure (ex​ am​ ple for view​ ing audit log on a ten​ ant) 1 On the menu bar. Audit logs can be found in sev​ eral places within the GUI. Wher​ ever a His​ tory tab ap​ pears in the GUI Work pane.00% packet loss round-trip min/avg/max = 0. . choose Tenants > ALL TENANTS.59.10. when the change was made. An out​ age re​ ported on a host or ap​ pli​ ca​ tion in the fab​ ric may need to be tracked. and a de​ scrip​ tion of the ac​ tion.154 ping statistics --5 packets transmitted. Audit logs also record when users logon and lo​ goff.245/0.23 ms --.241 ms 64 bytes from 10. 4 In the Work pane. choose the History tab.0.59. fil​ tered to show only those events rel​ e​ vant to the cur​ rent GUI con​ text.

bridge do​ main and Ten​ ant. 3 In the Navigation pane. 2 Verify that the required contracts are in place between the EPs. and each EP is con​ nected to dif​ fer​ ent leaf switches. choose Tenants > ALL TENANTS. Ca​ bles and con​ nec​ tiv​ ity have been ver​ if​ ied. choose the tenant.Monitoring 345 Reactive Monitoring Use Cases This chap​ ter will show some ex​ am​ ples of how ACME's op​ er​ at​ ions teams can use their ACI mon​ it​ or​ ing tools to react to a few com​ mon pos​ si​ ble sce​ nar​ ios. Verify the required contracts are in place between the EPs . Check that the EPs have been learned by the leaf switches 1 On the menu bar. mem​ ory and CPU uti​ liza​ tion has been checked. there​ fore the de​ fault gate​ way for both EPs exist in the leaf switches. etc. These two End Points (EPs) be​ long to two dif​ fer​ ent End Point Groups (EPGs). 2 In the Work pane. hosts are up. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name. Note: these ex​ am​ ples as​ sume that basic low-level in​ ves​ ti​ ga​ tion has been done and the issue has been iso​ lated to an issue with traf​ fic flows across the fab​ ric. processes are run​ ning. The bridge do​ main has uni​ cast rout​ ing en​ abled. within the same sub​ net. VMs are run​ ning. 4 In the Work pane. Loss of Connectivity to Endpoint The ACME ap​ pli​ ca​ tion owner has re​ ported that two servers have lost con​ nec​ tiv​ ity. choose the Operational tab and verify that the endpoint is present. 5 Repeat this procedure for the destination EPG. The fol​ low​ ing steps trou​ bleshoot this sit​ ua ​t​ ion: 1 Check that the EPs have been learned by the leaf switches.

examine the troubleshooting policies. choose the tenant. One method could be to use endpoint-to-endpoint traceroute to show if there are paths available between those endpoints. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name. 10 Alternate techniques are available to validate communications between endpoints within the fabric. choose Tenants > ALL TENANTS.346 Monitoring Verify the required contracts are in place between the EPs 1 On the menu bar. A good starting place is the Enhanced Troubleshooting Wizard. 6 Check for a relationship between the source EPG and destination EPG. 7 Click on the contract to verify the contents of the contract. This displays the filters that are present within that contract. To check the connectivity to the remote end. b. and make sure frames have the right format. 4 In the Work pane. . noting the direction of the arrows. Another option inside the fabric could be to utilize the iPing tool to verify connectivity between the default gateway and the endpoints. 2 In the Work pane. 8 Inspect the contents of each filter by examining the contract under the Security Policies folder and verifying that each filter contains the appropriate filter entries. a. choose the Operational tab. the connectivity between endpoints and leaf switches is not the problem. Each leaf has an SVI used as default gateway. use iPing from each leaf to the remote endpoint using the default gateway as source. 9 If the endpoints are discovered within each EPG and the contract relationships look correct. In the event that these things seem valid. If this test is successful. 3 In the Navigation pane. it then might be necessary to use SPAN to verify where the traffic is entering and leaving the fabric. 5 Choose the Contracts subtab.

packet drops. or in​ ter​ mit​ tent con​ nec​ tiv​ ity loss. . Look​ ing through the Traf​ fic Map can also show an at-a-glance view in the fab​ ric and high​ light any pos​ si​ ble hotspots or con​ ges​ tion in cer​ tain el​ e​ ments of the fab​ ric. SPAN may be used to ver​ ify where the traf​ fic is en​ ter​ ing and leav​ ing the fab​ ric and make sure frames have the right for​ mat. they can use EP-to-EP Tracer​ oute to show the path va​ lid​ ity. In this case. Cor​ rupted frames could cause drops in dif​ fer​ ent points. Check spe​ cific coun​ ters for CRC er​ rors on the in​ ter​ faces used to trans​ mit EP spe​ cific traf​ fic. Atomic coun​ ters can also be de​ ployed to check if there are any drops or ir​ reg​ ul​ ar​ it​ ies be​ tween the de​ fined de​ vices. The VM and the web por​ tal be​ long to a dif​ fer​ ent EPGs in the same bridge do​ main and ten​ ant. This per​ for​ mance issue could be caused due to la​ tency. If every​ thing in the fab​ ric looks fine.Monitoring 347 Users Report that an Application Running in the Fabric is Slow ACME test users re​ port slow re​ sponse of the web servers run​ ning in the fab​ ric. from the EPs on a OS level to the switches. First the op​ er​ at​ ions staff should make sure end points are learned by the leaf switches in a con​ sis​ tent way so that the EPs are vis​ ib ​le in the op​ er​ at​ ional tab under the Ten​ ant-> Ap​ pli​ ca​ tion Pro​ file-> EPGs-> Op​ er​ at​ ional Tab Once they ver​ ify end​ points have been learned. the end-user is try​ ing to ac​ cess the web por​ tal from a test VM.

.

349 Scripting .

.

Scripting 351 Section Content • Leveraging Network Programmability Reference to Object Model Programmatic Interfaces REST Read Operations Write Operations Authentication Filters • API Inspector • Development Techniques • POSTman Installation Collections Build Login request Make Query to APIC Make Configuration Change in APIC Use API Inspector for Query Guidance • Cobra SDK and Arya Establish Session Work with Objects Cisco APIC REST to Python Adapter .

352 Scripting • ACI Toolkit ACI Toolkit Applications Endpoint Tracker ACI Lint • GitHub Source Control GitHub "It's on github" .

using com​ mon tools and lan​ guages to pro​ vide net​ work en​ gi​ neers. every​ one can ben​ e​ fit. al​ low​ ing peo​ ple to free them​ selves from repet​ it​ ive tasks that could be more eas​ ily ac​ com​ plished by a ma​ chine. de​ vel​ op​ ers and even novices an ap​ proach​ able path to​ ward au​ toma​ tion. yield​ ing faster time to res​ ol​ u​ tion and higher cus​ tomer sat​ is​ fac​ tion. Though ACME is just get​ ting started with true De​ vOps in their IT or​ ga​ ni​ za​ tion. they re​ al​ ize that these ben​ e​ fits will allow them to keep up with the pace of busi​ ness.Scripting 353 Leveraging Network Programmability The in​ dus​ trial rev​ ol​ u​ tion mod​ ern​ ized the tech​ niques used to man​ uf​ ac​ ture goods. ACI is able to take ad​ van​ tage of all of these ben​ e​ fits by com​ pletely ex​ pos​ ing all of the na​ tive func​ tion​ al​ ity in pro​ gram​ ma​ ble ways. a more cost-ef​ fec​ tive way to rapidly pro​ vi​ sion in​ fra​ struc​ ture in a timely fash​ ion ac​ cord​ ing to de​ mand. Their op​ er​ at​ ions teams can uti​ lize the plethora of in​ for​ ma​ tion con​ tained within the APIC to stream​ line their processes. The as​ so​ ci​ ated de​ crease in costs. yield​ ing a higher qual​ ity prod​ uct. ACME's net​ work en​ gi​ neer​ ing and de​ sign teams can ben​ e​ fit from the quick time to pro​ vi​ sion large con​ fig​ ur​ a​ tions. in​ crease in speed and in​ creased qual​ ity al​ lowed for more work to be done for less money in less time. and yielded more con​ sis​ tency in the con​ fig​ ured re​ sults. . and the con​ sis​ tency pro​ vided by the abil​ ity to au​ to​ mate all of the mov​ ing parts. Pro​ gram​ ma​ bil​ ity promises to offer the same out​ come for net​ works as the in​ dus​ trial rev​ ol​ u​ tion did for goods. The in​ evitable move to​ ward au​ toma​ tion in the IT in​ dus​ try has pro​ vided peo​ ple and busi​ nesses a faster way to achieve their de​ sired goals. going from hand pro​ duc​ tion meth​ ods to mech​ an ​ized man​ uf​ ac​ tur​ ing. gather bet​ ter met​ rics and cor​ re​ late events more ac​ cu​ rately. This move​ ment from man​ ual to au​ to​ mated op​ er​ at​ ions changed human pro​ duc​ tiv​ ity. Given the com​ pre​ hen​ sive​ ness of the pro​ gram​ ma​ bil​ ity fea​ tures avail​ able on ACI.

.

ACI uses an ad​ vanced ob​ ject model that rep​ re​ sents net​ work con​ fig​ u​ ra​ tion with ap​ pli​ ca​ tion-based se​ man​ tics which can be con​ sumed and posted against using a well doc​ u​ mented REST API. Tra​ di​ tional net​ work​ ing de​ vices pro​ vide out​ put that is meant for vi​ sual con​ sump​ tion by peo​ ple. at a va​ ri​ ety of lev​ els that will cater to the level of com​ fort the user has with pro​ gram​ ming. how​ ever the meth​ ods by which these goals may be re​ al​ ized have been more dif​ fi​ cult to grasp. and also can eas​ ily rep​ re​ sent the full de​ tail that a com​ pre​ hen​ sive ob​ ject-ori​ ented con​ fig​ u​ ra​ tion model may rep​ re​ sent. and con​ fig​ u​ ra​ tions are dri​ ven using text input that is sim​ pler for a per​ son to type. In ad​ di​ tion to pro​ vid​ ing this in​ ter​ face into the ob​ ject model. Struc​ tured data that may not be vi​ su​ ally ap​ peal​ ing can be rapidly parsed. from a pro​ gram​ ma​ bil​ ity per​ spec​ tive it is im​ por​ tant to note that every as​ pect of ACI func​ tion​ al​ - . all of which use open stan​ dards and open source. Ma​ chines are able to more eas​ ily process data that is pro​ vided in some struc​ tured form. ACI also pro​ vides a num​ ber of ac​ cess meth​ ods to read and ma​ nip​ u​ late this data. how​ ever these goals stand in con​ trast to an au​ toma​ tion-dri​ ven ap​ proach. Reference to Object Model Representation of the top levels of the Object Model While a com​ pre​ hen​ sive overview of the Ob​ ject Model is out​ side of this book.Scripting 355 ACI and Scripting The goals for net​ work pro​ gram​ ma​ bil​ ity are clear.

and made to cater for the user's needs. so long as the de​ vice has some form of in​ ter​ face which is ac​ ces​ si​ ble via IP. into sub​ or​ di​ nate de​ vices. which can reach the de​ vice through their na​ tive con​ fig​ ur​ a​ tion in​ ter​ faces. This in​ cludes in​ ter​ nal fab​ ric net​ work​ ing. This in​ ter​ face is a cru​ cial com​ po​ nent for the GUI and CLI. and every as​ pect can be in​ spected.356 Scripting ity is en​ com​ passed within the ob​ ject model. These de​ vices can be from any ven​ dor. SOAP or oth​ ers. CLI. with​ out the need to in​ di​ vid​ u​ ally con​ fig​ ure those de​ vices. This is a key as​ pect to the open​ ness of the ACI fab​ ric. This data is stored within the Man​ age​ ment In​ for​ ma​ tion Tree. and also pro​ vides a touch point for au​ toma​ tion tools. The ac​ tual im​ ple​ men​ ta​ tion of de​ vice pack​ ages is done via Python scripts which run on the APIC within a con​ tained ex​ e​ cu​ tion en​ vi​ ron​ ment. ex​ ter​ nal net​ work​ ing. as well as pro​ vid​ ing ac​ cess to man​ age​ ment func​ tions for the con​ troller. As a . be that REST. as well as ex​ tend​ ing that con​ fig​ ur​ a​ tion into sub​ or​ di​ nate com​ po​ nents. can be made pro​ gram​ mat​ ic ​ally using the REST API. South​ bound in​ ter​ faces on APIC allow for the de​ clar​ at​ ive model of in​ tent to be ex​ tended be​ yond the fab​ ric. and as such is a crit​ ic ​al as​ pect of the ar​ chi​ tec​ ture for being able to pro​ vide a con​ sis​ tent pro​ gram​ matic ex​ pe​ ri​ ence. and con​ sis​ tency rules that are en​ forced. This en​ sures that the con​ fig​ ured state of the model will never get out of hand with stale nodes or en​ tries. com​ pute in​ te​ gra​ tion. and all other facets of the prod​ uct. The L4-7 de​ vice pack​ age in​ ter​ face al​ lows for ACI to apply pol​ icy to ex​ ist​ ing L4-7 de​ vices that do not have an im​ plicit knowl​ edge of ACI pol​ icy. Programmatic Interfaces APIC is very flex​ ib ​le in terms of how it can ac​ cept con​ fig​ ur​ a​ tion and pro​ vide ad​ min​ is​ tra​ tive and op​ er​ ab ​le states. iden​ tity. with every piece of the model rep​ re​ sented as a pro​ gram​ matic ob​ ject with prop​ er​ ties. vir​ tu​ al​ iza​ tion in​ te​ gra​ tion. This means that all of the con​ fig​ ur​ a​ tion that can be made on the fab​ ric. This south​ bound ex​ ten​ sion is re​ al​ ized through two meth​ ods: L4-7 De​ vice Pack​ ages and OpFlex. in that pol​ icy can be pro​ grammed once via APIC and then pushed out to hy​ per​ vi​ sors. pro​ vi​ sion​ ing scripts and third party mon​ it​ or​ ing and man​ age​ ment tools. L4-7 de​ vices and po​ ten​ tially more in the fu​ ture. There are two pri​ mary cat​ e​ gories of in​ ter​ faces that fa​ cil​ it​ ate these func​ tions: the north​ bound REST API and the south​ bound pro​ gram​ matic in​ ter​ faces. The north​ bound REST API is re​ spon​ si​ ble for ac​ cept​ ing con​ fig​ ur​ a​ tion. The REST API is a sin​ gu​ lar entry point to the fab​ ric for mak​ ing con​ fig​ ur​ a​ tion changes. ma​ nip​ u-​ lated.

REST The Cisco APIC REST API is a pro​ gram​ matic in​ ter​ face for Cisco APIC that uses REST ar​ chi​ tec​ ture. In ACI. so that when​ ever in​ for​ ma​ tion is dis​ played. OpFlex is cur​ rently used to ex​ tend pol​ icy to the Ap​ pli​ ca​ tion Vir​ tual Switch as well as ex​ tend Group Based Pol​ icy into Open​ Stack. and any prop​ er​ ties of that ob​ ject are spec​ if​ ied as at​ trib​ utes of that el​ e​ ment. faults. and DELETE op​ er​ at​ ions through HTTP. and SDK. The GET method is nul​ lipo​ tent. OpFlex it​ self does not dic​ tate the in​ for​ ma​ tion model. Pay​ loads to and from the REST in​ ter​ face can be en​ cap​ su​ lated through ei​ ther XML or JSON en​ cod​ ing. GUI. The pro​ to​ col is de​ signed to sup​ port XML and JSON (as well as the bi​ nary en​ cod​ ing used in some sce​ nar​ ios) and to use stan​ dard re​ mote pro​ ce​ dure call (RPC) mech​ an ​isms such as JSON-RPC over TCP. and audit events. The REST API is the in​ ter​ face into the MIT and al​ lows ma​ nip​ ul​ a​ tion of the ob​ ject model state. and when con​ fig​ ur​ a​ tion changes are made. mean​ ing that there is no ad​ di​ tional ef​ fect if they are called more than once with the same input pa​ ra​ me​ ters. OpFlex is de​ signed to allow a data ex​ change of a set of man​ aged ob​ jects that is de​ fined as part of an in​ for​ ma​ tional model. and it even pro​ vides a means of sub​ scrib​ ing to push-based event no​ ti​ fi​ ca​ tion. Stan​ dard REST meth​ ods are sup​ ported on the API. .Scripting 357 user makes changes to ser​ vice graphs or EPG pol​ icy. You can use any pro​ gram​ ming lan​ guage to gen​ er​ ate the mes​ sages and the JSON or XML doc​ um ​ents that con​ tain the API meth​ ods or MO de​ scrip​ tions. in​ clud​ ing sta​ tis​ tics. Con​ tain​ ment is de​ fined by cre​ at​ ing child el​ e​ ments. which in​ cludes POST. The POST and DELETE meth​ ods are idem​ po​ tent. In the case of XML. an event can be sent through a web socket. the en​ cod​ ing op​ er​ at​ ion is sim​ ple: the el​ e​ ment tag is the name of the pack​ age and class. and can be used with any tree-based ab​ stract model in which each node in the tree has a uni​ ver​ sal re​ source iden​ ti​ fier (URI) as​ so​ ci​ ated with it. mean​ ing that it can be called zero or more times with​ out mak​ ing any changes (or that it is a read-only op​ er​ at​ ion). they are writ​ ten through the REST API. The REST API also pro​ vides an in​ ter​ face through which other in​ for​ ma​ tion can be re​ trieved. The same REST in​ ter​ face is used by the Cisco APIC com​ mand-line in​ ter​ face (CLI). so that when a change oc​ curs in the MIT. GET. it is read through the REST API. The API ac​ cepts and re​ turns HTTP (not en​ abled by de​ fault) or HTTPS mes​ sages that con​ tain JavaScript Ob​ ject No​ ta​ tion (JSON) or Ex​ ten​ si​ ble Markup Lan​ guage (XML) doc​ um ​ents. the de​ vice pack​ age will trans​ late the APIC pol​ icy into API calls on the L4-7 de​ vice.

in​ di​ cat​ ing that the API will be in​ voked. and the value is another nested dictionary with two keys: attribute and children. Next in the re​ quest URI is the lit​ eral string /api. up​ date. so the next part of the URI spec​ i​ fies . • The children key contains a list that defines all the child objects. • All objects are described as JSON dictionaries. as dis​ cussed ear​ lier. Read Operations After the ob​ ject pay​ loads are prop​ erly en​ coded as XML or JSON. The first two sec​ tions of the re​ quest URI sim​ ply de​ fine the pro​ to​ col and ac​ cess de​ tails of Cisco APIC.358 Scripting For JSON. • The attribute key contains a further nested dictionary describing key-value pairs that define attributes on the object. read. The children in this list are dictionaries containing any nested objects. en​ cod​ ing re​ quires de​ f​ i​ n​ i​ tion of cer​ tain en​ ti​ ties to re​ flect the tree-based hi​ er​ ar​ chy. read op​ er​ a​ tions are for an ob​ ject or class. The fol​ low​ ing di​ a​ gram shows the syn​ tax for a read op​ er​ a​ tion from the REST API. which are defined as described here. REST syntax Be​ cause the REST API is HTTP based. so it is fairly sim​ ple to im​ ple​ ment after it is ini​ tially un​ der​ stood. they can be used in cre​ ate. the de​ f​ i​ n​ i​ tion is re​ peated at all lev​ els of the tree. how​ ever. Gen​ er​ ally. or delete op​ er​ a​ tions on the REST API. in which the key is the name of the package and class. defin​ ing the uni​ ver​ sal re​ source iden​ ti​ fier (URI) to ac​ cess a cer​ tain re​ source type is im​ por​ tant.

Both cre​ ate and up​ date op​ er​ a​ tions can con​ tain com​ plex ob​ ject hi​ er​ ar​ chies. it will be cre​ ated. or the pack​ age and class name for class-based queries. the con​ fig​ u​ ra​ tion should be trans​ par​ ent to the user. The final manda​ tory part of the re​ quest URI is the en​ cod​ ing for​ mat: ei​ ther .Scripting 359 whether the op​ er​ a​ tion will be for an MO or class.xml or . The next com​ po​ nent de​ fines ei​ ther the fully qual​ i​ fied Dn being queried for ob​ ject-based queries. REST Payload .json. so that a com​ plete tree can be de​ fined in a sin​ gle com​ mand so long as all ob​ jects are within the same con​ text root and are under the 1MB limit for data pay​ loads for the REST API. For the most part. Write Operations Cre​ ate and up​ date op​ er​ a​ tions in the REST API are both im​ ple​ mented using the POST method. The con​ text root helps de​ fine a method by which Cisco APIC dis​ trib​ utes in​ for​ ma​ tion to mul​ ti​ ple con​ trollers and helps en​ sure con​ sis​ tency. This is the only method by which the pay​ load for​ mat is de​ fined (Cisco APIC ig​ nores Con​ tent-Type and other head​ ers). it will be up​ dated to re​ flect any changes be​ tween its ex​ ist​ ing state and de​ sired state. This limit is in place to guar​ an​ tee per​ for​ mance and pro​ tect the sys​ tem under high load. though very large con​ fig​ u​ ra​ tions may need to be bro​ ken into smaller pieces if they re​ sult in a dis​ trib​ uted trans​ ac​ tion. so that if an ob​ ject does not al​ ready exist. and if it does al​ ready exist.

be​ cause you can​ not make changes to every ob​ ject of a spe​ cific class (nor would you want to). fol​ lowed next by the ac​ tual Dn. Filters The REST API sup​ ports a wide range of flex​ i​ ble fil​ ters. <aaaUser name='admin' pwd='in​ sieme'/>. in​ clud​ ing aaaLo​ gin. you can pass the rsp-sub​ tree=mod​ i​ fied query string to in​ di​ cate that you want the re​ sponse to in​ clude any ob​ jects that have been mod​ i​ fied by your POST op​ er​ a​ tion. if you want to re​ trieve the re​ sults of your POST op​ er​ a​ tion in the re​ sponse. and aaaRe​ fresh as the Dn tar​ gets of a POST op​ er​ a​ tion. aaaL​ o​ gout. Their pay​ loads con​ tain a sim​ ple XML or JSON pay​ load con​ tain​ ing the MO rep​ re​ sen​ ta​ tion of an aaaUser ob​ ject with the at​ tribute name and pwd defin​ ing the user​ name and pass​ word: for ex​ am​ ple. for ex​ am​ ple. use​ ful for nar​ row​ ing the scope of your search to allow in​ for​ ma​ tion to be lo​ cated more quickly. The pay​ load of the POST op​ er​ a​ tion will con​ tain the XML or JSON en​ coded data rep​ re​ sent​ ing the man​ aged ob​ ject the de​ fines the Cisco API com​ mand body.and pass​ word-based au​ then​ ti​ ca​ tion uses a spe​ cial sub​ set of re​ quest URIs. Sub​ se​ quent op​ er​ a​ tions on the REST API can use this token value as a cookie named APIC-cookie to au​ then​ ti​ cate fu​ ture re​ quests. The fil​ ters them​ selves are ap​ pended as query URI op​ tions. The re​ sponse to the POST op​ er​ a​ tion will con​ tain an au​ then​ ti​ ca​ tion token as both a SetCookie header and an at​ tribute to the aaaLo​ gin ob​ ject in the re​ sponse named token. Mul​ ti​ ple con​ di​ tions can be joined to​ gether to form com​ plex fil​ ters. The cre​ ate or up​ date op​ er​ a​ tion should tar​ get a spe​ cific man​ aged ob​ ject. so the lit​ eral string /mo in​ di​ cates that the Dn of the man​ aged ob​ ject will be pro​ vided. Fil​ ter strings can be ap​ plied to POST op​ er​ a​ tions. .360 Scripting Cre​ ate and up​ date op​ er​ a​ tions use the same syn​ tax as read op​ er​ a​ tions. for which the XPath is /im​ data/aaaLo​ gin/@to​ ken if the en​ cod​ ing is XML. start​ ing with a ques​ tion mark (?) and con​ cate​ nated with an am​ per​ sand (&). ex​ cept that they al​ ways are tar​ geted at an ob​ ject level. Authentication REST API user​ name.

Scripting 361 The fol​ low​ ing query fil​ ters are avail​ able: .

.

the GET re​ quest is dis​ played. it can be ac​ cessed from the Ac​ count menu. The API In​ spec​ tor fur​ ther sim​ pli​ fies the process of ex​ am​ in​ ing what is tak​ ing place on the REST in​ ter​ face as the GUI is nav​ i​ gated by dis​ play​ ing in real time the URIs and pay​ loads. API Inspector . vis​ i​ ble at the top right of the Cisco APIC GUI. When a new con​ fig​ u​ ra​ tion is com​ mit​ ted. Click Wel​ come. URIs. the API In​ spec​ tor dis​ plays the re​ sult​ ing POST re​ quests. time stamps will ap​ pear along with the REST method. There may also be oc​ ca​ sional up​ dates in the list as the GUI re​ freshes sub​ scrip​ tions to data being shown on the screen. To get started with the API In​ spec​ tor. and pay​ loads.Scripting 363 API Inspector All op​ er​ a​ tions that are per​ formed in the GUI in​ voke REST calls to fetch and com​ mit the in​ for​ ma​ tion being ac​ cessed. and when in​ for​ ma​ tion is dis​ played on the GUI. <user​ name> and then choose the Show API In​ spec​ tor op​ tion After the API In​ spec​ tor is brought up.

"children": [ { "fvRsCtx": { "attributes": { "tnFvCtxName": "CiscoVrf".modified" }.​ 16. { "fvCtx": { "attributes": { "name": "CiscoVrf".​ json { "fvTenant": { "attributes": { "name": "Cisco". "name": "CiscoBd".​ 176/​ api/​ node/​ mo/​ uni/​ tn-Cisco. "status": "created" }. "children": [ { "fvBD": { "attributes": { "mac": "00:22:BD:F8:19:FF".​ 176. "status": "created" }. "children": [] } } ] } }. "status": "created" .364 Scripting From the out​ put above it can see that the last logged item has a POST re​ quest with the JSON pay​ load con​ tain​ ing a ten​ ant named Cisco and some at​ trib​ utes de​ fined on that ob​ ject: url: http://​ 172. "status": "created.

Scripting 365 }. "children": [] } } ] } } .

.

The stream​ lined in​ ter​ face pro​ vided makes it very quick to adopt and al​ lows users to begin to quickly de​ velop their ap​ pli​ ca​ tions. com​ pre​ hen​ sive data val​ id ​a​ tion. ap​ pli​ ca​ tion pro​ files. and ex​ ten​ sive sup​ port for query​ ing and fil​ ter​ ing. cou​ pling these with the power and flex​ ib ​il​ ity of the ACI pol​ icy lan​ guage and the pop​ ul​ ar Python pro​ gram​ ming lan​ guage to con​ fig​ ure ACI in a pro​ gram​ matic fash​ ion. Vi​ sore. and using com​ mon freely avail​ able tools. such as POST​ man. or by sav​ ing XML/JSON di​ rectly from the GUI.Scripting 367 Development Techniques ACI has a num​ ber of meth​ ods for de​ vel​ op​ ing code that can be used by en​ gi​ neers who have vary​ ing lev​ els of com​ fort with pro​ gram​ ming or in​ ter​ act​ ing with pro​ gram​ matic in​ ter​ faces. The most basic and straight-for​ ward tech​ nique in​ volves sim​ ply tak​ ing in​ for​ ma​ tion gleaned by the API in​ spec​ tor. Cobra en​ sures that the com​ plete ACI ex​ pe​ ri​ ence is avail​ able to de​ vel​ op​ ers and users alike. to send this in​ for​ ma​ tion back to the REST API. to en​ able users to rapidly cre​ ate ten​ ants. . A step up from this method en​ ables users to use com​ mon ter​ mi​ nol​ ogy and well un​ der​ stood net​ work​ ing con​ structs. EPGs and the as​ so​ ci​ ated con​ cepts to con​ nect those to phys​ ic ​al in​ fra​ struc​ ture. The most pow​ er​ ful of the de​ vel​ op​ ment tools avail​ able is the Cobra SDK. ACI Toolkit is a util​ ity de​ vel​ oped in open-source that ex​ poses the most com​ mon ACI build​ ing blocks. With a com​ plete rep​ re​ sen​ ta​ tion of the ACI ob​ ject model avail​ able.

.

For this ex​ am​ ple.Scripting 369 POSTman POST​ man is an open source ex​ ten​ sion for the Chrome web browser. . a small folder with a plus (+) sign will be​ come vis​ ib ​le. to both send and re​ ceive data which may rep​ re​ sent con​ fig​ ur​ a​ tion. and then use POST​ man to re​ play those op​ er​ at​ ions. which should then be clicked. Using the side​ bar. Ini​ tially the user will be pre​ sented with an in​ ter​ face that has two pri​ mary sec​ tions: the side​ bar on the left and the re​ quest con​ struc​ tor on the right. Once the plu​ gin is in​ stalled. the user can switch be​ tween the his​ tory of REST re​ quests sent by POST​ man. ac​ tions. Collections A use​ ful post to cre​ ate in a col​ lec​ tion is a basic Login op​ er​ at​ ion.​ getpostman. Installation To get started with POST​ man. after which the Cre​ ate but​ ton should be clicked. the col​ lec​ tion can be named “APIC”. it is very sim​ ple to uti​ lize the API In​ spec​ tor to view what the un​ der​ ly​ ing calls being made to the GUI are for cer​ tain op​ er​ at​ ions. In order to do this. at which point a popup will ap​ pear prompt​ ing the user to give a name to the col​ lec​ tion. which is avail​ able at www. as well as Col​ lec​ tions of re​ quests that con​ tain com​ mon tasks. cap​ ture those. For an in​ di​ vid​ ual un​ fa​ mil​ iar with the struc​ ture of REST. Fur​ ther​ more POST​ man al​ lows for the re​ quests to be mod​ if​ ied: GUI op​ er​ at​ ions can be made once. Within the side​ bar. pol​ icy and op​ er​ at​ ional state data. POST​ man can be used to in​ ter​ act with the APIC REST in​ ter​ face. which pro​ vides REST client func​ tion​ al​ ity in an easy-to-use pack​ age. the first step is to down​ load the plu​ gin for the Chrome web browser. the user should first click into the Col​ lec​ tions tab in the side​ bar.​ com. it can be ac​ cessed using the Chrome App launcher. at​ trib​ utes changed in the cap​ tured data and then sent back to the REST API to make the mod​ if​ i​ ca​ tions.

370 Scripting

Build Login request
Now a new re​
quest can be built. In the re​
quest con​
struc​
tor, where “Enter re​
quest URL
here” is shown, the fol​
low​
ing re​
quest URI should be en​
tered, sub​
sti​
tut​
ing APIC​
IP
​AD​
DRESS with the IP of the APIC: https://<APIC-IP>/api/mo/aaaLogin.​
xml
This re​
quest URI will call the Login method in the REST API. Since a Login will re​
quire
post​
ing data, the HTTP method should be changed, which can be done by click​
ing the
drop​
down list to the right of the re​
quest URL. By de​
fault it will be a GET re​
quest, but
POST will need to be se​
lected from the drop down list.
With the POST method se​
lected, it is now pos​
si​
ble to pro​
vide the REST pay​
load. Given
that the data will be sent via REST, the “raw” Re​
quest body se​
lec​
tor should be picked.
Now the pay​
load for the re​
quest can be en​
tered, which will be the sim​
ple XML con​
tain​
ing the user​
name and pass​
word that will be used for au​
then​
ti​
ca​
tion. Note that the URL
is https, mean​
ing that it will be en​
crypted be​
tween the web browser and the APIC, so
no data is being trans​
mit​
ted in clear text. The fol​
low​
ing re​
quest body should be en​
tered, sub​
sti​
tut​
ing the cor​
rect user​
name and pass​
word in place of USER​
NAME and
PASS​
WORD: <aaaUser name='USERNAME' pwd='PASSWORD'/>
With this re​
quest built, it is now pos​
si​
ble to Send the re​
quest, but since this will be a
com​
monly used method, the re​
quest should be added to a col​
lec​
tion. This can be ac​
com​
plished by click​
ing the “Add to col​
lec​
tion” but​
ton be​
neath the re​
quest body. Se​
lect
the “APIC” col​
lec​
tion from the ex​
ist​
ing col​
lec​
tion list, and change the Re​
quest name to
“Login” and then click “Add to col​
lec​
tion”.
By adding the re​
quest to a col​
lec​
tion it can later be quickly ac​
cessed to es​
tab​
lish a login
ses​
sion with APIC as needed.
After com​
plet​
ing the above steps, the re​
quest is ready to be sent. Click the “Send” but​
ton in the re​
quest con​
struc​
tor, and the REST API will re​
turn the XML rep​
re​
sent​
ing a
login ses​
sion with the APIC. The fol​
low​
ing will be vis​
ib
​le in the POST​
man GUI:

Scripting 371

Login request in POSTman

Make Query to APIC
The next re​
quest that will be built is one that queries the APIC for a list of ten​
ants on
the sys​
tem. First click the “Reset” but​
ton in the re​
quest con​
struc​
tor, and pro​
ceed with
the same steps as above, ex​
cept that the re​
quest URL will be shown as https://<APICIP>/api/class/fvTenant.​
xml, and the re​
quest method will be changed to GET.
Click “Add to col​
lec​
tion” and place the re​
quest into the APIC col​
lec​
tion, and for the
name enter “Query APIC for ten​
ants”
Now upon click​
ing “Send”, this re​
quest will re​
turn an XML en​
coded list of ten​
ants in the
re​
sponse body sec​
tion of the con​
struc​
tor pane on the right.

Make Configuration Change in APIC
Mak​
ing a con​
fig​
u​
ra​
tion change will use a POST re​
quest sim​
i​
lar to log​
ging in, how​
ever
the re​
quest URL and body will con​
tain a dif​
fer​
ent set of in​
for​
ma​
tion.

372 Scripting

For this ex​
am​
ple, a new ten​
ant will be cre​
ated in the fab​
ric. Click the “Reset” but​
ton in
the re​
quest con​
struc​
tor to clear out all ex​
ist​
ing re​
quest fields, and use URL
https://<APIC-IP>/api/mo/uni.​
xml and change the method to POST.
In the re​
quest pay​
load, enter the fol​
low​
ing data:

<fvTenant name="Cisco"/>

The re​
quest URL spec​
if​
ies that the tar​
get for this query will be the pol​
icy uni​
verse,
which is where ten​
ants live. With this tar​
get prop​
erly scoped, the data rep​
re​
sent​
ing the
ten​
ant can be pro​
vided in the pay​
load, in this case cre​
at​
ing a ten​
ant named Cisco.

Use API Inspector for Query Guidance
As dis​
cussed in the In​
tro​
duc​
tion to Script​
ing sec​
tion, API in​
spec​
tor can be used as a
guide​
line for build​
ing cus​
tom REST re​
quests. Fur​
ther​
ing on the ex​
am​
ple in that sec​
tion,
where the re​
quest URL is https://<APIC-IP>/api/node/mo/uni/tn-Cisco.​
json and
the pay​
load is the fol​
low​
ing com​
pacted ver​
sion of JSON:
{"fvTenant": {"attributes": {"name": "Cisco", "status": "created"}, "children":
[{"fvBD": {"attributes": {"mac": "00:22:BD:F8:19:FF", "name": "CiscoBd", "status":
"created"}, "children": [{"fvRsCtx": {"attributes": {"tnFvCtxName": "CiscoVrf",
"status": "created,modified"}, "children": [] } } ] } }, {"fvCtx": {"attributes":
{"name": "CiscoVrf", "status": "created"}, "children": [] } } ] } }

It is pos​
si​
ble to mod​
ify the re​
quest URI and pay​
load and sub​
sti​
tute the ten​
ant name
“Cisco” with an​
other ten​
ant name, to cre​
ate an en​
tirely new ten​
ant, with the same con​
fig​
ur​
a​
tion. The new re​
quest URL and JSON would be: https://<APICIP>/api/node/mo/uni/tn-Acme.​
json
{"fvTenant": {"attributes": {"name": "Acme", "status": "created"}, "children":
[{"fvBD": {"attributes": {"mac": "00:22:BD:F8:19:FF", "name": "AcmeBd", "status":
"created"}, "children": [{"fvRsCtx": {"attributes": {"tnFvCtxName": "AcmeVrf",
"status": "created,modified"}, "children": [] } } ] } }, {"fvCtx": {"attributes":
{"name": "AcmeVrf", "status": "created"}, "children": [] } } ] } }

Scripting 373

These val​
ues can be placed into a POST re​
quest in POST​
man, and after es​
tab​
lish​
ing a
Login ses​
sion using the saved Login re​
quest, the new ten​
ant “Acme” can be cre​
ated,
iden​
ti​
cal to the pre​
vi​
ously cre​
ated Cisco ten​
ant, with​
out need​
ing to man​
ua
​lly click
through the GUI or use other man​
ual meth​
ods.

Scripting 375

Cobra SDK and Arya
The com​
plete Cisco ACI Python SDK is named Cobra. It is a pure Python im​
ple​
men​
ta​
tion of the API that pro​
vides na​
tive bind​
ings for all the REST func​
tions and also has a
com​
plete copy of the ob​
ject model so that data in​
tegrity can be en​
sured, as well as sup​
port​
ing the com​
plete set of fea​
tures and func​
tions avail​
able in ACI. Cobra pro​
vides
meth​
ods for per​
form​
ing lookups and queries and ob​
ject cre​
ation, mod​
if​
i​
ca​
tion, and
dele​
tion that match the REST meth​
ods used by the GUI and those that can be found
using API In​
spec​
tor. As a re​
sult, pol​
icy cre​
ated in the GUI can be used as a pro​
gram​
ming tem​
plate for rapid de​
vel​
op​
ment.
The in​
stal​
la​
tion process for Cobra is straight​
for​
ward, and you can use stan​
dard Python
dis​
tri​
bu
​t​
ion util​
it​
ies. Cobra is dis​
trib​
uted on the APIC as an .egg file and can be in​
stalled
using easy_in​
stall, and is also avail​
able on github at http://​
github.​
com/​
datacenter/​
cobra.
The com​
plete doc​
um
​en​
ta​
tion for the Cobra SDK is avail​
able at http://​
cobra.​
readthedocs.​
org/​
en/​
latest/​

Establish Session
The first step in any code that uses Cobra is es​
tab​
lish​
ing a login ses​
sion. Cobra cur​
rently sup​
ports user​
name- and pass​
word-based au​
then​
ti​
ca​
tion, as well as cer​
tifi​
catebased au​
then​
ti​
ca​
tion. The ex​
am​
ple here uses user​
name- and pass​
word-based au​
then​
ti​
ca​
tion.
import cobra.mit.access
import cobra.mit.session
apicUri = 'https://10.0.0.2'
apicUser = 'username'
apicPassword = 'password'
ls = cobra.mit.session.LoginSession(apicUri, apicUser, apicPassword)
md = cobra.mit.access.MoDirectory(ls)
md.login()

376 Scripting

This ex​
am​
ple pro​
vides an MoDi​
rec​
tory ob​
ject named md, which is logged into and au​
then​
ti​
cated for Cisco APIC. If for some rea​
son au​
then​
ti​
ca​
tion fails, Cobra will dis​
play a
cobra.​
mit.​
request.​
CommitEr​
ror ex​
cep​
tion mes​
sage. With the ses​
sion logged in, you are
ready to pro​
ceed.

Work with Objects
Use of the Cobra SDK to ma​
nip​
ul​
ate the MIT gen​
er​
ally fol​
lows this work​
flow:
1

Identify the object to be manipulated.

2

Build a request to change attributes or add or remove children.

3

Commit the changes made to the object.

For ex​
am​
ple, if you want to cre​
ate a new ten​
ant, you must first iden​
tify where the ten​
ant will be placed in the MIT, where in this case it will be a child of the pol​
icy Uni​
verse man​
aged ob​
ject (pol​
Un
​iMo):
import cobra.model.pol
polUniMo = cobra.model.pol.Uni('')

With the pol​
Un
​iMo ob​
ject de​
fined, you can cre​
ate a ten​
ant ob​
ject as a child of pol​
U-​
niMo:
import cobra.model.fv
tenantMo = cobra.model.fv.Tenant(polUniMo, 'cisco')

All these op​
er​
at​
ions have re​
sulted only in the cre​
ation of Python ob​
jects. To apply the
con​
fig​
ur​
a​
tion, you must com​
mit it. You can do this using an ob​
ject called a Con​
fi​
gRe​
quest. Con​
fi​
gRe​
quest acts as a con​
tainer for MO-based classes that fall into a sin​
gle
con​
text, and they can all be com​
mit​
ted in a sin​
gle atomic POST op​
er​
at​
ion.
import cobra.mit.request
config = cobra.mit.request.ConfigRequest()
config.addMo(tenantMo)
md.commit(config)

Scripting 377

The Con​
fi​
gRe​
quest ob​
ject is cre​
ated, then the ten​
antMo ob​
ject is added to the re​
quest,
and then you com​
mit the con​
fig​
ur​
a​
tion through the MoDi​
rec​
tory ob​
ject.
For the pre​
ced​
ing ex​
am​
ple, the first step builds a local copy of the pol​
Uni ob​
ject. Be​
cause it does not have any nam​
ing prop​
er​
ties (re​
flected by the empty dou​
ble sin​
gle
quo​
ta​
tion marks), you don’t need to look it up in the MIT to fig​
ure out what the full Dn
for the ob​
ject is; it is al​
ways known as uni.
If you wanted to post some​
thing deeper in the MIT, where the ob​
ject has nam​
ing prop​
er​
ties, you would need to per​
form a lookup for that ob​
ject. For ex​
am​
ple, if you wanted
to post a con​
fig​
ur​
a​
tion to an ex​
ist​
ing ten​
ant, you could query for that ten​
ant and cre​
ate
ob​
jects be​
neath it.
tenantMo = md.lookupByClass('fvTenant', propFilter='eq(fvTenant.name, "cisco")')
tenantMo = tenantMo[0] if tenantMo else None

The re​
sult​
ing ten​
antMo ob​
ject will be of class cobra.​
model.​
fv.​
Tenant and will con​
tain
prop​
er​
ties such as .dn, .sta​
tus, and .name, all de​
scrib​
ing the ob​
ject it​
self. The lookup​
By​
Class() entry re​
turns an array, be​
cause it can re​
turn more than one ob​
ject. In this
case, the com​
mand is spe​
cific and is fil​
ter​
ing on an fv​
Tenant ob​
ject with a par​
tic​
ul​
ar
name. For a ten​
ant, the name at​
tribute is a spe​
cial type of at​
tribute called a nam​
ing at​
tribute. The nam​
ing at​
tribute is used to build the rel​
at​
ive name, which must be unique
in its local name​
space. As a re​
sult, you can be as​
sured that lookup​
By​
Class on an fv​
Tenant ob​
ject with a fil​
ter on the name al​
ways re​
turns ei​
ther an array of length 1 or
None, mean​
ing that noth​
ing was found.
To en​
tirely avoid a lookup, you can build a Dn ob​
ject and make an ob​
ject a child of that
Dn. This method works only in cases in which the par​
ent ob​
ject al​
ready ex​
ists.
topDn = cobra.mit.naming.Dn.fromString('uni/tn-cisco')
fvAp = cobra.model.fv.Ap(topMo, name='AppProfile')

These fun​
da​
men​
tal meth​
ods for in​
ter​
act​
ing with Cobra pro​
vide the build​
ing blocks
nec​
es​
sary to cre​
ate more com​
plex work​
flows that can help au​
to​
mate net​
work con​
fig​
u-​
ra​
tion, per​
form trou​
bleshoot​
ing, and man​
age the net​
work.

378 Scripting

Cisco APIC REST to Python Adapter
The process of build​
ing a re​
quest can be time con​
sum​
ing, be​
cause you must rep​
re​
sent
the ob​
ject data pay​
load as Python code re​
flect​
ing the ob​
ject changes that you want to
make. Be​
cause the Cobra SDK is di​
rectly mod​
eled on the Cisco ACI ob​
ject model, you
should be able to gen​
er​
ate code di​
rectly from what re​
sides in the ob​
ject model. As ex​
pected, you can do this using a tool de​
vel​
oped by Cisco Ad​
vanced Ser​
vices. The tool is
the Cisco APIC REST to Python Adapter, known as Arya.

Sample REST to Python Adapter
The above fig​
ure clearly shows how the input that might come from the API In​
spec​
tor,
Vi​
sore, or even the out​
put of a REST query and can then be quickly con​
verted into
Cobra SDK code, to​
k​
enized, and reused in more ad​
vanced ways.
In​
stal​
la​
tion of Arya is rel​
a​
tively sim​
ple, and the tool has few ex​
ter​
nal de​
pen​
den​
cies. To
in​
stall Arya, you must have Python 2.7.5 and git in​
stalled. Use the fol​
low​
ing quick in​
stal​
la​
tion steps to in​
stall it and place it in your sys​
tem Python.
git clone https://github.com/datacenter/ACI.git
cd ACI/arya
sudo python setup.py install

Scripting 379 After Arya has been in​ stalled.mit.mit.xml The entry will yield the fol​ low​ ing Python code: #!/usr/bin/env python ''' Autogenerated code using arya.py -f /home/palesiak/simpletenant.1'.mit.LoginSession('https://1.1.model. Some placeholders will ' + 'need to be changed') # list of packages that should be imported for this code to work import cobra.pol from cobra.py Original Object Document Input: <fvTenant name='bob'/> ''' raise RuntimeError('Please review the auto generated code before ' + 'executing the output.request import cobra. you can take XML or JSON rep​ re​ sent​ ing Cisco ACI mod​ eled ob​ jects and con​ vert it to Python code quickly.model.MoDirectory(ls) md.access.mit.xmlcodec import toXMLStr # log into an APIC and create a directory object ls = cobra. 'admin'.Tenant(topMo.fv.addMo(topMo) md.mit.login() # the top level object on which operations will be made topMo = cobra.ConfigRequest() c.model. For ex​ am​ ple.Uni('') # build the request using cobra syntax fvTenant = cobra.1.session.mit.model.codec.access import cobra.pol.fv import cobra. enter: arya. 'password') md = cobra. name='bob') # commit the generated code to APIC print toXMLStr(topMo) c = cobra.session import cobra.internal.request.commit(c) .

the Cisco APIC IP ad​ dress. The same ap​ plies to the cre​ den​ tials and any other place​ hold​ ers.​ 1. should be up​ dated to re​ flect the ac​ tual Cisco APIC IP ad​ dress. Note that if you pro​ vide input XML or JSON that does not have a fully qual​ if​ ied hi​ er​ ar​ chy. Arya may not be able to iden​ tify it through heuris​ tics. In this case. it is pur​ posely put in place to help en​ sure that any other to​ ke ​nized val​ ues that must be up​ dated are cor​ rected. which de​ faults to 1. For ex​ am​ ple.​ 1.380 Scripting The place​ holder rais​ ing a run​ time error must first be re​ moved be​ fore this code can be ex​ e​ cuted. . You can find this Dn by query​ ing for the ob​ ject in Vi​ sore or in​ spect​ ing the re​ quest URI for the ob​ ject shown in the API In​ spec​ tor.​ 1. which you will need to re​ place with the cor​ rect Dn. a place​ holder will be pop​ ul​ ated with the text RE​ PLACEME.

fire​ walls.py—This script creates the subscription to the endpoint class and populates the MySQL database • aci-endpoint-tracker-gui. so we will as​ sume you have ac​ cess to cre​ ate a new data​ base on a MySQL server. which may be daunt​ ing for a user being first in​ tro​ duced to net​ work pro​ gram​ ma​ bil​ ity. The com​ plete doc​ um ​en​ ta​ tion for ac​ it​ oolkit is avail​ able at http://​ datacenter. A sample is shown below: . In ad​ di​ tion. load bal​ ancers.​ github.Scripting 381 ACI Toolkit The com​ plete ACI ob​ ject model con​ tains many en​ ti​ ties. • aci-endpoint-tracker. and other de​ vices). In​ stalling MySQL is out​ side the scope of this book. a num​ ber of ap​ pli​ ca​ tions have been built on top of ACI toolkit. The end​ point tracker ap​ pli​ ca​ tion has two pri​ mary com​ po​ nents that are both python scripts.​ io/​ acitoolkit/​ ACI Toolkit Applications Endpoint Tracker The end​ point tracker ap​ pli​ ca​ tion cre​ ates a sub​ scrip​ tion to the end​ point class (fvCEp) and pop​ ul​ ates a MySQL data​ base with per​ ti​ nent de​ tails about each end​ point pre​ sent on the fab​ ric (for ex​ am​ ple servers. and give users a way to quickly bring up com​ mon tasks and work​ flows.py—This script creates a web interface that provides a way to present the contents of the database to the operator. The ac​ it​ oolkit makes avail​ able a sim​ pli​ fied sub​ set of the model that can act as an in​ tro​ duc​ tion to the con​ cepts in ACI.

the IP ad​ dress 192. The second script enables the content to be viewed in an understandable web UI.1 MySQL login username: root MySQL Password: user@linuxhost::~/acitoolkit/applications/endpointtracker$ python aci-endpointtracker-gui.382 Scripting To launch Endpoint Tracker run the following python scripts. and the match​ ing re​ sults are dis​ played.py.0.​ 168. and the table is fil​ tered ac​ cord​ ingly.0. will actually connect to the APIC and populate the database. Using the ACI End​ point Tracker is sim​ ply a mat​ ter of in​ putting an IP or MAC ad​ dress into the search field.0. aciendpoint-tracker. user@linuxhost:~/acitoolkit/applications/endpointtracker$ .0. The first script.1 MySQL login username: root MySQL Password: * Running on http://127.1:5000/ * Restarting with reloader After run​ ning those python scripts you can now bring up a browser and go the Web UI.​ 5.0.py MySQL IP address: 127.0.py MySQL IP address: 127. In the ex​ am​ ple below. ./aci-endpoint-tracker.​ 20 has been input into the search field.

Pie chart view of endpoint distribution . Some sam​ ple screen​ shots are shown below. and EPGs.Scripting 383 One more interesting usage of the endpoint tracker applications is a series of visualizations that can represent how various endpoints are mapped to other fabric constructs including Tenants. These are rep​ re​ sen​ ta​ tions of where end points are within the ACI fab​ ric and how they re​ late to or de​ pend on other ob​ jects in the en​ vi​ ron​ ment. Applications.

384 Scripting Tree view of endpoint relationships Force diagram of endpoint location ACI Lint .

. Warning 007: Contract 'External' in Tenant 'Acme' is not consumed at all. Warning 007: Contract 'default' in Tenant 'common' is not consumed at all. A sam​ ple out​ put is pro​ vided here for ref​ er​ ence: user@linuxhost:~/acitoolkit/applications/lint$ ./acilint.py Getting configuration from APIC.Scripting 385 ACI Lint In com​ puter pro​ gram​ ming. Warning 001: Tenant 'Books' has no Application Profile. . Warning 005: BridgeDomain 'inb' in Tenant 'mgmt' has no EPGs. Warning 007: Contract 'outside-to-web' in Tenant 'roberbur' is not consumed at all.. Warning 005: BridgeDomain 'CiscoBd' in Tenant 'Cisco' has no EPGs.checks for com​ mon con​ fig​ ur​ a​ tion er​ rors and re​ ports them to the user.sup​ ports the abil​ ity to tag EPGs as ei​ ther se​ cure or in​ se​ cure.. Warning 004: Context 'oob' in Tenant 'mgmt' has no BridgeDomains. Warning 001: Tenant 'mgmt' has no Application Profile. Warning 007: Contract 'WebServers' in Tenant 'Acme' is not consumed at all. Processing configuration. Warning 002: Tenant 'Books' has no Context. Lint is a term that refers to iden​ ti​ fy​ ing dis​ crep​ an​ cies.... Con​ fig​ u​ ra​ tion Is​ sues . Warning 001: Tenant '3tierapp' has no Application Profile. or sim​ ple debug tool for com​ mon er​ rors. Warning 006: Contract 'default' in Tenant 'common' is not provided at all. In the sense that ACI pro​ vides in​ fra​ struc​ ture as code. Warning 006: Contract 'WebServers' in Tenant 'Acme' is not provided at all. Warning 006: Contract 'External' in Tenant 'Acme' is not provided at all. ACI Lint is an ap​ pli​ ca​ tion that checks and no​ ti​ fies an op​ er​ at​ or of mis​ con​ fig​ ur​ a​ tion er​ rors in two pri​ mary ca​ pac​ it​ ies: Se​ cu​ rity Is​ sues . ACI Toolkit pro​ vides just that. Warning 002: Tenant '3tierapp' has no Context. Critical 001: EPG 'default' in tenant 'infra' app 'access' is not assigned security clearance Critical 001: EPG 'x' in tenant 'common' app 'default' is not assigned security clearance Warning 001: Tenant 'Cisco' has no Application Profile. it is ap​ pro​ pri​ ate for ACI to also have a Lint ap​ pli​ ca​ tion. and then runs a val​ id ​a​ tion that con​ tracts are not used to cross se​ cu​ rity bound​ aries.

386 Scripting While the ACI Toolkit pro​ vides some use​ ful tools for an op​ er​ at​ or to im​ me​ di​ ately use. Give it a try! Be sure to share your work back with the com​ mu​ nity! . the real value is in the abil​ ity to take these ex​ am​ ples as a start​ ing point. and mod​ ify or ex​ tend these sam​ ples to suit your par​ tic​ ul​ ar needs.

or al​ ter​ nately move for​ ward to a newer ver​ sion. so it’s not un​ com​ mon for users to store doc​ um ​en​ ta​ tion or other con​ stantly chang​ ing files in git. What is stored on GitHub is usu​ ally source code. Pre​ vi​ ous tools like Con​ cur​ rent Ver​ sion Con​ trol (CVS). While these tools have and con​ tinue to work well. that allow for in​ di​ vid​ ua ​ls to col​ lab​ o​ rate with over eight-mil​ lion other GitHub users on pro​ jects to​ gether. with the fore​ most being Git. web servers. the au​ thor of the pop​ ul​ ar open-source op​ er​ at​ ing sys​ tem Linux. in​ clud​ ing con​ sumer soft​ ware. Aside from being a wrap​ per around git. GitHub GitHub is a host​ ing plat​ form based around git. there has been a slow mi​ gra​ tion away from those server-based tools to de​ cen​ tral​ ized util​ it​ ies. and has been the mo​ ti​ va​ tion be​ hind many suc​ cess​ ful pro​ jects. not lim​ ited to any spe​ cific lan​ guage. how​ ever the git pro​ to​ col it​ self sup​ ports stor​ age and ver​ sion con​ trol of any file type. Git was cre​ ated by Linus Tor​ valds. and more ef​ fi​ cient sup​ port for branches. The com​ bi​ na​ tion of all of these fea​ tures has made GitHub a very com​ mon place for mem​ bers of the com​ mu​ nity to share code with one an​ other.Scripting 387 GitHub Source Control Open source soft​ ware has been a pop​ ul​ ar move​ ment in IT. build on each other's work. GitHub also pro​ vides tech​ niques for track​ ing is​ sues. se​ cur​ ing ac​ cess to pro​ jects. with a cen​ tral server main​ tain​ ing a com​ mon data​ base of source code. . and con​ tribute their ef​ forts back into larger pro​ jects. Git has a num​ ber of ad​ van​ tages over most other source con​ trol tools: com​ plete local repos​ it​ ory copies. Git also main​ tains an audit of changes that have been made to files and even has ad​ vanced sup​ port for branch​ ing ver​ sions of files to allow mul​ ti​ ple con​ cur​ rent mod​ if​ i​ ca​ tions to a file to take place. dis​ trib​ uted ar​ chi​ tec​ ture. One of the key as​ pects to the suc​ cess of open source is the abil​ ity for many de​ vel​ op​ ers around the globe to col​ lab​ or​ ate to​ gether on a sin​ gle pro​ ject. which pro​ vides both free and paid host​ ing ser​ vices. and Sub​ ver​ sion (SVN) were used to allow many de​ vel​ op​ ers to work to​ gether. and allow for them to be merged after work ef​ forts have com​ pleted. The pri​ mary ad​ van​ tage is that the ver​ sion con​ trol pro​ vided by git al​ lows a user to re​ vert a file back to any pre​ vi​ ously stored ver​ sion. and built-in pro​ ject doc​ um ​en​ ta​ tion. data​ bases and even en​ tire op​ er​ at​ ing sys​ tems.

how​ ever one of the most highly en​ cour​ aged be​ hav​ iors on GitHub is to pro​ vide clear and ob​ vi​ ous doc​ um ​en​ ta​ tion for a pro​ ject. and down​ load a graph​ ic ​al-based client to pro​ vide a sim​ pler in​ ter​ face to the com​ mand line-based git tool. . mod​ ify and con​ tribute to the pro​ ject. For users look​ ing to con​ tribute back to a pro​ ject.388 Scripting "It's on github" A com​ mon phrase in mod​ ern IT jar​ gon is.​ github. so if a new user ac​ cesses the front page of a pro​ ject on Git. and the sim​ plest way to begin to take ad​ van​ tage of the in​ for​ ma​ tion stored on GitHub is to sim​ ply ac​ cess a pro​ jects main page and look for the “Down​ load ZIP” but​ ton at the bot​ tom right of any pro​ ject's main page. GitHub it​ self has a graph​ ic ​al client with the Win​ dows ver​ sion avail​ able at http://​ windows. Other com​ mon source con​ trol tools in​ clude Source​ Tree from At​ lass​ ian. “It’s on github”. Once a user has an ac​ count and a github client. how​ ever for those who have not had an in​ tro​ duc​ tion it can seem like a com​ plex topic. If those changes work. they can “Fork”.​ com and the Mac ver​ sions at http://​ mac. What a user does with these files will greatly de​ pend on what the con​ tents are. the next step would be to sign up for an ac​ count on GitHub.​ com. this is an in​ vi​ ta​ tion to down​ load. and for users fa​ mil​ iar with GitHub. The process can be that sim​ ple. avail​ able at http://​ sourcetreeapp. The re​ sult​ ing down​ loaded file will con​ tain the lat​ est ver​ sion of the files in the pro​ ject. GitHub is ac​ tu​ ally a very sim​ ple tool to use.​ com.​ github. or split off a pro​ ject that is avail​ able into their own pri​ vate repos​ it​ ory. though many more ad​ vanced pro​ jects have stan​ dards and rules for con​ tribut​ ing to those pro​ jects that put in place re​ quire​ ments around how work is com​ mit​ ted back into the pro​ jects. which may re​ quire some read​ ing be​ fore at​ tempt​ ing to con​ tribute. it is pos​ si​ ble to sub​ mit a “Pull” re​ quest. they will typ​ ic ​ally be able to find in​ struc​ tions on how to down​ load and in​ stall the pro​ ject. which es​ sen​ tially means that the user is propos​ ing their ef​ forts should be pulled back into the orig​ in ​al pro​ ject. make changes and com​ mit those back to their pri​ vate branch. right on the first page they see. and the user wishes to con​ tribute them back into the orig​ in ​al pro​ ject.

389 Hardware Expansion and Replacement .

.

Hardware Expansion and Replacement 391 Section Content • Expanding and Shrinking the Fabric ​Switches Add Connected Switch Pre-Provision Switch Before Connection Decommission Existing Switch APICs Add New APIC Decommission Existing APIC • Hardware Diagnostics and Replacement ​Identify Hardware Failure Resolve Leaf Hardware Failure Resolve APIC Hardware Failure Diagnose Equipment Failures .

.

Both meth​ ods have the same out​ come: an ex​ panded fab​ ric in the mat​ ter of min​ utes. Ad​ di​ tion​ ally. This sec​ tion will walk through the op​ er​ at​ ions of adding and re​ mov​ ing switches and APICs in your ex​ ist​ ing ACI fab​ ric. Gen​ er​ ally. Switches There are two ways switches can be added to ACME's ex​ ist​ ing fab​ ric: by dis​ cov​ er​ ing the switches au​ to​ mat​ ic ​ally in the APIC after they have been ca​ bled to the fab​ ric. This is done the same way for both spine and leaf switches. there may be times when you need to re​ place failed hard​ ware. a best-practice ACI fabric is connected in a full mesh topology with every leaf cabled to every spine. and leaf switches are added for more ac​ cess ports. Add Connected Switch To add a switch that has al​ ready been at​ tached to the fab​ ric go through the fol​ low​ ing steps in the APIC GUI: 1 In the case of a leaf switch. cable it to all of the spine switches. which is dis​ cussed in the Hard​ ware Re​ place​ ments chap​ ter. . All devices should connect to the leaf switches. Ideally. which means adding new leaf and spine switches. APICs are added as the num​ ber of poli​ cies and end​ points in​ crease.Hardware Expansion and Replacement 393 Expanding and Shrinking the Fabric ACME may de​ cide to ex​ pand their ACI fab​ ric as their data cen​ ter grows. and pos​ si​ bly APICs. spine switches are added for more through​ put. and spines should never connect to other spines. In the case of a spine switch. leaves should never connect to other leaves. cable it to all the leaf switches. or by prepro​ vi​ sion​ ing the switches by adding their se​ ri​ al num​ bers and later con​ nect​ ing them phys​ ic ​ally to the fab​ ric when the switches ar​ rive. 2 On the APIC click on Fabric at the top of the screen. This sec​ tion will also cover de​ com​ mis​ sion​ ing switches. some switches or APICs may need to be de​ com​ mis​ sioned. Also. 3 Click on Fabric Membership in the left navigation pane. Adding APICs will also be cov​ ered.

2 In the Navigation pane. Pre-Provision Switch Before Connection Pre-pro​ vi​ sion​ ing a switch is a handy op​ er​ at​ ionally proac​ tive step to get the switch reg​ is​ tered be​ fore it even ar​ rives to your data cen​ ter. Lower numbers are reserved for APICs. choose Fabric. Note: Repeat this process for all switches you wish to pre-provision. The fol​ low​ ing steps walk you through switch pre-pro​ vi​ sion​ ing for both leaves and spines: 1 On the menu bar. Double click the switch and assign a Node ID and a Node Name. To be proac​ tive.394 Hardware Expansion and Replacement 4 When the new switch appears. For more in​ for​ ma​ tion on pre-pro​ vi​ sion​ ing poli​ cies. In the pop-up window. 5 Optionally. Fab​ ric poli​ cies are cov​ ered in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter. perform the following actions: a. number leaf nodes starting with 101. you can also pre-pro​ vi​ sion fab​ ric poli​ cies. 5 Click Submit. refer to the fol​ low​ ing white paper: . 7 Repeat this process for all new switches connected to the fabric. 3 In the Work pane. choose Actions > Create Add Fabric Node Member. and spine nodes with 201. The new entry in the Fab​ ric Mem​ ber​ ship win​ dow will show Un​ sup​ ported in the Role col​ umn until the switch is ac​ tu​ ally con​ nected to the fab​ ric. choose Fabric Membership. As a best practice. b. but the switch will im​ me​ di​ ately be​ come a mem​ ber of the fab​ ric once it ar​ rives and is ca​ bled. This is commonly used to identify the physical location of the switch in the data center. 6 Click Submit. Lower numbers are reserved for APICs. As a best practice. enter the serial number of the switch that will be arriving. Assign a Node ID and a Switch Name. add a Rack Name name. You will need to know the se​ ri​ al num​ ber of the switch you will re​ ceive to pre-pro​ vi​ sion. you'll see a node with a serial number but no Node ID or Node Name configured. and spine nodes with 201. 4 In the Create Add Fabric Node Member dialog box. number leaf nodes starting with 101.

as the switch will not for​ ward traf​ fic after de​ com​ mis​ sion​ ing. The Re​ move from Con​ troller op​ tion will com​ pletely re​ move the switch from the ACI fab​ ric and all APICs. Per​ form the fol​ low​ ing steps to ver​ ify clus​ ter health: . and Re​ move from Con​ troller. choose Inventory > Pod 1. APICs Add New APIC Be​ fore mak​ ing any changes to an APIC clus​ ter. En​ sure you do not have any de​ vices con​ nected. Click the General tab. The switch will no longer show up in the fab​ ric mem​ ber​ ship as a reg​ is​ tered node and the in​ fra​ struc​ ture VTEP IP ad​ dresses it was as​ signed will be re​ moved. 3 Click the switch to decommission in the Navigation pane. 4 Click Submit. 2 In the Navigation pane. Per​ form the fol​ low​ ing steps from the APIC GUI to de​ com​ mis​ sion a switch from the ACI fab​ ric: 1 On the menu bar. There are two types of switch de​ com​ mis​ sion​ ing: Reg​ ul​ ar. c. while keep​ ing the switch's node ID and fab​ ric mem​ ber​ ship. choose either Regular or Remove from Controller. The switch will show up under the Dis​ abled In​ ter​ faces and De​ com​ mis​ sioned Switches folder in the left nav​ i​ ga​ tion pane. a.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c11-731960.Hardware Expansion and Replacement 395 http://​ www. choose Fabric. en​ sure each APIC in the clus​ ter is fully fit and change the clus​ ter size to re​ flect the new con​ troller you are adding to the clus​ ter.​ html#_​ Toc405844675 Decommission Existing Switch De​ com​ mis​ sion​ ing a switch can ei​ ther re​ move it from the fab​ ric en​ tirely. Chose the Actions > Decommission. or re​ move a switch tem​ porar​ ily to per​ form main​ te​ nance. Reg​ u​ lar de​ com​ mis​ sion​ ing can be used for main​ te​ nance and es​ sen​ tially si​ lences the switch from re​ port​ ing faults and send​ ing SNMP in​ for​ ma​ tion as a tem​ po​ rary so​ lu​ tion. In the pop-up. b.

4 Click Submit. 2 In the Navigation pane. choose Actions > Change Cluster Size.​ html Per​ form the fol​ low​ ing steps to change the APIC clus​ ter size: 1 On the menu bar. Change the Target Cluster Administrative Size to reflect the new APIC(s) being added.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ application-policy-infrastructure-controller-apic/​ products-installationguides-list. .396 Hardware Expansion and Replacement 1 2 On the menu bar.​ cisco.​ html 1 On the menu bar. Click the Cluster folder. and the health state of each controller is Fully Fit from the Cluster folder under the new controller. If any of the APICs are not fully fit. Per​ form the fol​ low​ ing steps to add a new APIC to the clus​ ter: 1 Install and stage the APIC by connecting it to the fabric by following the hardware installation guide: http://​ www. c. Verify every controller shows Fully Fit under the Heath State column. The APIC controllers are added one by one and displayed in the sequential order starting with N + 1 and continuing until the target cluster size is achieved. choose Controllers > APIC_Name > Cluster. Expand the first APIC in the folder. In the Navigation pane.​ cisco. refer to the fol​ low​ ing trou​ bleshoot​ ing guide: http://​ www. a. choose Controllers > APIC_Name > Cluster. choose System Controllers. a. Verify that the APIC controllers are in operational state. choose System > Controllers. a. choose System > Controllers.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ troubleshooting/​ b_​ APIC_​ Troubleshooting. b. 3 In the Work pane. 2 In the Navigation pane. b. choose Controllers. Note: A cluster size of two is not permitted as that does not allow for quorum amongst APICs.

choose System > Controllers.Hardware Expansion and Replacement 397 Note: It will take sev​ eral min​ utes for the APICs to syn​ chro​ nize and join the new APIC to the clus​ ter. APIC5 must be de​ com​ mis​ sioned be​ fore APIC4. en​ sure each APIC in the clus​ ter is fully fit with the ex​ cep​ tion of the faulty APIC being de​ com​ mis​ sioned. 6 In the Work pane. 2 In the Navigation pane. For ex​ am​ ple. Per​ form the fol​ low​ ing steps to de​ com​ mis​ sion an APIC that needs to be re​ moved from the fab​ ric: 1 On the menu bar. choose Actions > Actions > Decommission. Decommission Existing APIC When de​ com​ mis​ sion​ ing APICs. choose Actions > Change Cluster Size. a. Again. a. Note: In the main pane. Change the Target Cluster Administrative Size to reflect the new APIC(s) being added. Fab​ ric op​ er​ at​ ion will con​ tinue nor​ mally. 4 Click Submit. Note: A cluster size of two is not permitted as that does not allow for quorum ​ amongst APICs. Note: Select an APIC that is NOT being decommissioned. choose Controllers > APIC_Name > Cluster. choose Controllers > APIC_Name > Cluster. 3 In the Work pane. Verify the APIC no longer appears in the Cluster folder under any of the remaining APICs. they must be de​ com​ mis​ sioned se​ quen​ tially in re​ verse order. 5 In the Navigation pane. . click the APIC to be decommissioned. You can​ not de​ com​ mis​ sion a pow​ ered on fully fit APIC. be​ fore mak​ ing any changes to an APIC clus​ ter.

.

Hardware Expansion and Replacement 399 Hardware Diagnostics and Replacement The Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric em​ ploys a com​ bi​ na​ tion of key soft​ ware and hard​ ware fea​ tures that are specif​ ic ​ally de​ signed to re​ duce the mean time be​ tween fail​ ures (MTBF) and the mean time to re​ pair (MTTR). or a com​ bi​ na​ tion of the two that ne​ ces​ si​ tates a leaf re​ place​ ment. the state​ less na​ ture of the ACI fab​ ric pro​ vides sig​ nif​ ic ​ant ad​ van​ tages to ad​ min​ is​ tra​ tors from an op​ er​ at​ ions stand​ point. Re​ gard​ ing hard​ ware. the hot-swap​ pable com​ po​ nents en​ able them to re​ place failed hard​ ware quickly and non-dis​ rup​ tively. For cases where there is a com​ po​ nent level . If ACME ever ex​ pe​ ri​ ences some sort of power surge or sees a com​ po​ nent of their switches go bad. faults are raised in the sys​ tem dash​ board and are pre​ sented to the ad​ min​ is​ tra​ tor. Ex​ am​ ples of hot-swap​ pable com​ po​ nents on both the leaves and spines in​ clude: • Power supplies • Fan trays Ex​ am​ ples of hot-swap​ pable com​ po​ nents on the spines in​ clude: • Power supplies • Fan trays • Supervisors • System Controller cards • Linecard modules De​ spite sig​ nif​ ic ​ant ad​ vances in the above com​ po​ nents that re​ duce the MTBF. there are sev​ eral hot-swap​ pable com​ po​ nents on both the leaf and spine switches in ad​ di​ tion to a few com​ po​ nents that are fixed on the chas​ sis. Identify Hardware Failure When a hard​ ware fail​ ure oc​ curs in the fab​ ric. In such an event. there is al​ ways the pos​ si​ bil​ ity of a fail​ ure on a leaf switch ei​ ther in switch hard​ ware or soft​ ware.

Resolve Leaf Hardware Failure As an ex​ am​ ple of a leaf fail​ ure. a Nexus 9396 leaf switch that is a part of the fab​ ric is un​ reach​ able. while the leaf and spine switches re​ port SNMP and Sys​ log mes​ sages for com​ po​ nent level fail​ ures. For ex​ am​ ple a power sup​ ply fail​ ure on the APIC will not gen​ er​ ate an SNMP or sys​ log mes​ sage and must be mon​ it​ ored and re​ me​ di​ ated using the APIC dash​ board. The first step in re​ plac​ ing the failed switch . You can use the GUI to de​ ter​ mine the node health to con​ firm that the leaf has failed. choose Pod 1. there is a pos​ si​ bil​ ity that the hard​ ware has failed. After con​ firm​ ing that the leaf node has failed. If re​ sponses are not re​ ceived from the switch in a cer​ tain time​ frame. the APICs them​ selves do not have the abil​ ity to gen​ er​ ate alerts using SNMP or sys​ log. 2 In the Navigation pane. Log​ ging mes​ sages can be sent to sys​ log servers. to pro​ vide alert​ ing. choose Fabric > Inventory. such as SNMP polling at a set in​ ter​ val. op​ er​ at​ ions teams can lever​ age their ex​ ist​ ing NMS tools. To view the node health score: 1 On the menu bar. you want to re​ move the failed switch and pro​ vi​ sion a new switch as part of the fab​ ric. such as Splunk. per​ haps due to a hard​ ware fail​ ure on the up​ link mod​ ules. The leaf and spine switches in the ACI fab​ ric also sup​ port tra​ di​ tional meth​ ods of de​ tect​ ing fail​ ures. Note: The pod health displays in the Work pane and is zero.400 Hardware Expansion and Replacement fail​ ure with re​ dun​ dant com​ po​ nents pre​ sent in the sys​ tem. Ex​ am​ ples of hard​ ware events that gen​ er​ ate sys​ log mes​ sages and SNMP traps in​ clude: • Linecard failure on a spine switch • Supervisor failure on a spine switch • System controller failure on a spine switch • Power supply or fan failures on a leaf or a spine switch While Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) is a cen​ tral point of man​ age​ ment for the en​ tire fab​ ric. sys​ log mes​ sages and SNMP traps are gen​ er​ ated. or SNMP mes​ sages can be sent to NMS sys​ tems. such as ZenOSS. How​ ever.

. choose Fabric > Inventory. as shown in the fol​ low​ ing ex​ am​ ple: {{protocol}}://{{apic}}/api/class/topSystem. To de​ com​ mis​ sion and recom​ mis​ sion a switch: 1 On the menu bar. 6 Bring up links one by one and verify if data traffic is flowing correctly. which is the ref​ er​ ence ob​ ject that al​ lows a re​ place​ ment switch with a new se​ ri​ al num​ ber to in​ herit the same state​ less con​ fig​ ur​ a​ tion that was as​ signed to the old node. To view the fab​ ric node IDs using the GUI: 1 On the menu bar. Re​ plac​ ing the node is as sim​ ple as de​ com​ mis​ sion​ ing the switch and recom​ mis​ sion​ ing it. You can also use a sin​ gle REST API call to pe​ ri​ od​ ic ​ally poll for a full list of nodes that are at or below a cer​ tain health level. 2 In the Navigation pane. NTP. choose Fabric Membership. 4 Stage the device with the right configuration file and eliminate any errors. choose Fabric > Inventory. you can take ad​ van​ tage of the state​ less na​ ture of the hard​ ware to in​ stan​ ti​ ate the log​ ic ​al con​ fig​ ur​ a​ tion pro​ files. the fol​ low​ ing high-level pro​ ce​ dure re​ places the switch: 1 Stand up the replacement switch. 3 Attempt to obtain the latest version of configurations from a configuration repository server. update the AAA.Hardware Expansion and Replacement 401 is to get the failed switch's unique ID (node ID). 3 Right click the failed node and choose Decommission. and syslog servers and the ACLs that are associated with each of them. expand Pod 1. In an ACI fab​ ric. 2 Load the correct version of code. For example. Each node is as​ signed an ID in the fab​ ric.xml?rsp-subtree-include=health&rspsubtree-filter=le(healthInst. 2 In the Navigation pane. 5 Copy the old configuration over to the switch."0") In the case of a tra​ di​ tional op​ er​ at​ ions model where each switch was man​ aged as an in​ de​ pen​ dent en​ tity.cur.

When the new leaf switch is com​ mis​ sioned suc​ cess​ fully. 3 Find the new switch and record its name and node ID. you can see the target firmware version. When prompted for the name and node ID. choose Fabric Node Firmware > Firmware Groups > All.402 Hardware Expansion and Replacement 4 Replace the failed leaf switch with the new leaf switch. To view which ver​ sion of the firmware that the APIC will load: 1 On the menu bar. choose Admin > Firmware. You can get the name and ID by view​ ing the un​ reach​ able nodes. 10 When prompted for the node ID. Note: In the Work pane. 8 In the Work pane. which is automatically set to the latest firmware version. enter the information that you recorded in this procedure. To view the un​ reach​ able nodes: 1 On the menu bar. 7 The new leaf appears with a node ID of 0 and an IP address of 0. In most cases. . 2 In the Navigation pane. 5 On the menu bar. 6 In the Navigation pane. 2 In the Navigation pane. 11 Click Update. If the new switch is not op​ er​ at​ ional. choose Unreachable Nodes. the new switch's name and node ID are dif​ fer​ ent from the name and ID that you en​ tered. you can also reuse the same leaf name. the APIC au​ to​ mat​ ic ​ally loads the cor​ rect ver​ sion of the firmware into the leaf.0. 4 Repeat the "To decommission and recommission a switch" procedure. click on the new leaf. starting with step 5. choose Fabric Membership. 9 Choose Actions > Commission Switch.0.0. enter the old node's ID. choose Fabric > Inventory. choose Fabric > Inventory.

switch(config)# copy running-config startup-config . by lever​ ag​ ing the state​ less ob​ ject mod​ el​ ing that re​ places the tra​ di​ tional run​ ning con​ fig​ u​ ra​ tion on a de​ vice. you might need to copy the ACI switch soft​ ware image to the switch in ques​ tion. ensure that images are copied to the standby supervisor in case of a full chassis replacement by using the command: # copy bootflash:aci_image bootflash://sup-standby/ 6 Configure the switch not to boot from Cisco NX-OS. and EPGs. bridge do​ mains. sys​ log. 2 Set the IP address on the mgmt0 interface to allow connectivity between the switch and the APIC. switch(config)# no boot nxos 7 Save the configuration. such as AAA. ACLs. SNMP.Hardware Expansion and Replacement 403 In ad​ di​ tion. To copy the ACI switch soft​ ware image to the switch: 1 Connect to the switch console. NTP. In the event that the re​ place​ ment switch runs stand​ alone NX-OS soft​ ware in​ stead of ACI switch soft​ ware. APIC au​ to​ mat​ i​ cally loads the cor​ rect run​ ning con​ fig​ u​ ra​ tion onto the de​ vice. 3 Enable SCP services: # feature scp-server 4 Copy the firmware image from APIC to the switch: # scp -r /firmware/fwrepos/fwrepo/switch_image_name admin@switch_ip_address:switch_image_name 5 For dual supervisor systems.

contact TAC to generate and install the correct certificates for your device. the switch should ap​ pear as an un​ man​ aged fab​ ric node when con​ nected to the fab​ ric. 1 On the menu bar choose System > Controllers. Resolve APIC Hardware Failure In this ex​ am​ ple. you must iden​ tify and re​ me​ di​ ate a hard​ ware fail​ ure on one of the APICs in your APIC clus​ ter.404 Hardware Expansion and Replacement 8 Boot the active and standby supervisor modules with the ACI image. switch(config)# reload 11 Log in to the switch as an administrator. switch(config)# boot aci bootflash:aci-image-name 9 Verify the integrity of the file by displaying the MD5 checksum. From the GUI of an op​ er​ at​ ional APIC. switch(config)# show file bootflash:aci-image-name md5sum 10 Reload the switch. If something else is listed. . admin@apic1:aci> openssl asn1parse /securedata/ssl/server. Login: admin 12 Verify whether you must install certificates for your device.crt 13 Look for PRINTABLESTRING in the command output. Once you have con​ firmed the that cer​ tifi​ cate is in​ stalled and the switch is in ACI mode. the correct certificates are installed. If "Cisco Manufacturing CA" is listed.

choose Controllers > apic_name > Cluster. Decommissioning a Failed APIC 6 Remove the failed APIC from your rack and install the replacement. 3 Record the fabric name. click the failed APIC to select it. 5 Choose Actions > Decommission. in the Navigation pane. Note: In the Work pane. and the TEP address space. click the new APIC to select it. This information is also available through the acidiag avread command on APIC’s CLI. The new APIC should boot to the initial setup script. you should see the failed APIC in the "Unavailable" operational state. 7 Proceed through the setup script and enter the values of the failed APIC that you recorded in step 3. 9 In the Work pane. . target size. node ID of the failed APIC. 4 In the Work pane. You can choose any APIC. 8 Once the new APIC finishes booting. The APIC changes to an "Out of Service" admin state. choose Controllers > apic_name > Cluster. Failure to configure the APIC with the same settings could result in the fabric entering a partially diverged state.Hardware Expansion and Replacement 405 2 In the Navigation pane.

which will be reflected in the APIC GUI. . The new APIC might also cycle between the Available and Unavailable operational states before becoming Fully Fit. run​ time and on-de​ mand di​ ag​ nos​ tics to help as​ sess the hard​ ware health of sev​ eral sub-sys​ tems on each leaf and spine switch. Diagnose Equipment Failures The ACI fab​ ric pro​ vides bootup. Recommissioning an APIC 11 The new APIC will receive an IP address.406 Hardware Expansion and Replacement 10 Choose Actions > Commission. you can verify that it has joined the fabric by logging in using the credentials that are configured for the rest of the fabric. Waiting for Cluster Convergence 12 On the command line of the new APIC. It might take 5 to 10 minutes for this to occur.

To look at the de​ fault di​ ag​ nos​ tic poli​ cies. card boots up. 2 Health (aka On-going) tests run periodically. Deployed via selectors. there are no defaults. Comes with default set of tests that can be modified. Can only run non-disruptive tests. Viewing Diagnostic Monitoring Policies in the GUI . These are typically ONLY disruptive tests.Hardware Expansion and Replacement 407 1 Boot-up tests run when switch. and they can be disruptive. Comes with default set of tests that can be modified and are deployed via selectors 3 On-Demand Tests are to be run on specific ports or cards for troubleshooting. tests are log​ i​ cally grouped into col​ lec​ tions. click fab​ ric > fab​ ric poli​ cies > Mon​ i​ tor​ ing poli​ cies > de​ fault > di​ ag​ nos​ tics pol​ icy In the work pane se​ lect the fab​ ric el​ e​ ment that you would like to view the di​ ag​ nos​ tic mon​ i​ tor​ ing pol​ icy for. By de​ fault.

. in the work pane se​ lect the Trou​ bleshoot​ ing tab to view GOLD di​ ag​ nos​ tic re​ sults for the su​ per​ vi​ sor. sys​ tem con​ troller and the fab​ ric mod​ ules in the sys​ tem. Viewing GOLD Diagnostics Information in the GUI In a mod​ u​ lar chas​ sis-based sys​ tem such as the Cisco Nexus 9500 se​ ries switch. it may be nec​ es​ sary to val​ i​ date a sys​ tem is healthy by run​ ning non-dis​ rup​ tive tests on the sys​ tem. mod​ ules.408 Hardware Expansion and Replacement Test re​ sults are view​ able by click​ ing on Fab​ ric > in​ ven​ tory > Pod-1 > Leaf-xx or Spine-xx > Chas​ sis > Su​ per​ vi​ sor mod​ ules > Slot-1 AND Fab​ ric > in​ ven​ tory > Pod-1 > Leaf-xx or Spine-xx > Chas​ sis > Line mod​ ules > Slot-1 Once there. Create new on-demand diagnostic test As a part of op​ er​ a​ tional pro​ ce​ dures. di​ ag​ nos​ tic re​ sults are avail​ able for all the su​ per​ vi​ sor.

se​ lect a test con​ fig (de​ fault is "no tests") and se​ lect the fab​ ric ports that the test would need to be run on. an op​ er​ a​ tor is cre​ at​ ing a fab​ ric-port on-de​ mand diag pol​ icy that al​ lows the abil​ ity to test the leaf up​ link ports going to the spine. the APIC GUI al​ lows cre​ ation of an On-de​ mand di​ ag​ nos​ tic test. Enter a name for the test. right click on the test or test set you would like to run on de​ mand. In this case. under the di​ ag​ nos​ tic tests. To do this. and click on Cre​ ate. the op​ er​ a​ tor se​ lects the op​ tions for a non-dis​ rup​ tive test to be done on leaf-2. en​ sure you check the box against "in​ clude dis​ rup​ tive tests" if this is the in​ tent. se​ lect the admin state. fab​ ric-port 1/97.Hardware Expansion and Replacement 409 In order to do this. . nav​ i​ gate to Fab​ ric > Fab​ ric Poli​ cies > Trou​ bleshoot Poli​ cies > On de​ mand di​ ag​ nos​ tics Once there. In the ex​ am​ ple below. Creating a On-Demand Diag Policy in the GUI Once you click cre​ ate.

410 Hardware Expansion and Replacement Creating a Fabric Port On-Demand Diag Policy in the GUI After ver​ i​ fy​ ing the pa​ ra​ me​ ters above. Once sub​ mit​ ted. you can kick off the di​ ag​ nos​ tic test by click​ ing on the test it​ self and click​ ing sub​ mit. Note that APIC dis​ plays a warn​ ing mes​ sage in cases where a non-dis​ rup​ tive tests are se​ lected to the ef​ fect of below. . click "SUB​ MIT" to sub​ mit the pol​ icy.

the di​ ag​ nos​ tic re​ sults can be ob​ tained from the same lo​ ca​ tion as the lo​ ca​ tion for di​ ag​ nos​ tics that are run at bootup or are on​ go​ ing.Hardware Expansion and Replacement 411 GUI Warning for Executing a Disruptive Diag Policy Once you con​ firm the test run. Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-2 > Line-mod​ ules > Slot-1 > Fab​ ric ports > 1/97 Viewing On-Demand Diag Policy Test Results in the GUI Note the on-de​ mand re​ sults in the right hand cor​ ner after the test has com​ pleted its run. .

.

413 Appendix .

.

or error in​ for​ ma​ tion.Appendix 415 Classes The Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) classes are cru​ cial from an op​ er​ at​ ional per​ spec​ tive to un​ der​ stand how sys​ tem events and faults re​ late to ob​ jects within the ob​ ject model. The API ac​ cepts and re​ turns HTTP or HTTPS mes​ sages that con​ tain JSON or XML doc​ um ​ents. The HTML body of the POST mes​ sage con​ tains a JSON or XML data struc​ ture that de​ scribes an MO or an API method. You can in​ voke an API func​ tion by send​ ing an HTTP/1. Each node in the tree rep​ re​ sents a man​ aged ob​ ject (MO) or group of ob​ jects that con​ tains its ad​ min​ is​ tra​ tive state and its op​ er​ at​ ional state. All the phys​ ic ​al and log​ ic ​al com​ po​ nents that com​ prise the Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture fab​ ric are rep​ re​ sented in a hi​ er​ ar​ chi​ cal man​ age​ ment in​ for​ ma​ tion tree (MIT).1 or HTTPS POST. and/or sta​ tis​ tics. fault. The fol​ low​ ing sec​ tion is a rep​ re​ sen​ ta​ tion of use​ ful classes for es​ tab​ lish​ ing a foun​ da​ tion for mon​ it​ or​ ing and man​ age​ ment of the fab​ ric. or DELETE mes​ sage to the APIC. con​ fir​ ma​ tion of a re​ quested ac​ tion. The APIC REST API is a pro​ gram​ matic in​ ter​ face to the APIC that uses a REST ar​ chi​ tec​ ture. Each event and/or fault in the sys​ tem is a unique ob​ ject that can be ac​ cessed for con​ fig​ ur​ a​ tion. To ac​ cess the com​ plete list of classes. The list below is a sub​ set of the full list of the avail​ able classes. You can use any pro​ gram​ ming lan​ guage to gen​ er​ ate the mes​ sages. point to the APIC and ref​ er​ ence the doc/html di​ rec​ tory at the end of the URL: https://apic_ip_address/doc/html/ . and the JSON or XML doc​ um ​ents that con​ tain the API meth​ ods or man​ aged ob​ ject (MO) de​ scrip​ tions. The HTML body of the re​ sponse mes​ sage con​ tains a JSON or XML struc​ ture that con​ tains re​ quested data. GET. health.

One in​ stance ob​ ject is cre​ ated for each fault con​ di​ tion of the par​ ent ob​ ject. cur​ rent time. node model num​ bers and de​ vice roles. leafs and spines. in​ clud​ ing con​ trollers. topSystem REST :: https://172. This ob​ ject is at​ tached as a child of the ob​ ject on which the fault con​ di​ tion oc​ curred.96.96. . in​ clud​ ing con​ trollers. Usage: The top​ Sys​ tem class can be used to de​ rive ob​ ject prop​ er​ ties in​ clud​ ing inb/oob man​ age​ ment de​ tails.json faultInst Name: fault:Inst De​ scrip​ tion: Con​ tains de​ tailed in​ for​ ma​ tion of the fault. leafs and spines.16.json fabricNode Name: fab​ ric:Node De​ scrip​ tion: Pro​ vides a list of all the nodes that are part of the fab​ ric. A fault in​ stance ob​ ject is iden​ ti​ fied by a fault code.416 Appendix Fabric Monitoring topSystem Name: top:Sys​ tem De​ scrip​ tion: Pro​ vides a list of all the de​ vices within the fab​ ric. Usage: The fab​ ric​ N​ ode class can be used to de​ rive ob​ ject prop​ er​ ties in​ clud​ ing node se​ ri​ al num​ bers. ten​ ant or in​ di​ vid​ ual man​ aged ob​ jects within the APIC. Usage: The fault​ Inst class can be used to de​ rive all faults as​ so​ ci​ ated with the fab​ ric. fabricNode REST :: https://172.16. as​ signed node ids.2/api/node/class/fabricNode.2/api/node/class/topSystem. sys​ tem up​ time and cur​ rent state.

Appendix 417 faultInst REST :: https://172.2/api/node/class/faultInst. Usage: The fvCEp class can be used to de​ rive a list of end points at​ tached to the fab​ ric and the as​ so​ ci​ ated ip/mac ad​ dress and en​ cap​ su​ la​ tion for each ob​ ject. The fab​ ricHealth​ To​ tal class can be used to de​ rive the over​ all sys​ tem fabricHealthTotal REST :: https://172.96.json fvRsCEpToPathEp Name: De​ scrip​ tion: fv:RsCEp​ ToPathEp This is an in​ ter​ nal ob​ ject that pro​ vides a re​ la​ tion to a path end​ point.16. Usage: The fvRsCEp​ ToPathEp class can be used to de​ rive path fab​ ric de​ tails such as the node and port as well as the ten​ ant de​ tails such as the ten​ ant name.2/api/node/class/fvCEp.json eqptFabP .16.96. fab​ ric:Health​ To​ tal The fab​ ric total health score in​ stance.16.2/api/node/class/fvRsCEpToPathEp.96.96. fvCEp REST :: https://172.json fabricHealthTotal Name: De​ scrip​ tion: Usage: health. fvRsCEpToPathEp REST :: https://172.16.json fvCEp Name: De​ scrip​ tion: fv:CEp A client end​ point at​ tach​ ing to the net​ work. ap​ pli​ ca​ tion pro​ file and end point group.2/api/node/class/fabricHealthTotal.

16.json eqptLC Name: eqpt:LCA .2/api/node/class/eqptCh. Usage: The eqpt​ FabP class can be used to de​ rive a list of non-fab​ ric port and the as​ so​ ci​ ated de​ tails such as the line card and chas​ sis place​ ment.json eqptCh Name: De​ scrip​ tion: eqpt:ChA The hard​ ware chas​ sis con​ tainer.2/api/node/class/eqptLeafP. the non-fab​ ric fac​ ing ex​ ter​ nal leaf IO port. Usage: The eqpt​ FabP class can be used to de​ rive a list of fab​ ric port and the as​ so​ ci​ ated de​ tails such as the line card and chas​ sis place​ ment.json eqptLeafP Name: De​ scrip​ tion: eqpt:LeafP Fab​ ric port.96.96. se​ ri​ al num​ ber and model num​ ber.418 Appendix eqptFabP Name: De​ scrip​ tion: eqpt:FabP Fab​ ric port.16. eqptFabP REST :: https://172.2/api/node/class/eqptFabP.96.16. eqptLeafP REST :: https://172. Usage: The eqptCh class can be used to de​ rive a chas​ sis list and the as​ so​ ci​ ated de​ tails such as the op​ er​ at​ ional state. the fab​ ric fac​ ing ex​ ter​ nal IO port. eqptCh REST :: https://172.

se​ ri​ al num​ ber. con​ tain​ ing IO ports. The eqptLC class can be used to de​ rive a list of line cards de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the re​ dun​ dancy state.2/api/node/class/eqptFt.96. and the volt​ age source.json eqptPsu Name: De​ scrip​ tion: eqpt:Psu The power sup​ ply unit. model. .2/api/node/class/eqptPsu.16.96. se​ ri​ al num​ bers and the num​ ber of ports.Appendix 419 De​ scrip​ tion: Usage: The line card (IO card).json eqptFt Name: De​ scrip​ tion: eqpt:Ft The in​ ven​ to​ ried fan tray.96.2/api/node/class/eqptLC. eqptPsu REST :: https://172.json eqptSupC Name: De​ scrip​ tion: eqpt:SupC The su​ per​ vi​ sor card. eqptFt REST :: https://172. eqptLC REST :: https://172. which con​ tains the CPU run​ ning con​ trol plane.16. Usage: The eqptFt class can be used to de​ rive a list of fan trays and the as​ so​ ci​ ated de​ tails such as the op​ er​ a​ tional sta​ tus.16. model num​ ber. Usage: The eqptFt class can be used to de​ rive a list of power sup​ plies within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the model num​ ber. op​ er​ at​ ional sta​ tus. se​ r​ ial num​ ber and hard​ ware ver​ sion.

Usage: The eth​ pm​ PhysIf class can be used to de​ rive a list of phys​ ic ​al in​ ter​ faces in the fab​ ric and the as​ so​ ci​ ated de​ tails such as a the speed.2/api/node/class/ethpmPhysIf.16.json dbgEpgToEpgRslt Name: De​ scrip​ tion: entry. Usage: The db​ gAc​ Trail class can be used to de​ rive a list of the atomic coun​ ters de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts.json dbgAcTrail Name: De​ scrip​ tion: dbg:Ac​ Trail The atomic counter trail. dbg:Epg​ ToEp​ gRslt The end​ point group to end​ point group atomic counter. op​ er​ at​ ional sta​ tus and re​ dun​ dancy state. on-de​ mand. du​ plex.96. eqptSupC REST :: https://172. se​ ri​ al num​ ber.2/api/node/class/dbgAcTrail.16. ethpmPhysIf REST :: https://172.json ethpmPhysIf Name: De​ scrip​ tion: ethpm:PhysIf The phys​ ic ​al in​ ter​ face in​ for​ ma​ tion holder. .16.420 Appendix Usage: The eqptFt class can be used to de​ rive a list of su​ per​ vi​ sor cards de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the model num​ ber.96.2/api/node/class/eqptSupC. op​ er​ at​ ional sta​ tus. dbgAcTrail REST :: https://172. and usage state.96.

16.16.96.2/api/node/class/compVm.json dbgEpToEpRslt Name: De​ scrip​ tion: Usage: dbg:Ep​ ToEpRslt The end​ point to end​ point atomic counter.96. .2/api/node/class/dbgEpgToEpgRslt. The dbgEp​ ToEpT​ sIt class can be used to de​ rive a list of the end​ point to end​ point atomic coun​ ters de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts.16. dbgEpToEpTsIt REST :: https://172.json compHv Name: De​ scrip​ tion: comp:Hv An ob​ ject rep​ re​ sent​ ing the com​ pute hy​ per​ vi​ sor. dbgEpgToEpgRsIt REST :: https://172. and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts. On-de​ mand.json VMM Monitoring compVm Name: De​ scrip​ tion: comp:Vm The Vir​ tual ma​ chine ob​ ject. compVm REST :: https://172. Entry. Usage: The com​ pVm class can be used to de​ rive a list of vir​ tual ma​ chines de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the name and state.Appendix 421 Usage: The dbgEpg​ ToEp​ gR​ sIt class can be used to de​ rive a list of the EPG to EPG atomic coun​ ters de​ ployed within the fab​ ric.96.2/api/node/class/dbgEpToEpRslt.

Usage: The fvR​ sHy​ per class can be used to de​ rive the re​ la​ tion​ ship of the hy​ per​ vi​ sor that con​ trols and mon​ it​ ors the APIC VMs.json fvRsHyper Name: De​ scrip​ tion: fv:RsHy​ per A re​ la​ tion to the hy​ per​ vi​ sor that con​ trols and mon​ it​ ors the APIC VMs.16. For ex​ am​ ple.2/api/node/class/fvRsVm.422 Appendix Usage: The com​ pVm class can be used to de​ rive a list of com​ pute hy​ per​ vi​ sor de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the name and sta​ tus.96. the VMM con​ troller pro​ file could be a pol​ icy to con​ nect a VMware vCen​ ter that is part a VMM do​ main.json fvRsVm Name: De​ scrip​ tion: ter​ nal ob​ ject. .2/api/node/class/compHv.16.16.96. fvRsHyper REST :: https://172. vRsVm REST :: https://172. compHv REST :: https://172. which spec​ if​ ies how to con​ nect to a sin​ gle VM man​ age​ ment con​ troller that is part of con​ tain​ ing pol​ icy en​ force​ ment do​ main. This is an in​ - Usage: The fvRsVm class can be used to de​ rive the re​ la​ tion​ ship of the vir​ tual ma​ chines con​ nected to the hy​ per​ vi​ sor.json vmmCtrlrP Name: vmm:CtrlrP De​ scrip​ tion: The VMM con​ troller pro​ file. fv:RsVm A re​ la​ tion to a vir​ tual ma​ chine con​ nected to a hy​ per​ vi​ sor.2/api/node/class/fvRsHyper. This is an in​ ter​ nal ob​ ject.96.

json . Usage: The class vnsLDevVip can be used to de​ rive all the VIPs con​ fig​ ured for the log​ ic ​al de​ vice clus​ ters in the fab​ ric vnsLDevVip REST :: https://172. ab​ stract term nodes (the nodes that are con​ nected to end​ point groups).16.16. along with their prop​ er​ ties. Usage: The class vns​ Ab​ s​ Graph can be used to de​ rive a list of ser​ vice graph tem​ - plates con​ fig​ ured on the APIC. and con​ nec​ tions.2/api/node/class/vnsAbsGraph.json Layer 4 to Layer 7 Monitoring vnsAbsGraph Name: vns​ Ab​ s​ Graph De​ scrip​ tion: The ab​ stract graph is made up of ab​ stract nodes and used to de​ fine the traf​ fic flow through a ser​ vice func​ tion such as load bal​ anc​ ing.2/api/node/class/vmmCtrlrP.2/api/node/class/vnsLDevVip. which is rep​ re​ sented by a sin​ gle vir​ tual IP (VIP).96.16. Ab​ stract nodes are com​ prised of ser​ vice nodes such as a ser​ vice node bal​ ancer (SLB) or fire​ wall (FW). vnsAbsGraph REST :: https://172.Appendix 423 Usage: The vmm​ C​ trlrP class can be used to de​ rive the ip ad​ dress and the dat​ a-​ cen​ ter name of the con​ nected VM do​ main. SSL of​ fload.json vnsLDevVip Name: vnsLDevVip De​ scrip​ tion: An L4-L7 de​ vice clus​ ter. vmmCtrlrP REST :: https://172. or fire​ wall. The con​ fig​ ur​ a​ tion is pushed down to the VIP ad​ dress.96.96.

json vnsRsLDevCtxToLDev . which points to the de​ vice clus​ ter used to pick a spe​ cific de​ vice based on con​ tract.16.96.96. which is as​ so​ ci​ ated with a set of con​ crete in​ ter​ - faces from the L4-L7 de​ vice clus​ ter.16.96. set the name to Any. and func​ tion label or names. Usage: The class vn​ sCDev can be used to de​ rive a list of con​ crete de​ vices con​ fig​ ured as part of the L4-7 ser​ vice in​ te​ gra​ tion vnsCDev REST :: https://172. vnsLif REST :: https://172.json vnsLif Name: vnsLif De​ scrip​ tion: The log​ ic ​al in​ ter​ face.424 Appendix vnsCDev Name: vn​ sCDev De​ scrip​ tion: The in​ di​ vid​ ual ser​ vice de​ vice. sub​ ject.2/api/node/class/vnsLDevCtx. Usage: The class vnsLDe​ vCtx can be used to de​ rive the node and con​ tract name. nsLDevCtx REST :: https://172.2/api/node/class/vnsCDev. To spec​ ify a wild card. which is used to de​ fine a con​ crete l4-l7 ser​ vice de​ vice.16. Usage: The class vnsLif can be used to de​ rive the con​ nec​ tion be​ tween a ser​ vice graph and de​ vice in​ ter​ faces.2/api/node/class/vnsLIf.json vnsLDevCtx Name: vnsLDe​ vCtx De​ scrip​ tion: A de​ vice clus​ ter con​ text.

Usage: The com​ pRcvdEr​ rP​ kt​ s​ 1h class can be used to de​ rive the most cur​ rent sta​ tis​ tics for re​ ceived error pack​ ets.json .96.Appendix 425 vnsRsLDevCtxToLDev Name: De​ scrip​ tion: vn​ sRsLDe​ vC​ tx​ ToLDev A source re​ la​ tion to the ab​ strac​ tion of a ser​ vice de​ vice clus​ ter or of a proxy ob​ ject for a log​ ic ​al de​ vice clus​ ter in the ten​ ant.2/api/node/class/compRcvdErrPkts1h.2/api/node/class/compHostStats1h. compRcvdErrPkts1h REST :: https://172. Usage: The class vn​ sRsLDe​ vC​ tx​ ToLDev can be used to de​ rive the re​ la​ tion​ ship be​ tween vnsLDe​ vCtx and vnsLDev.96.16. vnsRsLDevCtxToLDev REST :: https://172.16. This class up​ dates every 15 min​ utes. Usage: The com​ pHost​ Stat​ s1h class can be used to de​ rive the sta​ tis​ tics as​ so​ ci​ ated with the com​ pute hy​ per​ vi​ sor.16.json compRcvdErrPkts1h Name: comp:RcvdEr​ rP​ kt​ s​ 1h De​ scrip​ tion: A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for re​ ceived error pack​ ets in a 1 hour sam​ pling in​ ter​ val.96. compHostStats1h REST :: https://172. This class up​ dates every 15 min​ utes.2/api/node/class/vnsRsLDevCtxToLDev.json Statistics compHostStats1h Name: comp:Host​ Stat​ s1h De​ scrip​ tion: A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for host in a 1 hour sam​ pling in​ ter​ val.

json aaaUser Name: De​ scrip​ tion: aaa:User A lo​ cally-au​ then​ ti​ cated user ac​ count. compTrnsmtdErrPkts1h REST :: https://172.96.json .426 Appendix compTrnsmtdErrPkts1h Name: De​ scrip​ tion: comp:Trnsmt​ dEr​ rP​ kt​ s​ 1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for trans​ mit​ ted error pack​ ets in a 1 hour sam​ pling in​ ter​ val. Usage: The aaaUser class can be used to de​ rive a list of user ac​ counts de​ ployed within the fab​ ric.16. Authorization.json Authentication. aaaUser REST :: https://172.96. This class up​ dates every 15 min​ utes.16.16. Usage: The aaaModLR class can be used to de​ rive a fab​ ric based audit log for all changes and events. Usage: The compTrnsmt​ dEr​ rP​ kt​ s​ 1h class can be used to de​ rive the most cur​ rent sta​ tis​ tics for trans​ mit​ ted error pack​ ets. and Accounting aaaModLR Name: aaa:ModLR De​ scrip​ tion: The AAA audit log record.96.2/api/node/class/aaaUser.2/api/node/class/compTrnsmtdErrPkts1h. aaaModLR REST :: https://172.2/api/node/class/aaaModLR. A log record is au​ to​ mat​ ic ​ally gen​ er​ ated when​ ever a user mod​ if​ ies an ob​ ject.

Usage: The aaaUser class can be used to de​ rive a list of re​ mote user ac​ counts de​ ployed within the fab​ ric.96.16. eqptcapacityPolEntry5min REST :: http://172. This class up​ dates every 10 sec​ onds.2/api/class/eqptcapacityPolEntry5min. .16. Usage: The eqpt​ ca​ pac​ it​ y​ Po​ lEn​ try5min class can be used to de​ rive the cur​ rent value as​ so​ ci​ ated with the Pol​ icy TCAM usage.96. A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for lay​ er3 entry in a 5 minute sam​ pling in​ ter​ val.2/api/node/class/aaaRemoteUser. aaaRemoteUser REST :: https://172. This class up​ dates every 10 sec​ onds.json Prefix TCAM Name: eqpt​ ca​ pac​ ityL3En​ try5min De​ scrip​ tion: Lay​ er3 entry sta​ tis​ tics. Usage: The eqpt​ ca​ pac​ ityL3En​ try5min class can be used to de​ rive the cur​ rent value as​ so​ ci​ ated with the Pre​ fix TCAM usage. A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for pol​ icy entry in a 5 minute sam​ pling in​ ter​ val.Appendix 427 aaaRemoteUser Name: aaa:Re​ mo​ teUser De​ scrip​ tion: A re​ mote user login ac​ count.json Fabric Capacity Policy TCAM Name: eqpt​ ca​ pac​ it​ y​ Po​ lEn​ try5min De​ scrip​ tion: Pol​ icy CAM entry sta​ tis​ tics.

json Use Cases The class fault​ Inst used in Use Case #1 and Use Case #2 below can be re​ placed with any of the man​ aged ob​ ject classses dis​ cussed above or spec​ if​ ied within the APIC doc​ u-​ men​ ta​ tion.96.96.2/api/node/class/syslogRemoteDest.16. syslogRemoteDest REST :: https://172. Usage: The sys​ lo​ gRe​ mot​ eDest class can be used to de​ rive the cur​ rent list of sys​ log re​ mote des​ ti​ na​ tions im​ ple​ mented within the fab​ ric.96.16.​ cisco.see: http://​ www. The Cisco APIC Com​ mand-Line In​ ter​ face User Guide may also be help​ ful for un​ der​ stand​ ing the fol​ low​ ing sec​ tions .428 Appendix eqptcapacityL3Entry5min REST :: https://172.json Prefix TCAM Name: sys​ lo​ gRe​ mot​ eDest De​ scrip​ tion: The sys​ log re​ mote des​ ti​ na​ tion host en​ ables you to spec​ ify sys​ log servers to which mes​ sages from the APIC and fab​ ric nodes should be for​ warded. De​ scrip​ tion: Usage: The sn​ mpTrapDest class can be used to de​ rive the cur​ rent list of snmp trap des​ ti​ na​ tions im​ ple​ mented within the fab​ ric.json SNMP/SYSLOG SNMP Trap Destination Name: sn​ mpTrapDest A des​ ti​ na​ tion to which traps and in​ forms are sent. snmpTrapDest REST :: https://172.​ html .16.2/api/class/eqptcapacityL3Entry5min.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ cli/​ b_​ APIC_​ CLI_​ User_​ Guide.2/api/node/class/snmpTrapDest.

access import cobra.mit. The user has the op​ tion of col​ lect​ ing the re​ sults via CLI.session from cobra.Appendix 429 Case 1: Creating an application script to retrieve the current list of faults in the fabric.mit. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -c faultInst From a Vi​ sore per​ spec​ tive. user the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/class/faultInst.xml From a Cobra per​ spec​ tive. Vi​ sore. From a CLI per​ spec​ tive.query(classQuery): print fault.: Class or DN :: faultInst Property :: n/a Op :: n/a Value :: n/a From a POST​ MAN per​ spec​ tive.session import LoginSession .mit. use the fol​ low​ ing class query: # Class Query classQuery= ClassQuery('faultInst') for fault in md. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions.name Sam​ ple Cobra script to cap​ ture faults within the fab​ ric: #!/usr/bin/env python import cobra. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query. This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to ob​ tain the list of cur​ rent faults in the fab​ ric. POST​ MAN and/or Cobra.

LoginSession('https://'<your apic ip address>.faultInst.mit. Vi​ sore. From a CLI per​ spec​ tive. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions. POST​ MAN and/or Cobra.mit. This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cob​ tain the list of cur​ rent faults in the fab​ ric. secure=False) md = cobra.xml?query-targetfilter=and(eq(faultInst.MoDirectory(ls) md.name Case 2: Creating an application script to retrieve the current list of faults in the fabric that have been caused by a failed configuration. The user has an op​ tion of col​ lect​ ing the re​ sults via CLI.query(classQuery): print fault.request import ClassQuery ls = cobra.cause. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -c faultInst -f 'fv. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query: Class or DN :: faultInst Property :: cause Op :: == Value :: config-failure From a POST​ MAN per​ spec​ tive. <password>. <username>.cause=="config-failure"' From a Vi​ sore per​ spec​ tive."config-failure")) . use the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/class/faultInst.login() # Class Query classQuery= ClassQuery('faultInst') for fault in md.access.430 Appendix from cobra.session.mit.

#!/usr/bin/env python import cobra.query(classQuery): print fault.session import LoginSession from cobra.format('config-failure') for fault in md. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -d uni/tn-common . POST​ MAN and/or Cobra. "{0}")'.LoginSession('https://'<your apic ip address>. <password>.session. DN This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to ob​ tain the prop​ er​ ties of the ten​ ant name Com​ mon.name Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures.mit.MoDirectory(ls) md.name Case 3: Creating an application script to retrieve the properties for a specific managed object.query(classQuery): print fault.login() # Class Query classQuery= ClassQuery('faultInst') for fault in md. Vi​ sore.cause. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions.propFilter = 'wcard(faultInst. secure=False) md = cobra.mit. From a CLI per​ spec​ tive.mit. The user has an op​ tion of col​ lect​ ing the re​ sults via CLI.session from cobra.Appendix 431 From a Cobra per​ spec​ tive.access import cobra.mit.access.mit. use the fol​ low​ ing class query: # Class Query classQuery= ClassQuery('faultInst') classQuery.mit.request import ClassQuery ls = cobra. <username>.

login() # DN Query dnQuery= DnQuery('uni/tn-common') for results in md.dn Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures. #!/usr/bin/env python import cobra.query(dnQuery): print results.access import cobra. secure=False) md = cobra.session import LoginSession from cobra.mit.mit.session. <username>.xml?query-target=self From a Cobra per​ spec​ tive.dn .mit.session from cobra. use the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/mo/uni/tn-common. use the fol​ low​ ing class query: # DN Query dnQuery= DnQuery('uni/tn-common') for results in md.access.request import DnQuery ls = cobra. <password>.432 Appendix From a Vi​ sore per​ spec​ tive. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query: Class or DN :: uni/tn-common Property :: n/a Op :: n/a Value :: n/a From a POST​ MAN per​ spec​ tive.mit.query(dnQuery): print results.MoDirectory(ls) md.mit.LoginSession('https://'<your apic ip address>.mit.

dn .session.query(q) for mo in mos: for child in mo. #!/usr/bin/env python from cobra. secure=False) md = MoDirectory(ls) md.mit.login() q = ClassQuery('fvCEp') q.Appendix 433 Case 4: Creating an application script to retrieve the current list of endpoints (mac-addresses) attached to the fabric This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate an ap​ pli​ ca​ tion script to cap​ ture the list of cur​ rent end​ points at​ tached to the fab​ ric along with the node de​ tails per​ tain​ ing to each end​ point. <password>.mit.subtree = 'children' q.request import ClassQuery lls = cobra. <username>.rscEpToPathEp: print child. Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures.LoginSession('https://'<your apic ip address>.mit.subtreeClassFilter = 'fvRsCEpToPathEp' mos = md.mit.access import MoDirectory from cobra.session import LoginSession from cobra.

.

Here are some de​ scrip​ tions of com​ monly used ab​ bre​ vi​ at​ ions. which may help when de​ ci​ pher​ ing what class ob​ jects are when using them with REST calls. au​ tho​ riza​ tion. Package Decoder Aaa: au​ then​ ti​ ca​ tion. ac​ count​ ing ac: atomic coun​ ters actrl: ac​ cess con​ trol ac​ trl​ cap: ac​ cess con​ trol ca​ pa​ bil​ ity adcom: ap​ pli​ ance di​ rec​ tor com​ mu​ ni​ ca​ tion aib: ad​ ja​ cency in​ for​ ma​ tion base arp: ad​ dress res​ o​ lu​ tion pro​ to​ col bgp: bor​ der gate​ way pro​ to​ col call​ home: Cisco smart call home ser​ vices cap: ca​ pa​ bil​ ity cdp: Cisco dis​ cov​ ery pro​ to​ col cnw: node clus​ ter comm: com​ mu​ ni​ ca​ tion pol​ icy comp: com​ pute .Appendix 435 Package Decoder There are sev​ eral ab​ bre​ vi​ at​ ions used in the names of classes in the ACI ob​ ject model.

436 Appendix com​ pat: com​ pat​ ib ​il​ ity con​ di​ tion: health pol​ icy con​ fig: con​ fig​ ur​ a​ tion pol​ icy coop: Coun​ cil of Or​ ac ​les pro​ to​ col copp: con​ trol plane polic​ ing pol​ icy: con​ tains set of rules de​ scrib​ ing po​ licer rates ctrlr: con​ troller ctx: con​ text date​ time: date/time pol​ icy dbg: debug dbgac: debug atomic coun​ ters dbg​ exp: debug ex​ port pol​ icy dhcp: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col dhcptlv: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col type length value dhcptlvpol: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col type length value pol​ icy dns: do​ main name ser​ vice draw: graph vi​ su​ al​ iza​ tion for GUI epm: end​ point man​ ager eqpt: equip​ ment eqpt​ cap: equip​ ment ca​ pa​ bil​ ity eqpt​ ca​ pac​ ity: equip​ ment ca​ pac​ ity .

coun​ ters file: file path.Appendix 437 eqpt​ diag: equip​ ment di​ ag​ nos​ tics eqpt​ di​ agp: equip​ ment di​ ag​ nos​ tics pol​ icy ethpm: eth​ er​ net pol​ icy man​ ager event: event pol​ icy extnw: ex​ ter​ nal net​ work fab​ ric: fab​ ric fault: fault pol​ icy. con​ fig im​ port/ex​ port pol​ icy firmware: firmware fm​ cast: fab​ ric mul​ ti​ cast fsm: fi​ nite state ma​ chine fv: fab​ ric vir​ tu​ al​ iza​ tion fvns: fab​ ric vir​ tu​ al​ iza​ tion name​ space fv​ topo: fab​ ric vir​ tu​ al​ iza​ tion topol​ ogy geo: ge​ olo​ ca​ tion glean: glean ad​ ja​ cency ha: high avail​ abil​ ity health: health score hvs: hy​ per​ vi​ sors vir​ tual switch icmp: in​ ter​ net con​ trol pro​ to​ col .

438 Appendix icm​ pv4: in​ ter​ net con​ trol pro​ to​ col ver​ sion 4 icm​ pv6: in​ ter​ net con​ trol pro​ to​ col ver​ sion 6 ident: iden​ tity igmp: in​ ter​ net group man​ age​ ment pro​ to​ col igmp​ snoop: in​ ter​ net group man​ age​ ment pro​ to​ col snoop​ ing im: in​ ter​ face man​ ager mod​ ule im​ gin​ stall: image in​ stall infra: in​ fra​ struc​ ture ip: in​ ter​ net pro​ to​ col ipv4: in​ ter​ net pro​ to​ col ver​ sion 4 ipv6: in​ ter​ net pro​ to​ col ver​ sion 6 isis: in​ ter​ me​ di​ ate sys​ tem to in​ ter​ me​ di​ ate sys​ tem isistlv: in​ ter​ me​ di​ ate sys​ tem to in​ ter​ me​ di​ ate sys​ tem type length value l1: layer 1 l1cap: layer 1 ca​ pa​ bil​ ity l2: layer 2 l2cap: layer 2 ca​ pa​ bil​ ity l2ext: layer 2 ex​ ter​ nal l3: layer 3 l3cap: layer 3 ca​ pa​ bil​ ity .

not in the fab​ ric) lldp: link layer dis​ cov​ ery pro​ to​ col lldptlv: link layer dis​ cov​ ery pro​ to​ col type length value lldptlvpol: link layer dis​ cov​ ery pro​ to​ col type length value pol​ icy maint: main​ te​ nance mcast: mul​ ti​ cast mcp: mas​ ter con​ trol proces​ sor mem​ ory: mem​ ory sta​ tis​ tics mgmt: man​ age​ ment mo: man​ aged ob​ ject mock: mock (ob​ jects used on the sim​ ul​ a​ tor mostly for show​ ing stats/faults/etc) mon: mon​ it​ or​ ing mon​ i​ tor: mon​ it​ or (SPAN) nam​ ing: ab​ stract for ob​ jects with names nd: neigh​ bor dis​ cov​ ery nw: net​ work .Appendix 439 l3ext: layer 3 ex​ ter​ nal l3vm: Layer 3 Vir​ tual Ma​ chine lacp: link ag​ gre​ ga​ tion pro​ to​ col lbp: load bal​ anc​ ing pol​ icy leqpt: loose equip​ ment (un​ man​ aged nodes.

440 Appendix oam: eth​ er​ net op​ er​ at​ ions. cpu. and mem​ ory uti​ liza​ tion sta​ tis​ tics psu: power sup​ ply unit pol​ icy qos: qual​ ity of ser​ vice pol​ icy qosm: qos sta​ tis​ tics qosp: qos/ 802. logs/his​ tory opflex: OpFlex os: op​ er​ at​ ing sys​ tem ospf: open short​ est path first pc: port chan​ nel pcons: **gen​ er​ ated and used by in​ ter​ nal processes** phys: phys​ ic ​al do​ main pro​ file ping: ping ex​ e​ cu​ tion and re​ sults pki: pub​ lic key in​ fra​ struc​ ture pol: pol​ icy de​ f​ in ​i​ t​ ion po​ licer: traf​ fic polic​ ing (rate lim​ it​ ing) pool: ob​ ject pool pres: **gen​ er​ ated and used by in​ ter​ nal processes** proc: sys​ tem load. ad​ min​ is​ tra​ tions and man​ age​ ment ob​ server: ob​ server for sta​ tis​ tics.1p rbqm: de​ bug​ ging . fault. state. health.

Appendix 441 regress: re​ gres​ sion reln: **gen​ er​ ated and used by in​ ter​ nal processes** repl: **gen​ er​ ated and used by in​ ter​ nal processes** res: **gen​ er​ ated and used by in​ ter​ nal processes** rib: rout​ ing in​ for​ ma​ tion base rmon: re​ mote net​ work mon​ it​ or​ ing/ in​ ter​ face stats/coun​ ters rpm: route pol​ icy map rtcom: route con​ trol com​ mu​ nity list rtc​ trl: route con​ trol rtextcom: router ex​ tended com​ mu​ nity rtflt: route fil​ ter rtleak: route leak rtmap: RPM route map rtpfx: route pre​ fix list rtreg​ com: route reg​ ul​ ar com​ mu​ nity list rtsum: route sum​ ma​ riza​ tion ad​ dress/pol​ icy satm: satel​ lite man​ ager snmp: sim​ ple net​ work man​ age​ ment pro​ to​ col span: switched port an​ al​ yzer stats: sta​ tis​ tics col​ lec​ tion poli​ cies .

length. in​ stance. and re​ sult test: ab​ stract class for test rule. sys​ tem states. and re​ sult testin​ fralab: test in​ fra​ struc​ ture tlv: type. sub​ ject.442 Appendix stat​ store: sta​ tis​ tics data hold​ ers storm​ c​ trl: storm con​ trol (traf​ fic sup​ pres​ sion) pol​ icy stp: span​ ning tree pro​ to​ col de​ f​ in ​i​ t​ ions and pol​ icy sts: Ser​ vice Tag Switch​ ing (used for ser​ vices in​ ser​ tion) svc​ core: core pol​ icy svi: switched vir​ tual in​ ter​ face/ routed VLAN in​ ter​ face syn​ thetic: syn​ thetic ob​ jects (for test​ ing) sys​ de​ bug: sys​ tem debug sys​ file: sys​ tem files syshist: sys​ tem cards reset records/his​ tory sys​ log: sys​ log pol​ icy sys​ mgr: sys​ tem man​ ager (firmware. su​ per​ vi​ sor. etc) sys​ m​ grp: con​ tainer for cores pol​ icy & ab​ stract class for all qos pol​ icy de​ f​ in ​i​ t​ ions tag: alias (use de​ scrip​ tive name for dn). tags (group mul​ ti​ ple ob​ jects by a de​ scrip​ tive name) task: task ex​ e​ cu​ tion. value sys​ tem struc​ tures top: sys​ tem task man​ ager for proces​ sor ac​ tiv​ ity .

fab​ ric VxLan.e. vmm pol​ icy and de​ f​ in ​i​ t​ ions) vns: vir​ tual net​ work ser​ vice (L4-L7 pol​ icy and de​ f​ in ​i​ t​ ions) vpc: vir​ tual port chan​ nel (vpc pol​ icy and de​ f​ in ​i​ t​ ions) vsvc: ser​ vice la​ bels (provider/con​ sumer) vtap: trans​ lated ad​ dress of ex​ ter​ nal node (NATed IP of ser​ vice node) vxlan: Vir​ tu​ ally ex​ ten​ si​ ble LAN de​ f​ in ​i​ t​ ions vz: vir​ tual zones (for​ mer name of the pol​ icy con​ trols) i.Appendix 443 topoc​ trl: topol​ ogy con​ trol pol​ icy (shard​ ing. fab​ ric LB. Con​ tracts Model Naming schemes Rs: Re​ la​ tion​ ship source Rt: Re​ la​ tion​ ship tar​ get Ag: Ag​ gre​ gated stats BrCP: Bi​ nary Con​ tract Pro​ file . etc) tracer​ oute: tracer​ oute ex​ e​ cu​ tion and re​ sults tracer​ outep: tracer​ oute end points trig: trig​ ger​ ing pol​ icy tun​ nel: tun​ nel​ ing uribv4: ipv4 uni​ cast rout​ ing in​ for​ ma​ tion base en​ tity vlan: vlan in​ stances vlan​ mgr: vlan man​ ager con​ trol plane vmm: vir​ tual macine man​ ager (con​ troller.

.

Au​ tho​ riza​ tion. ALE: Ap​ pli​ ca​ tion Leaf En​ gine. AEPs are also the mech​ an ​ism that ties the phys​ ic ​al port to the do​ main (phys​ ic ​al or vir​ tual) to a switch pol​ icy. Dura​ bil​ ity – prop​ er​ ties of trans​ ac​ tions that en​ sure con​ sis​ tency in data​ base trans​ ac​ tions. only the key ones that may not be a part of the com​ mon ver​ nac​ ul​ ar. Trans​ ac​ tions to APIC de​ vices in an ACI clus​ ter are con​ sid​ ered ACID. This is not meant to be an ex​ haus​ tive list nor a com​ pletely de​ tailed dic​ tio​ nary of all of the terms and con​ cepts. and Ac​ count​ ing. there are some new terms and con​ cepts em​ ployed. Con​ sis​ tency. While ACI does not change how pack​ ets are trans​ mit​ ted on a wire. A AAA: acronym for Au​ then​ ti​ ca​ tion. AEP: At​ tach​ able En​ tity Pro​ file – this is a con​ fig​ ur​ a​ tion pro​ file of the in​ ter​ face that gets ap​ plied when an en​ tity at​ taches to the fab​ ric. This means that if one part of a trans​ ac​ tion fails the en​ tire trans​ ac​ tion fails. where end​ points fall out​ side of the man​ aged scope of the fab​ ric. An AEP rep​ re​ sents a group of ex​ ter​ nal en​ ti​ ties with sim​ il​ ar in​ fra​ struc​ ture pol​ icy re​ quire​ ments. . As​ so​ ci​ ated new acronyms are also pro​ vided. and un​ der​ stand​ ing those new terms and con​ cepts will help those work​ ing on ACI com​ mu​ ni​ cate with one an​ other about the con​ structs used in ACI to trans​ mit those bits. ACI Ex​ ter​ nal Con​ nec​ tiv​ ity: Any con​ nec​ tiv​ ity to and from the fab​ ric that uses an ex​ ter​ nal routed or switched in​ ter​ me​ di​ ary sys​ tem. or which would be rel​ e​ vant to the trou​ bleshoot​ ing ex​ er​ cises that were cov​ ered in the trou​ bleshoot​ ing sce​ nar​ ios dis​ cussed. ACID trans​ ac​ tions: ACID is an acronym for Atom​ ic​ ity. Iso​ la​ tion. an ASIC on a leaf switch. to en​ sure that data​ base con​ sis​ tency is main​ tained.Appendix 445 Acronyms and Definitions Overview This sec​ tion is de​ signed to pro​ vide a high level de​ scrip​ tion of terms and con​ cepts that get brought up in this book.

a BD is a child of a Pri​ vate Layer 3 or con​ text. API: Ap​ pli​ ca​ tion Pro​ gram​ ming In​ ter​ face used for pro​ gram​ ma​ ble ex​ ten​ si​ bil​ ity.) for each unique Layer 2 for​ ward​ ing do​ main. Bridge Do​ mains are also a con​ tainer for IP sub​ nets and are where fab​ ric Layer 3 gate​ way func​ tion​ al​ ity is con​ fig​ ured. In the ACI ob​ ject model. . ACI works on a white list pol​ icy model. on the ACI fab​ ric Multi-Pro​ to​ col BGP is used to dis​ trib​ ute reach​ ab ​il​ ity in​ for​ ma​ tion within the fab​ ric. C CLOS fab​ ric: A multi-tier non​ block​ ing leaf-spine ar​ chi​ tec​ ture net​ work. ARP flood​ ing. an ASIC on a Spine switch. Bridge Do​ main: An ACI con​ struct that de​ fines Layer 2 for​ ward​ ing be​ hav​ iors (Broad​ cast. and In​ ter​ nal BGP is used to peer the fab​ ric with ex​ ter​ nal Layer 3 de​ vices. BDs can em​ u​ late the be​ hav​ ior of a tra​ di​ tional VLAN but are not con​ strained by for​ ward​ ing scale lim​ i​ ta​ tions. the de​ fault for​ ward​ ing pol​ icy is to not allow any com​ mu​ ni​ ca​ tion be​ tween EPGs. Con​ tracts: A log​ ic ​al con​ tainer for the sub​ jects which re​ late to the fil​ ters that gov​ ern the rules for com​ mu​ ni​ ca​ tion be​ tween end​ point groups. but com​ mu​ ni​ ca​ tion within an EPG is al​ lowed. The AP is the key ob​ ject used to rep​ re​ sent an ap​ pli​ ca​ tion and is also the an​ chor point for the au​ to​ mated in​ fra​ struc​ ture man​ age​ ment in an ACI fab​ ric.446 Appendix APIC: Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller is a cen​ tral​ ized pol​ icy man​ age​ ment con​ troller clus​ ter. etc. The APIC con​ fig​ ures the in​ tended state of the pol​ icy to the fab​ ric. Clus​ ter: Set of de​ vices that work to​ gether as a sin​ gle sys​ tem to pro​ vide an iden​ ti​ cal or sim​ il​ ar set of func​ tions. With​ out a con​ tract. B BGP: Bor​ der Gate​ way Pro​ to​ col. ASE: Ap​ pli​ ca​ tion Spine En​ gine. Ap​ pli​ ca​ tion Pro​ file: Term used to ref​ er​ ence an ap​ pli​ ca​ tion pro​ file-man​ aged ob​ ject ref​ er​ ence that mod​ els the log​ ic ​al com​ po​ nents of an ap​ pli​ ca​ tion and how those com​ po​ nents com​ mu​ ni​ cate.

End​ point groups can be dy​ namic or sta​ tic. if pol​ icy ob​ ject of type Ap​ pli​ ca​ tion Pro​ file is cre​ ated. As an ex​ am​ ple. which is used to iden​ tify the end​ point. the dn would be ex​ pressed as uni/tn-Prod/ap-com​ merce​ work​ space.Appendix 447 Con​ text: A Layer 3 for​ ward​ ing do​ main. dMIT: dis​ trib​ uted Man​ age​ ment In​ for​ ma​ tion Tree. D DLB: Dy​ namic Load Bal​ anc​ ing – a net​ work traf​ fic load-bal​ anc​ ing mech​ an ​ism in the ACI fab​ ric based on flowlet switch​ ing. or po​ ten​ tially some other at​ tribute. A fault con​ tains the con​ di​ tions. F Fault: When a fail​ ure oc​ curs or an alarm is raised. etc. Ex​ am​ ples in​ clude vir​ tual-ma​ chines. End​ points have spe​ cific prop​ er​ ties like an ad​ dress. EPG: End Point Group. the sys​ tem cre​ ates a fault-man​ aged ob​ ject for the fault. a rep​ re​ sen​ ta​ tion of the ACI ob​ ject model with the root of the tree at the top and the leaves of the tree at the bot​ tom. in​ for​ ma​ tion about the op​ er​ at​ ional state of the af​ fected ob​ ject and po​ ten​ tial res​ ol​ u​ tions for the prob​ lem. a ser​ vice that runs on the APIC that man​ ages data for the data model. stor​ age de​ vices.Any log​ ic ​al or phys​ ic ​al de​ vice con​ nected di​ rectly or in​ di​ rectly to a port on a leaf switch that is not a fab​ ric fac​ ing port. DME: Data Man​ age​ ment En​ gine. E EP: End​ point . . It is made up of a con​ cate​ na​ tion of all of the rel​ at​ ive names from it​ self back to the root of the tree. servers. lo​ ca​ tion. Dn: Dis​ tin​ guished name – a fully qual​ if​ ied name that rep​ re​ sents a spe​ cific ob​ ject within the ACI man​ age​ ment in​ for​ ma​ tion tree as well as the spe​ cific lo​ ca​ tion in​ for​ ma​ tion in the tree. and in ACI ver​ nac​ ul​ ar a Pri​ vate Layer 3. equiv​ al​ ent to a VRF. The tree con​ tains all as​ pects of the ob​ ject model that rep​ re​ sent an ACI fab​ ric. named com​ merce work​ space within a Ten​ ant named Prod. A col​ lec​ tion of end​ points that can be grouped based on com​ mon re​ quire​ ments for a com​ mon pol​ icy.

a markup lan​ guage that fo​ cuses on the for​ mat​ ting of web pages. FCAPS is n acronym for FCAPS: The ISO model de​ fault. Flowlet Switch​ ing is a way to use TCP’s own bursty na​ ture to more ef​ fi​ ciently for​ ward TCP flows by dy​ nam​ ic ​ally split​ ting flows into flowlets. Flowlet switch​ ing: An op​ ti​ mized. I IFM: In​ tra-Fab​ ric Mes​ sages. load-bal​ anc​ ing method​ ol​ ogy based on re​ search from MIT in 2004. Used for com​ mu​ ni​ ca​ tion be​ tween dif​ fer​ ent de​ vices on the ACI fab​ ric. the man​ age​ ment cat​ e​ gories Fil​ ters: Fil​ ters de​ fine the rules out​ lin​ ing the Layer 2 to layer 4 fields that will be matched by a con​ tract. Hy​ per​ vi​ sor in​ te​ gra​ tion: Ex​ ten​ sion of ACI Fab​ ric con​ nec​ tiv​ ity to a vir​ tual ma​ chine man​ ager to pro​ vide the APIC with a mech​ an ​ism for vir​ tual ma​ chine vis​ ib ​il​ ity and pol​ icy en​ force​ ment. H HTML: Hy​ per​ Text Markup Lan​ guage. con​ fig​ ur​ a​ tion. Hy​ per​ vi​ sor: Soft​ ware that ab​ stracts the hard​ ware on a host ma​ chine and al​ lows the host ma​ chine to run mul​ ti​ ple vir​ tual ma​ chines. per​ for​ mance.448 Appendix Fab​ ric: The col​ lec​ tive end​ points as​ so​ ci​ ated with an ACI so​ lu​ tion (Leaf. Spine and Vir​ tual Switches plus APICs) fines net​ work man​ age​ ment tasks. ac​ count​ ing. . se​ cu​ rity. mul​ ti​ path. G GUI: Graph​ ic ​al User In​ ter​ face. and split​ ting traf​ fic across mul​ ti​ ple par​ al​ lel paths with​ out re​ quir​ ing packet re​ order​ ing.

etc. Loop​ back and VTEP ad​ dresses are in​ ter​ nally ad​ ver​ tised over IS-IS. . J JSON: JavaScript Ob​ ject No​ ta​ tion. Leafs never con​ nect to each other. Con​ nec​ tiv​ ity using an in​ band man​ age​ ment con​ fig​ ur​ a​ tion. IS-IS: Link local rout​ ing pro​ to​ col lever​ aged by the fab​ ric for in​ fra​ struc​ ture topol​ ogy. DLP. L Layer 2 Out (l2out): Layer 2 con​ nec​ tiv​ ity to an ex​ ter​ nal net​ work that ex​ ists out​ side of the ACI fab​ ric. Ser​ vice nodes op​ er​ ate be​ tween Lay​ ers 4 and Layer 7 of the OSI model. Leafs con​ nect only to hosts and spines.. Layer 3 Out (l3out): Layer 3 con​ nec​ tiv​ ity to an ex​ ter​ nal net​ work that ex​ ists out​ side of the ACI fab​ ric..e.) into the flow of traf​ fic..Appendix 449 In​ band Man​ age​ ment (INB): In​ band Man​ age​ ment. IS-IS an​ nounces the cre​ ation of tun​ nels from leaf nodes to all other nodes in fab​ ric. IDS/IPS. This uses a front panel (data plane) port of a leaf switch for ex​ ter​ nal man​ age​ ment con​ nec​ tiv​ ity for the fab​ ric and APICs. L4-L7 Ser​ vice In​ ser​ tion: The in​ ser​ tion and stitch​ ing of VLANs/Layer 3 con​ structs of vir​ tual or phys​ ic ​al ser​ vice ap​ pli​ ances (Fire​ wall. M MO: Man​ aged Ob​ ject – every con​ fig​ urable com​ po​ nent of the ACI pol​ icy model man​ aged in the MIT is called a MO. where as net​ work​ ing el​ e​ ments (i. a data en​ cap​ su​ la​ tion for​ mat that uses human read​ able text to en​ cap​ su​ late data ob​ jects in at​ tribute and value pairs. the fab​ ric) op​ er​ ate at lay​ ers 1-3). Load Bal​ ancers. La​ bels: Used for clas​ si​ fy​ ing which ob​ jects can and can​ not com​ mu​ ni​ cate with each other. Leaf: Net​ work node in fab​ ric pro​ vid​ ing host and bor​ der con​ nec​ tiv​ ity.

REST​ ful: An API that uses REST. Multi-tier Ap​ pli​ ca​ tion: Client–server ar​ chi​ tec​ ture in which pre​ sen​ ta​ tion. Out-of-Band man​ age​ ment (OOB man​ age​ ment): Ex​ ter​ nal con​ nec​ tiv​ ity using a spe​ cific out-of-band man​ age​ ment in​ ter​ face on every switch and APIC. Rep​ re​ sen​ ta​ tional State Trans​ fer (REST): a state​ less pro​ to​ col usu​ ally run over HTTP that al​ lows a client to ac​ cess a server-side or cloud-based API with​ out hav​ ing to write a local client for the host ac​ cess​ ing the API. O Ob​ ject Model: A col​ lec​ tion of ob​ jects and classes are used to ex​ am​ ine and ma​ nip​ ul​ ate the con​ fig​ ur​ a​ tion and run​ ning state of the sys​ tem that is ex​ pos​ ing that ob​ ject model. Data is usu​ ally ac​ cessed and re​ turned in ei​ ther XML or JSON for​ mat. ap​ pli​ ca​ tion logic. which is a method of man​ ag​ ing se​ cure ac​ cess to in​ fra​ struc​ ture by as​ sign​ ing roles to users. then using those roles in the process of grant​ ing or deny​ ing ac​ cess to de​ vices. The lo​ ca​ tion that the client ac​ cesses usu​ ally de​ fines the data the client is try​ ing to ac​ cess from the ser​ vice. and data​ base man​ age​ ment func​ tions re​ quire phys​ ic ​al or log​ ic ​al sep​ ar​ a​ tion and re​ quire net​ work​ ing func​ tions to com​ mu​ ni​ cate with the other tiers for ap​ pli​ ca​ tion func​ tion​ al​ ity. In ACI the ob​ ject model is rep​ re​ sented as a tree known as the dis​ trib​ uted man​ age​ ment in​ for​ ma​ tion tree (dMIT). R RBAC: Role Based Ac​ cess Con​ trol.450 Appendix Model: A model is a con​ cept which rep​ re​ sents en​ ti​ ties and the re​ la​ tion​ ships that exist be​ tween them. . or Rep​ re​ sen​ ta​ tional State Trans​ fer. P Port Chan​ nel: A Port link ag​ gre​ ga​ tion tech​ nol​ ogy that binds mul​ ti​ ple phys​ ic ​al in​ ter​ faces into a sin​ gle log​ ic ​al in​ ter​ face and pro​ vides more ag​ gre​ gate band​ width and link fail​ ure re​ cov​ ery with​ out a topol​ ogy change. ob​ jects and priv​ il​ ege lev​ els.

leaf nodes con​ nect to hosts and ex​ ter​ nal net​ works. and rep​ re​ sent each func​ tion as a node. con​ nected only to leafs in the fab​ ric and no other de​ vice types. Any ser​ vices that are re​ quired are treated as a ser​ vice graph that is in​ stan​ ti​ ated on the ACI fab​ ric from the APIC. Ser​ vice graphs iden​ tify the set of net​ work or ser​ vice func​ tions that are needed by the ap​ pli​ ca​ tion. Sub​ jects: Con​ tained by con​ tracts and cre​ ate the re​ la​ tion​ ship be​ tween fil​ ters and con​ tracts. T Ten​ ants: The log​ ic ​al con​ tainer to group all poli​ cies for ap​ pli​ ca​ tion poli​ cies. As an ex​ am​ ple. a sub​ net de​ fines the IP ad​ dress range that can be used within the bridge do​ main. A Rn is sig​ nif​ ic ​ant to the in​ di​ vid​ ual ob​ ject. but with​ out con​ text. Spine: Net​ work node in fab​ ric car​ ry​ ing ag​ gre​ gate host traf​ fic from leafs. if an Ap​ pli​ ca​ tion Pro​ file ob​ ject is cre​ ated named “com​ merce​ work​ space”.Appendix 451 Rn: Rel​ at​ ive name. the Rn would be “ap-com​ merce​ work​ space” be​ cause Ap​ pli​ ca​ tion Pro​ file rel​ at​ ive names are all pref​ aced with the let​ ters “ap-”. it’s not very use​ ful in nav​ ig ​a​ tion. Su​ per​ vi​ sor: Switch mod​ ule that pro​ vides the con​ trol plane for the 95xx switches. which then be​ comes use​ ful for nav​ ig ​a​ tion. S Ser​ vice graph: A mech​ an ​ism within ACI that au​ to​ mates redi​ rec​ tion of traf​ fic and VLAN stitch​ ing based on de​ fined pa​ ra​ me​ ters. . A Rn would need to be con​ cate​ nated with all the rel​ at​ ive names from it​ self back up to the root to make a dis​ tin​ guished name. Spine/Leaf topol​ ogy: A clos-based fab​ ric topol​ ogy in which spine nodes con​ nect to leaf nodes. See also the Dn de​ f​ in ​i​ t​ ion. a name of a spe​ cific ob​ ject within the ACI man​ age​ ment in​ for​ ma​ tion tree that is not fully qual​ if​ ied. Sub​ nets: Con​ tained by a bridge do​ main or an EPG.

Each seg​ ment rep​ re​ sents a unique Layer 2 broad​ cast do​ main. but is spread across no more or less than 2 phys​ ic ​al switches. vPC: vir​ tual Port Chan​ nel. A 24bit VXLAN seg​ ment ID (SID) or VXLAN net​ work iden​ ti​ fier (VNID) is in​ cluded in the en​ cap​ su​ la​ tion to pro​ vide up to 16 mil​ lion VXLAN seg​ ments for traf​ fic iso​ la​ tion or seg​ men​ ta​ tion. a markup lan​ guage that fo​ cuses on en​ cod​ ing data for doc​ um ​ents rather than the for​ mat​ ting of the data for those doc​ um ​ents. in which a port chan​ nel is cre​ ated for link ag​ gre​ ga​ tion.A Layer 3 name​ space iso​ la​ tion method​ ol​ ogy to allow for mul​ ti​ ple con​ texts to be de​ ployed on a sin​ gle de​ vice or in​ fra​ struc​ ture.452 Appendix V Vir​ tu​ al​ iza​ tion: Tech​ nol​ ogy used to ab​ stract hard​ ware re​ sources into vir​ tual rep​ re​ sen​ ta​ tions and al​ low​ ing soft​ ware con​ fig​ ura​ bil​ ity. . An ACI VXLAN header is used to iden​ tify the pol​ icy at​ trib​ utes if the ap​ pli​ ca​ tion end​ point within the fab​ ric. and every packet car​ ries these pol​ icy at​ trib​ utes. VXLAN: VXLAN is a Layer 2 over​ lay scheme trans​ ported across a Layer 3 net​ work. X XML: eX​ ten​ si​ ble Markup Lan​ guage. VRF: Vir​ tual Rout​ ing and For​ ward​ ing .

Appendix 453 Reference Material Top​ ics that are out​ side of the scope of this op​ er​ at​ ions guide may be doc​ um ​ented in other places.​ io/​ aci-troubleshooting-book/​ ACI White Pa​ pers http://​ www.Fab​ ric Ini​ tial​ iza​ tion http://​ www.​ cisco.​ html https://​ datacenter.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-troubleshooting-guides-list.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ white-paper-listing. This sec​ tion in​ cludes links to other help​ ful ref​ er​ ence doc​ um ​en​ ta​ tion for fur​ ther read​ ing and view​ ing.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-installation-guides-list.​ cisco.​ cisco.​ github.​ html ACI Get​ ting Started . ACI Install and Upgrade Guides http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c11-731960.​ cisco.​ html#_​ Toc405844675 ACI Trou​ bleshoot​ ing Guides http://​ www.​ html ACI De​ sign Guide http://​ www.​ cisco.​ html .​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-installation-and-configurationguides-list.

​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ presentations-listings.​ cisco.​ html ACI Toolkit http://​ datacenter.​ cisco.454 Appendix ACI Case Stud​ ies www.​ html .​ html AVS Con​ fig​ u​ ra​ tion and Scal​ a​ bil​ ity Guides http://​ www.​ io/​ acitoolkit/​ ACI Com​ pata​ bil​ ity Tool http://​ www.​ cisco.​ com/​ web/​ techdoc/​ aci/​ acimatrix/​ matrix.​ cisco.​ html ACI Part​ ners and Cus​ tomers Pre​ sen​ ta​ tions http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ solution-overview-listing.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ solution-overview-c22-732445. Pre​ sen​ ta​ tions and Train​ ings www.​ cisco.​ cisco.​ github.​ com/​ c/​ en/​ us/​ support/​ switches/​ application-virtual-switch/​ products-installation-and-configuration-guides-list.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ application-centricinfrastructure/​ customer-case-study-listing.​ html ACI So​ lu​ tions Overview http://​ www.​ html ACI Ecosys​ tem Com​ pata​ bil​ ity List http://​ www.​ html ACI Demos.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ application-centricinfrastructure/​ sales-resources-list.

​ com/​ watch?​ v=_​ iQvoC9zQ_​ A Nexus CLI to Cisco APIC Map​ ping http://​ www.​ cisco.​ html .​ cisco.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-configuration-examples-list.​ com/​ c/​ en/​ us/​ support/​ switches/​ application-virtual-switch/​ products-technical-reference-list.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c07-732033.​ html Fab​ ric Con​ nec​ tiv​ ity Video https://​ www.​ html APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide http://​ www.​ youtube.​ html APIC Com​ mand-Line In​ ter​ face User Guide http://​ www.​ com/​ datacenter/​ cobra Con​ nect​ ing ACI to Out​ side Layer 2 and 3 Net​ works http://​ www.​ readthedocs.​ org/​ en/​ latest/​ Cobra GitHub http://​ github.​ html Cobra Docs http://​ cobra.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-command-reference-list.​ cisco.Appendix 455 AVS Topolo​ gies and So​ lu​ tion Guide http://​ www.​ cisco.​ cisco.

​ cisco.456 Appendix POST​ man getpostman.​ com http://www.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.​ Sup​ ported SNMP MIB List http://​ www.​ html .