You are on page 1of 466

Operating Cisco

Application Centric
Infrastructure

Alejandra Sanchez, Andres Vega, Arvind Chari, Carly Stoughton,
Christopher Stokes, Gabriel Fontenot, Jonathan Cornell, Ken Fee,
Kevin Corbin, Lauren Malhoit, Loy Evans, Lucien Avramov,
Paul Lesiak, Steven Lym, Rafael Muller, Robert Burns

Operating Cisco
Application Centric
Infrastructure

Copyright
Op​
er​
at​
ing Cisco Ap​
pli​
ca​
tion Cen​
tric In​
fra​
struc​
ture
Ale​
jan​
dra Sanchez, An​
dres Vega, Arvind Chari, Carly Stoughton, Christo​
pher Stokes,
Gabriel Fontenot, Jonathan Cor​
nell, Ken Fee, Kevin Corbin, Lau​
ren Mal​
hoit, Loy Evans,
Lu​
cien Avramov, Paul Lesiak, Rafael Muller, Robert Burns. Steven Lym
Copy​
right © 2015 Cisco Sys​
tems, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or
agreed to in writing, software distributed under the License is distributed on an "AS IS"
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing permissions and
limitations under the License.
Cisco Press logo is a trade​
mark of Cisco Sys​
tems, Inc.
Pri​
vately pub​
lished by Cisco Sys​
tems, Inc.

Warning and Disclaimer
This book is de​
signed to pro​
vide in​
for​
ma​
tion about Cisco ACI. Every ef​
fort has been
made to make this book as com​
plete and as ac​
cu​
rate as pos​
si​
ble, but no war​
ranty or
fit​
ness is im​
plied.
The in​
for​
ma​
tion is pro​
vided on an “as is” basis. The au​
thors, and Cisco Sys​
tems, Inc.
shall have nei​
ther li​
ab
​il​
ity nor re​
spon​
si​
bil​
ity to any per​
son or en​
tity with re​
spect to any
loss or dam​
ages aris​
ing from the in​
for​
ma​
tion con​
tained in this doc​
um
​ent.
The opin​
ions ex​
pressed in this book be​
long to the au​
thor and are not nec​
es​
sar​
ily those
of Cisco Sys​
tems, Inc.

Feed​
back In​
for​
ma​
tion
Read​
ers’ feed​
back is a nat​
ural con​
tin​
ua
​t​
ion of this process. If you have any com​
ments
re​
gard​
ing how we could im​
prove the qual​
ity of this book, or oth​
er​
wise alter it to bet​
ter
suit your needs, you can contact
​ us through email at ops-booksprint@​
cisco.​
com . ​

Contents

. . . . . Prologue
........................................................................

1

. . . . . Abstract
...................................................................

3

. . . . . Authors
...................................................................

5

. . . . . Book
. . . . . Writing
. . . . . . . .Methodology
......................................................

7

. . . . . Hardware
. . . . . . . . . .and
. . . .Software
. . . . . . . . .Included
. . . . . . . . in
. . the
. . . .Book
..............................

9

. . . . . Introduction
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
. . . . . The
. . . . Story
. . . . . .of
. . ACME
. . . . . . Inc.
.................................................

13

. . . . . The
. . . . Why,
. . . . . .Who,
. . . . .What,
. . . . . .When
. . . . . .and
. . . .How
....................................

15

. . . . . Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
. . . . . APIC
. . . . . Overview
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
. . . . . Configuring
. . . . . . . . . . . .Management
. . . . . . . . . . . . Protocols
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
. . . . . Role-Based
. . . . . . . . . . . Access
. . . . . . .Control
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
. . . . . Import
. . . . . . . and
. . . . Export
. . . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

. . . . . Upgrading
. . . . . . . . . . .and
. . . Downgrading
. . . . . . . . . . . . . Firmware
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
. . . . . Section
. . . . . . . .Content
...........................................................

51

. . . . . Firmware
. . . . . . . . . .Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
. . . . . Upgrading
. . . . . . . . . . .and
. . . Downgrading
. . . . . . . . . . . . . Considerations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
. . . . . Upgrading
. . . . . . . . . . .the
. . . Fabric
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

. . . . . Fabric
. . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
. . . . . Section
. . . . . . . .Content
...........................................................

71

. . . . . Understanding
. . . . . . . . . . . . . . .Fabric
. . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
. . . . . Adding
. . . . . . . New
. . . . .Devices
. . . . . . . .to
. .the
. . . .Fabric
.........................................

81

. . . . . Server
. . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
. . . . . Virtual
. . . . . . . Machine
. . . . . . . . Networking
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
. . . . . Deploying
. . . . . . . . . . the
. . . .Application
. . . . . . . . . . .Virtual
. . . . . . Switch
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
. . . . . External
. . . . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
. . . . . Application
. . . . . . . . . . . Migration
. . . . . . . . . .Use
. . . .Case
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

. . . . . Tenants
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
. . . . . ACI
. . . . Tenancy
. . . . . . . . .Models
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
. . . . . Application
. . . . . . . . . . . Profile
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
. . . . . Endpoint
. . . . . . . . . Group
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
. . . . . Endpoint
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

. . . . . Private
. . . . . . . Network
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
. . . . . Bridge
. . . . . . .Domain
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
. . . . . Tenant
. . . . . . . Networking
. . . . . . . . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

. . . . . Working
. . . . . . . . .with
. . . . Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
. . . . . Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
. . . . . Filters
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
. . . . . Taboo
. . . . . . .Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
. . . . . Inter-Tenant
. . . . . . . . . . . . .Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
. . . . . Contracts
. . . . . . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

. . . . . Layer
. . . . . .4. .to
. .Layer
. . . . . .7. Services
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .249
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
. . . . . Understanding
. . . . . . . . . . . . . . .Layer
. . . . . 4. .to
. . Layer
. . . . . .7. Integration
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
. . . . . Services
. . . . . . . . .Deployment
. . . . . . . . . . . Guide
. . . . . . Reference
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
. . . . . Service
. . . . . . . .Graph
. . . . . .Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
. . . . . ASAv
. . . . . Sample
. . . . . . . .Configuration
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

. . . . . Health
. . . . . . .Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .269
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
. . . . . Understanding
. . . . . . . . . . . . . . .Health
. . . . . . Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
. . . . . Understanding
. . . . . . . . . . . . . . .Faults
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
. . . . . How
. . . . . Health
. . . . . . .Scores
. . . . . . Are
. . . . Calculated
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
. . . . . Health
. . . . . . .Score
. . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

. . . . . Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . -. .Tenant
. . . . . . .and
. . . .Fabric
. . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . -. .Infrastructure
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . Use
. . . . Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . Tools
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . Use
. . . . Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

. . . . . Scripting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .349
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
. . . . . Leveraging
. . . . . . . . . . .Network
. . . . . . . . Programmability
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
. . . . . ACI
. . . . and
. . . . Scripting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
. . . . . API
. . . .Inspector
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
. . . . . Development
. . . . . . . . . . . . . Techniques
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

. . . . . STman
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
. . . . . Cobra
. . . . . . SDK
. . . . .and
. . . .Arya
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
. . . . . ACI
. . . . Toolkit
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
. . . . . GitHub
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

. . . . . Hardware
. . . . . . . . . .Expansion
. . . . . . . . . .and
. . . .Replacement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .389
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
. . . . . Expanding
. . . . . . . . . . .and
. . . Shrinking
. . . . . . . . . .the
. . . Fabric
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
. . . . . Hardware
. . . . . . . . . .Diagnostics
. . . . . . . . . . . and
. . . .Replacement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

. . . . . Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .413
. . . . . Classes
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
. . . . . Package
. . . . . . . . Decoder
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
. . . . . Acronyms
. . . . . . . . . . and
. . . .Definitions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
. . . . . Reference
. . . . . . . . . . Material
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
....

1 Prologue .

.

method​ olo​ gies. This book ex​ am​ ines some of the com​ mon op​ er​ a​ tional ac​ tiv​ i​ ties that IT teams use to pro​ vide con​ tin​ ued in​ fra​ struc​ ture op​ er​ a​ tions and gives the reader ex​ po​ sure to the tools. and processes that can be em​ ployed to sup​ port day 1+ op​ er​ a​ tions within an ACI-based fab​ ric.Prologue 3 Abstract Cisco's Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) pro​ vides pow​ er​ ful new ways to dy​ nam​ i​ cally man​ age in​ fra​ struc​ ture in the mod​ ern world of IT au​ toma​ tion and De​ vOps. but being able to ef​ fec​ tively op​ er​ ate the in​ fra​ struc​ ture be​ yond the Day Zero build ac​ tiv​ i​ ties is cru​ cial to long term ef​ fec​ tive​ ness and ef​ fi​ ciency. or​ ga​ ni​ za​ tions will need to un​ der​ stand how to in​ cor​ po​ rate ACI into their daily op​ er​ a​ tions. To ef​ fec​ tively har​ ness the power of ACI. . Hav​ ing the tools to change how in​ fra​ struc​ ture is built is one thing.

.

Tech​ ni​ cal Writer.So​ lu​ tions Ar​ chi​ tect.Cus​ tomer Sup​ port En​ gi​ neer. and Sales.Tech​ ni​ cal Leader.Tech​ ni​ cal Mar​ ket​ ing En​ gi​ neer. Sales Lu​ cien Avramov .Sys​ tems En​ gi​ neer. Ad​ vanced Ser​ vices Carly Stoughton . IT.Con​ sult​ ing Sys​ tem En​ gi​ neer. INSBU Loy Evans .So​ lu​ tions Ar​ chi​ tect. Authors Ale​ jan​ dra Sanchez .Cus​ tomer Sup​ port En​ gi​ neer. Tech​ ni​ cal Ser​ vices An​ dres Vega . Ad​ vanced Ser​ vices Jonathan Cor​ nell . Prod​ uct Mar​ ket​ ing. INSBU Christo​ pher Stokes .Cisco IT Net​ work Con​ sult​ ing En​ gi​ neer Gabriel Fontenot . INSBU Paul Lesiak . So​ lu​ tion Val​ id ​a​ tion Ser​ vices Robert Burns .Tech​ ni​ cal Leader.Prologue 5 Authors This book rep​ re​ sents a joint in​ tense col​ lab​ or​ a​ tive ef​ fort over the course of a week.Con​ sult​ ing Ar​ chi​ tect. Busi​ ness Tech​ nol​ ogy Ar​ chi​ tects Kevin Corbin . Tech​ ni​ cal Ser​ vices.Tech​ ni​ cal Mar​ ket​ ing En​ gi​ neer. Sys​ tems En​ gi​ neer​ ing Lau​ ren Mal​ hoit .Net​ work Con​ sult​ ing En​ gi​ neer. Tech​ ni​ cal Ser​ vices Arvind Chari . Tech​ ni​ cal Ser​ vices Steven Lym .Mar​ ket​ ing Con​ sul​ tant. with par​ tic​ ip ​a​ tion of a num​ ber of Cisco func​ tional or​ ga​ ni​ za​ tions in​ clud​ ing Cisco Ad​ vanced Ser​ vices. Ad​ vanced Ser​ vices Rafael Muller . INSBU . So​ lu​ tion Val​ id ​a​ tion Test​ ing.Tech​ ni​ cal So​ lu​ tions Ar​ chi​ tect. Sales Ken Fee .

6 Prologue Book Sprint Facilitation Bar​ bara Rühling Faith Bosworth Illustrations Hen​ rik van Leuwen Book Production Julien Taquet Clean Up Raewyn Whyte .

the com​ plete book was writ​ ten in only five days.Prologue 7 Book Writing Methodology The Book Sprint (booksprints. By lever​ ag​ ing the input of many ex​ perts. The Book Sprint method​ ol​ ogy is an in​ no​ v​ at​ ive new style of co​ op​ er​ at​ ive and col​ lab​ o​ ra​ tive book pro​ duc​ tion. Book Sprints are strongly fa​ cil​ it​ ated and lever​ age team-ori​ ented in​ spi​ ra​ tion and mo​ ti​ va​ tion to rapidly de​ liver large amounts of well-au​ thored and re​ viewed con​ tent. . and in​ cor​ po​ rate it into a com​ plete nar​ ra​ tive in a short amount of time.​ net) method​ ol​ ogy was used for writ​ ing this book. but in​ volved hun​ dreds of au​ thor​ ing per​ son-hours. and in​ cluded thou​ sands of ex​ pe​ ri​ enced en​ gi​ neer​ ing hours. al​ low​ ing for ex​ tremely high qual​ ity in a very short pro​ duc​ tion time pe​ riod.

.

9396TX. and vir​ tu​ al​ iza​ tion. including Cisco Nexus 9508 and 9336PQ • ACI Leaf switches.1 soft​ ware re​ lease. . se​ cu​ rity. and 93128TX • Cisco Application Virtual Switch (AVS) • Cisco UCS B and C series servers • Several models of switches and routers. the fol​ low​ ing hard​ ware de​ vices were used: • Cisco Application Policy Infrastructure Controller (APIC) • ACI Spine switches. including Cisco Nexus 9396PX. ap​ pli​ ca​ tion. including Cisco Nexus 5000 switches and Cisco Integrated Services Routers (ISR) • A variety of hypervisors. For the pur​ pose of writ​ ing this book. and VMware • IXIA IxVM product family traffic generators vSphere This book was writ​ ten based on the Cisco ACI 1.Prologue 9 Hardware and Software Included in the Book The Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) com​ bines hard​ ware. soft​ ware. including KVM. Microsoft Hyper-V. and ASIC in​ no​ va​ tions into an in​ te​ grated sys​ tems ap​ proach and pro​ vides a com​ mon man​ age​ ment frame​ work for net​ work.

.

11 Introduction .

.

ACME Inc. in​ clud​ ing rocket-pow​ ered roller skates. caused cus​ tomer-im​ pact​ ing out​ ages. but have re​ cently de​ cided to pur​ sue a more di​ rect-to-con​ sumer busi​ ness model due to in​ tense pres​ sure from new com​ peti​ tors who have dom​ in ​ated the on​ line sales chan​ nels. the in​ fra​ struc​ ture com​ po​ nents need to be ca​ pa​ ble of map​ ping to these new par​ a-​ digms in a way that is not pos​ si​ ble using tra​ di​ tional con​ cepts. In an ef​ fort to be more com​ pet​ it​ ive.Introduction 13 The Story of ACME Inc. can be checked into ver​ sion con​ trol. and the new ap​ pli​ ca​ tion plat​ form is to be de​ vel​ oped in this man​ ner. These prod​ uct groups op​ er​ ate as sep​ ar​ ate busi​ ness groups within the com​ pany. . To sup​ port this. and have pre​ vi​ ously main​ tained sep​ ar​ ate in​ fra​ struc​ ture and ap​ pli​ ca​ tions. They have largely fo​ cused on re​ tail routes to mar​ ket. For this rea​ son. The ap​ pli​ ca​ tion de​ vel​ op​ ers have been look​ ing at new ap​ pli​ ca​ tion de​ vel​ op​ ment trends such as Con​ tin​ uo ​us De​ liv​ ery and Con​ tin​ uo ​us In​ te​ gra​ tion. and taken longer to ac​ com​ plish than the busi​ ness lead​ ers would have liked. has de​ cided to change by cre​ at​ ing an en​ vi​ ron​ ment where in​ fra​ struc​ ture ar​ ti​ facts are treated as part of the ap​ pli​ ca​ tion. can be tested along​ side the ac​ tual ap​ pli​ ca​ tion. ACME has un​ der​ taken a pro​ ject to build a mo​ bile ap​ pli​ ca​ tion plat​ form to sup​ port or​ der​ ing and lo​ gis​ tics for prod​ uct de​ liv​ ery to their cus​ tomers for their en​ tire port​ fo​ lio. This has led to sev​ eral sit​ u​ at​ ions where ap​ pli​ ca​ tion de​ ploy​ ments have meant long week​ end hours for all of the teams. jetpro​ pelled uni​ cy​ cles. ACME busi​ ness units have lever​ aged third party soft​ ware com​ pa​ nies and com​ mer​ cially avail​ able soft​ ware to meet their IT de​ mands. and var​ io ​us ex​ plo​ sive ma​ te​ ri​ als. they have lever​ aged a tra​ di​ tional in​ fra​ struc​ ture and soft​ ware model that does not allow them to keep up with the chang​ ing re​ quire​ ments. Tra​ di​ tion​ ally. and dis​ tri​ bu ​t​ ion of a di​ verse prod​ uct port​ fo​ lio. but would like to cre​ ate a more in​ ti​ mate re​ la​ tion​ ship with their con​ sumers and be able to take feed​ back on the plat​ form di​ rectly from those users. and can con​ tin​ ua ​lly im​ prove. ACME Inc. sales. Where they have used cus​ tom soft​ ware in the past. is a multi​ na​ tional cor​ po​ ra​ tion that spe​ cial​ izes in man​ uf​ ac​ tur​ ing. while in​ cor​ po​ rat​ ing an on​ go​ ing im​ prove​ ment cycle so they can react to chang​ ing mar​ ket dy​ nam​ ics in a more nim​ ble fash​ ion. and there​ fore ACME is look​ ing for a new ap​ proach to both ap​ pli​ ca​ tion and in​ fra​ struc​ ture life​ cy​ cle man​ age​ ment. One of the largest chal​ lenges ACME has his​ tor​ ic ​ally faced is that op​ er​ at​ ions and in​ fra​ struc​ ture has been an af​ ter​ thought to prod​ uct de​ vel​ op​ ment.

but con​ trol can be a bar​ rier to a pure con​ sump​ tion model. The abil​ ity to ab​ stract all phys​ ic ​al and vir​ tual in​ fra​ struc​ ture con​ fig​ ur​ a​ tion into a sin​ gle con​ fig​ ur​ a​ tion that is con​ sis​ tent across dev. this runs con​ trary to how most op​ er​ at​ ions groups exist in the re​ la​ tion​ ship be​ tween busi​ ness and tech​ nol​ ogy. Most IT op​ er​ at​ ions groups in​ vest a lot of time in the tools needed to de​ liver ser​ vices today and there is an or​ ganic re​ sis​ tance to re-in​ vest​ ing. Work​ ers in the IT in​ dus​ try need to adapt to keep up with the rapid change of the busi​ ness. Every com​ pany is a tech​ nol​ ogy com​ pany. and doubt. thus al​ low​ ing IT to op​ er​ ate and man​ age at the speed of busi​ ness. Un​ less com​ pa​ nies make in​ vest​ ments in tech​ nolo​ gies that allow for con​ sump​ tion of au​ to​ mated com​ po​ nents. you'll get left be​ hind. as well as portable across the var​ io ​us data cen​ ter lo​ ca​ tions cur​ rently main​ tained by ACME. and prod en​ vi​ ron​ ments. the CEO of Cisco Sys​ tems. After an​ al​ yz​ ing cur​ rent of​ fers from var​ io ​us tech​ nol​ ogy ven​ dors. such as Ama​ zon Web Ser​ vices (AWS) and Open​ stack. this is the true story of every com​ pany. and few peo​ ple would re​ ally choose to work for that type of com​ pany.'s busi​ ness own​ ers need. The thought is." As ev​ id ​enced by the suc​ cess of cloud plat​ forms. test. At an ex​ ec​ ut​ ive brief​ ing. ACI has been built from the ground up to change the sub​ struc​ ture used to build net​ work de​ vices and pro​ to​ cols.14 Introduction While ACME is in​ tensely fo​ cused on de​ liv​ er​ ing the new ap​ pli​ ca​ tion plat​ form in a timely man​ ner. ACME Inc. John Cham​ bers. told ACME: "The world is chang​ ing. How​ ever. This is the type of con​ sump​ tion that ACME Inc. con​ sump​ tion mod​ els of tech​ nol​ ogy de​ liv​ ery have the abil​ ity to adapt tech​ nol​ ogy more quickly to rapid busi​ ness re​ quire​ ments changes. How​ ever. ACME is also in​ ter​ ested in cre​ at​ ing a foun​ da​ tion on which it can grow to de​ liver a com​ mon pool of in​ fra​ struc​ ture that is shared across all busi​ ness groups and op​ er​ ated in a multi-ten​ ant fash​ ion to in​ crease ef​ fi​ ciency. This is where the ful​ crum will tilt in the favor of IT and in​ fra​ struc​ ture being more dy​ namic. and if you don't adapt. with a change of this na​ ture comes fear. is highly de​ sir​ able. the only other way to scale is by break​ ing the human level com​ po​ nent. "Why fix what is al​ ready work​ ing?" . In​ no​ va​ tion at this level will pro​ vide more op​ por​ tu​ ni​ ties for ex​ pand​ ing the tools with which users in​ ter​ act. This book will at​ tempt to bring some level of com​ fort and fa​ mil​ iar​ ity with op​ er​ at​ ions ac​ tiv​ i-​ ties within an ACI fab​ ric. Con​ trol of op​ er​ at​ ions is what op​ er​ at​ ions groups are fo​ cused on. un​ cer​ tainty. and just im​ por​ tant this is the story of the em​ ploy​ ees of those com​ pa​ nies. se​ lected Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI). is a fic​ ti​ tious com​ pany. While ACME Inc.

the fact that these dif​ fer​ ent or​ ga​ ni​ za​ tions exist should . au​ toma​ tion can be a scary propo​ si​ tion for some of the key stake​ hold​ ers. these groups can have dis​ tinct or​ ga​ ni​ za​ tional bound​ aries. but keep in mind that some of these char​ ac​ ter​ is​ tics might be com​ bined. al​ low​ ing them the free​ dom to in​ no​ vate and add value. Also. but rec​ og​ nizes that this ini​ tia​ tive. To achieve the de​ sired re​ sult.Introduction 15 The Why. au​ toma​ tion is about mak​ ing work more en​ joy​ able for all team mem​ bers. the key suc​ cess cri​ te​ ria is to stream​ line processes and pro​ ce​ dures re​ lated to the de​ ploy​ ment of in​ fra​ struc​ ture re​ quired to sup​ port the ap​ pli​ ca​ tion ini​ tia​ tives. Au​ toma​ tion adds speed to repet​ it​ ive tasks and elim​ in ​ates er​ rors or missed steps. In the case of ACME Inc. but more likely the bound​ aries are blurred to some de​ gree. look​ ing at why an op​ er​ at​ ional prac​ tice is done a spe​ cific way can help with fram​ ing the tools and processes that are em​ ployed. At the macro level. ACME is look​ ing to sim​ plify how it op​ er​ ates in​ fra​ struc​ ture. Ini​ tially. Listed below are some char​ ac​ ter​ is​ tics of these groups. repet​ it​ ive tasks. while re​ mov​ ing mun​ dane. and each has a spe​ cific el​ e​ ment of the in​ fra​ struc​ ture in which they have spe​ cific ex​ per​ tise and about which they care most. What. but more tac​ ti​ cal and pointin-time-rel​ e​ vant. and How they go about their tasks. In any IT or​ ga​ ni​ za​ tion. a high de​ gree of au​ toma​ tion is re​ quired. this ap​ pli​ ca​ tion. When and How Within ACI. Quite the op​ po​ site. and this in​ fra​ struc​ ture are new to ACME Inc. and Where they go to per​ form these op​ er​ at​ ions. are also con​ sid​ er​ at​ ions. Who. Why "Why" is the most im​ por​ tant as​ pect of what should be con​ sid​ ered in op​ er​ at​ ional​ iz​ ing an ACI fab​ ric. ACME Inc. This sec​ tion dis​ cusses the rel​ e​ vant as​ pects of these monikers as it re​ lates to ACI fab​ ric op​ er​ at​ ions and how a com​ pany such as ACME Inc. When dif​ fer​ ent groups per​ form reg​ ul​ ar op​ er​ at​ ions. as it could be looked at as a threat to their own job. Who As with most or​ ga​ ni​ za​ tions. ACME must ad​ dress fun​ da​ men​ tal ques​ tions in​ clud​ ing Who man​ ages What. tra​ di​ tion​ ally had dif​ fer​ ent types of stake​ hold​ ers in​ volved in mak​ ing any IT ini​ tia​ tive suc​ cess​ ful. can di​ vide the work​ load. Look​ ing at why an au​ to​ mated fab​ ric is ben​ e​ fi​ cial to an or​ ga​ ni​ za​ tion is im​ por​ tant for set​ ting ex​ pec​ ta​ tions of re​ turn on in​ vest​ ment.

all while main​ tain​ ing high lev​ els of avail​ abil​ ity. The team typ​ ic ​ally has some spe​ cific re​ quire​ ments around QoS. With ACI.16 Introduction not be ev​ id ​ent to the end-user. multi-pathing. and the team is mak​ ing key met​ rics avail​ able on de​ mand in a self-ser​ vice ca​ pac​ ity. The Ap​ pli​ ca​ tion part of the team con​ tains ap​ pli​ ca​ tion own​ ers and sub​ ject mat​ ter ex​ perts that en​ sure other busi​ ness units are able to do their jobs by uti​ liz​ ing the busi​ ness ap​ pli​ ca​ tions avail​ able. while al​ low​ ing other groups to lever​ age their spe​ cific ex​ per​ tise to ma​ nip​ ul​ ate se​ cu​ rity and ap​ pli​ ca​ tion level poli​ cies. and so on. In​ stead. per​ for​ mance. . ACME's Net​ work Team is pri​ mar​ ily fo​ cused on cre​ at​ ing and man​ ag​ ing net​ work​ ing con​ structs to for​ ward pack​ ets at Layer 2 (MAC/switch​ ing) and Layer 3 (IP rout​ ing). The team is also very in​ ter​ ested in being able to see how the stor​ age ac​ cess is per​ form​ ing and would like to be no​ ti​ fied in the event of con​ tention. The team is also in​ ter​ ested in al​ low​ ing more trans​ parency in the per​ for​ mance of the net​ work for​ ward​ ing. the en​ tire or​ ga​ ni​ za​ tion should be seen as one team with a com​ mon goal of de​ liv​ er​ ing value to their or​ ga​ ni​ za​ tion. The ca​ pa​ bil​ it​ ies pro​ vided by the ACI fab​ ric allow them to con​ fi​ dently de​ ploy newer IP-based stor​ age and clus​ ter​ ing tech​ nolo​ gies. ACME's Stor​ age Team is pri​ mar​ ily fo​ cused on de​ liv​ ery of data stor​ age re​ sources to the or​ ga​ ni​ za​ tion. ACI will pro​ vide the stor​ age team with the vis​ ib ​il​ ity they will re​ quire. the team is the most in​ ter​ ested in de​ cou​ pling over​ loaded net​ work con​ structs and re​ turn​ ing to the spe​ cific net​ work prob​ lems that the team was in​ tended to solve. Both parts of this team will need to work closely with the other teams in this sec​ tion to en​ able the best de​ sign. These ca​ pa​ bil​ it​ ies are pri​ mar​ ily dis​ cussed in the mon​ it​ or​ ing sec​ tions. how to tie Layer 2 to Layer 3. as well as mak​ ing sure that sen​ si​ tive data is se​ cure. ACME's De​ vel​ op​ ment and Ap​ pli​ ca​ tion Team is fo​ cused on the soft​ ware and ap​ pli​ ca​ tions the com​ pany uses in​ ter​ nally and the soft​ ware that it de​ liv​ ers to its cus​ tomers. and how to trou​ bleshoot net​ work for​ ward​ ing as​ pects in the fab​ ric. The team is chal​ lenged with jug​ gling ap​ pli​ ca​ tion re​ quire​ ments. The stor​ age team is con​ cerned with pro​ tect​ ing the data in terms of avail​ abil​ ity. the team had to worry about de​ liv​ er​ ing a stor​ age fab​ ric in ad​ di​ tion to man​ ag​ ing stor​ age de​ vices them​ selves. man​ ag​ ing SLA. and as​ sist​ ing in the en​ force​ ment of in​ for​ ma​ tion se​ cu​ rity. The stor​ age team has been very suc​ cess​ ful in main​ tain​ ing very tight SLAs and has tra​ di​ tion​ ally man​ aged sep​ ar​ ate in​ fra​ struc​ ture for stor​ age ac​ cess. His​ tor​ ic ​ally. and avail​ abil​ ity of ap​ pli​ ca​ tions for the end users. The De​ vel​ op​ ment part of the team will be writ​ ing the mo​ bile ap​ pli​ ca​ tion soft​ ware plat​ form. What the team needs to know is how to con​ fig​ ure the net​ work​ ing con​ structs. how to ver​ ify for​ ward​ ing.

you have some tra​ di​ tional tools. the In​ for​ ma​ tion Se​ cu​ rity Team is able to pro​ vide input ear​ lier in the process and avoid re-do​ ing work be​ cause of se​ cu​ rity or com​ pli​ ance is​ sues. and Layer 4 to Layer 7 ser​ vices. the ap​ pli​ ca​ tion de​ vel​ op​ ers are in​ creas​ ingly in​ ter​ ested in lever​ ag​ ing Linux con​ tainer tech​ nolo​ gies to allow for even greater ap​ pli​ ca​ tion porta​ bil​ ity. mon​ it​ or​ ing. In a tra​ di​ tional net​ work. How a tool is se​ lected in ACI will often be a prod​ uct of what is being done and the as​ pects of how the tool is used. is wrap​ ping up a major ini​ tia​ tive to vir​ tu​ al​ ize the server farms that it is re​ spon​ si​ ble for main​ tain​ ing. API in​ te​ gra​ tion. What The as​ pect of "what" can be looked at in many dif​ fer​ ent ways. or some com​ bi​ na​ tion of these. This is timely as the ap​ pli​ ca​ tion roll​ out will have both vir​ tu​ al​ ized and non-vir​ tu​ al​ ized work​ loads. These ca​ pa​ bil​ it​ ies are dis​ cussed fur​ ther in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter. but the fab​ ric man​ age​ ment is rooted in an ab​ stracted ob​ ject model that pro​ vides a more flex​ ib ​le base. Role Based Ac​ cess Con​ trol (RBAC). al​ low​ ing the team to pub​ lish end​ point groups to vir​ tu​ al​ iza​ tion clus​ ters from a cen​ tral​ ized place across mul​ ti​ ple hy​ per​ vi​ sors. Ad​ di​ tion​ ally. but the main con​ cept in the con​ text of this book is what tools are used to man​ age op​ er​ at​ ions of an ACI fab​ ric.Introduction 17 The Com​ pute and Vir​ tu​ al​ iza​ tion Team at ACME Inc. For ex​ am​ ple. has tra​ di​ tion​ ally been en​ gaged late in an ap​ pli​ ca​ tion de​ ploy​ ment process. With the cur​ rent pro​ ject. the op​ er​ at​ or of the fab​ ric can choose from mul​ ti​ ple modes of man​ age​ ment. script​ ing. CLI. In ACI there are some el​ e​ ments of the tra​ di​ tional tools. The In​ for​ ma​ tion Se​ cu​ rity Team at ACME Inc. the new ap​ pli​ ca​ tion will be stor​ ing sen​ si​ tive cus​ tomer in​ for​ ma​ tion. Due to the sen​ si​ tiv​ ity of this in​ for​ ma​ tion and the se​ cu​ rity as​ pects of the ACI fab​ ric. and these tools in​ te​ grate into man​ age​ ment plat​ forms and con​ fig​ ur​ a​ tion and man​ age​ ment processes. if an op​ er​ at​ ions staff is try​ ing to gather a bunch of in​ for​ ma​ tion across a num​ ber of in​ ter​ faces and switches or is man​ ag​ ing the con​ fig​ ur​ a​ - . to man​ age net​ work op​ er​ at​ ions. in​ clud​ ing credit card num​ bers. The In​ for​ ma​ tion Se​ cu​ rity Team is in​ ter​ ested in the op​ er​ at​ ional as​ pects of the ACI se​ cu​ rity model as it re​ lates to the fol​ low​ ing ca​ pa​ bil​ it​ ies: ten​ ancy. The Com​ pute and Vir​ tu​ al​ iza​ tion teams are in​ ter​ ested in ACI for its abil​ ity to pro​ vide com​ mon ac​ cess to phys​ ic ​al and vir​ tual servers. and has been re​ spon​ si​ ble for per​ form​ ing vul​ ner​ ab ​il​ ity as​ sess​ ment and data clas​ si​ fi​ ca​ tion ef​ forts. such as GUI. pro​ gram​ ming. The team also re​ cently em​ ployed new con​ fig​ ur​ a​ tion man​ age​ ment tools to ac​ count for new work​ loads that fell out​ side of the vir​ tu​ al​ iza​ tion ef​ fort to get sim​ il​ ar agility for bare metal servers that the team gained from its vir​ tu​ al​ iza​ tion ef​ forts. With this base. such as CLI and SNMP.

This is a key dif​ fer​ ence be​ tween ACI and tra​ di​ tional processes that were very se​ ri​ al in na​ ture. The ACI pol​ icy model has been de​ signed to re​ duce the over​ all size of a fault do​ main and pro​ vide a mech​ an ​ism for in​ cre​ men​ tal change. Through​ out this book we will high​ light the meth​ ods and pro​ ce​ dures to proac​ tively and re​ ac​ tively man​ age the fab​ ric. an im​ por​ tant con​ sid​ er​ at​ ion is change con​ trol. FCAPS. An eval​ ua ​t​ ion of cur​ rent change con​ trol and con​ tin​ uo ​us in​ te​ gra​ tion/de​ liv​ ery strate​ gies is war​ ranted as op​ er​ at​ ional pro​ ce​ dures evolve. most or​ ga​ ni​ za​ tions are im​ ple​ ment​ ing some kind of struc​ tured changecon​ trol method​ ol​ ogy to mit​ ig ​ate busi​ ness risk and en​ hance sys​ tem avail​ abil​ ity. How "How" an​ swers the fol​ low​ ing basic ques​ tions: • How does a networking person go about configuring the network forwarding? • How does the compute team get information from the infrastructure to make • How do the application team track performance and usage metrics? • How does a storage team track the access to storage subsystems and ensure optimal workload placement decisions? that it is performant? When "how" in​ volves mak​ ing a change to the con​ fig​ ur​ a​ tion of an en​ vi​ ron​ ment. re​ sult​ ing in a longer de​ ploy​ ment time for ap​ pli​ ca​ tions and a higher mean-time to res​ ol​ u​ tion when is​ sues arise. whereas sim​ ple dash​ board mon​ it​ or​ ing might be more suited to a GUI. There are a num​ ber of change/IT man​ age​ ment prin​ ci​ ples (Cisco Life​ cy​ cle ser​ vices. As a base​ line.Introduction 18 tion of many dif​ fer​ ent ob​ jects at once. script​ ing might be more ef​ fi​ cient. We will also dis​ cuss the model and which ob​ jects af​ fect the ten​ ants and the fab​ ric as a whole. There are mech​ an ​isms for backup and re​ store that will be dis​ cussed in fol​ low-on chap​ ters. When "When" refers to when the teams listed above are in​ volved in the plan​ ning. The col​ lab​ o​ ra​ tive na​ ture of ACI al​ lows for a high de​ gree of par​ al​ leliza​ tion of work flow. . It is a good idea to in​ volve the dif​ fer​ ent teams early when build​ ing poli​ cies and processes for how the fab​ ric is im​ ple​ mented and then man​ aged. Change con​ trol is a fact of life in the mis​ sioncrit​ ic ​al en​ vi​ ron​ ments that ACI has been de​ signed to sup​ port.

as​ sessed. in​ fra​ struc​ ture) and are per​ formed to re​ store ser​ vice to af​ fected sys​ tems. The multi-ten​ ant and role-based ac​ cess con​ trol fea​ tures in​ her​ ent to the ACI so​ lu​ tion allow the iso​ la​ tion or draw​ ing of a very clean box around the scope and im​ pact of the changes that can be made. Ul​ ti​ mately each change must be eval​ ua ​ted pri​ mar​ ily in terms of both its risk and value to the busi​ ness. soft​ ware. mon​ it​ or​ ing. which starts with the cre​ ation of a re​ quest for change which is then re​ viewed. A way to en​ able a low-over​ head change man​ age​ ment process is to re​ duce the risk of each change and in​ crease its value. and en​ sur​ ing that de​ liv​ ery teams are work​ ing on the most valu​ able thing they could be at any given time. and then (as​ sum​ ing it is au​ tho​ rized) planned and im​ ple​ mented. based on feed​ back from users. Train​ ing op​ er​ a​ tions teams on norms (a stated goal of this book) is also key. see the RBAC sec​ tion of this book. In an ACI en​ vi​ ron​ ment a nor​ mal change could apply to any​ thing within the fol​ low​ ing com​ po​ nents: • Fabric Policies (fabric internal and access will be discussed in detail later) • Configuration objects in the Common tenant that are shared with all other tenants (things that affect the entire fabric) Private Networks . Con​ tin​ uo ​us de​ liv​ ery does ex​ actly this by en​ sur​ ing that re​ leases are per​ formed reg​ ul​ arly from early on in the de​ liv​ ery process. there are three fun​ da​ men​ tal kinds of changes: • Emergency changes • Normal • Standard Emer​ gency changes are by de​ f​ i​ n​ i​ tion a re​ sponse to some kind of tech​ ni​ cal out​ age (hard​ ware. and pro​ vi​ sion​ ing. In the In​ for​ ma​ tion Man​ age​ ment Sys​ tems world. For more de​ tails. Ap​ ply​ ing change man​ age​ ment prin​ ci​ ples based on tech​ nol​ ogy from five years ago would not en​ able the rapid de​ ploy​ ment of tech​ nol​ ogy that ACI can de​ liver. A com​ mon sense ap​ proach to change man​ age​ ment and con​ tin​ uo ​us in​ te​ gra​ tion should be a premise that is dis​ cussed early in the de​ sign and im​ ple​ men​ ta​ tion cycle be​ fore hand​ ing the fab​ ric to the op​ er​ a-​ tions teams for day-to-day main​ te​ nance. Nor​ mal changes are those that go through the reg​ ul​ ar change man​ age​ ment process. and then ei​ ther au​ tho​ rized or re​ jected.Introduction 19 and ITIL) that are good guides from which to start.

the cri​ te​ ria for a change to be con​ sid​ ered "stan​ dard". APIC main​ tains an audit log for all con​ fig​ ur​ a​ tion changes to the sys​ tem. who is al​ lowed to ap​ prove them. This en​ ables the change to be re​ verted quickly. and the process for man​ ag​ ing them. they must still be recorded and ap​ proved. Im​ me​ di​ ate ac​ tion should be to check the audit log as it will tell who made what change and when.Introduction 20 Bridge Domains Subnets • Virtual Machine Manager (VMM) integrations • Layer 4 to Layer 7 devices Device packages Creation of logical devices Creation of concrete devices • Layer 2 or Layer 3 external configuration • Attachable Entity Profile (AEP) creation • Server or external network attachment • Changes to currently deployed contracts and filters that would materially change the way traffic flows Stan​ dard changes are low-risk changes that are pre-au​ tho​ rized. The abil​ ity to audit changes that are hap​ pen​ ing to the en​ vi​ ron​ ment is a re​ quire​ ment for ACME Inc. In the ACI en​ vi​ ron​ ment some ex​ am​ ples of "stan​ dard" changes could be: • Tenant creation • Application profile creation • Endpoint group (EPG) creation • Contracts scoped at a tenant level • Layer 4 to Layer 7 service graphs • Domain associations for EPGs The items men​ tioned above are not in​ tended to be all-in​ clu​ sive. Each or​ ga​ ni​ za​ tion will de​ cide the kind of stan​ dard changes that they allow. As with nor​ mal changes. cor​ re​ lat​ ing this to any faults that re​ sult from said change. . This is a key trou​ bleshoot​ ing tool for "when some​ thing mag​ ic ​ally stops work​ ing". but are rep​ re​ sen​ ta​ tive of com​ mon tasks per​ formed day-to-day and week-to-week.

ACI en​ ables ACME Inc. ACI addresses IT requirements from across the organization . The re​ main​ der of this book an​ swers these ques​ tions. the most im​ por​ tant thing is for these groups to work in​ sieme (to​ gether). and ul​ ti​ mately the busi​ ness. to pro​ vide the great​ est value to the cus​ tomer. How​ ever. and this book il​ lus​ trates how the stake​ hold​ ers can work to​ gether in a more col​ lab​ o​ ra​ tive man​ ner than they have in the past. The book is laid out in a spe​ cific order. While or​ ga​ ni​ za​ tional struc​ tures might be siloed into these teams. user. While some script​ ing op​ por​ tu​ ni​ ties are called out through​ out the book. there is a more in-depth sec​ tion at the end that ex​ plains how to use the ACI API to au​ to​ mate most op​ er​ a​ tional tasks. pro​ vid​ ing you with a frame​ work of how to take the con​ cepts and pro​ ce​ dures and apply them to sim​ i​ lar ini​ tia​ tives within your or​ ga​ ni​ za​ tions. to com​ plete these tasks in par​ al​ lel with the var​ i​ ous stake​ hold​ ers who are high​ lighted through​ out.Introduction 21 A more in-depth dis​ cus​ sion of con​ tin​ u​ ous de​ liv​ ery in the con​ text of in​ fra​ struc​ ture man​ age​ ment is out​ side of the scope of this book.

.

23 Management .

.

Management Section Content • APIC Overview • Configuring Management Protocols Cisco Discovery Protocol Link Layer Discovery Protocol Time Synchronization and NTP Out-of-Band Management NTP In-Band Management NTP Verify NTP Operation Verifying that the NTP Policy Deployed to Each Node Domain Name Services (DNS) Verifying DNS Operation • Role-Based Access Control Multiple Tenant Support User Roles Security Domains Creation of a Security Domain Adding Users Remote Authentication • Import and Export Policies Configuration Export (Backup) Add a Remote Location (SCP) Create a One Time Export Policy Verify Export Policy was Successful Extract and View Configuration Files Configuration Import (Restore/Merge) 25 .

.

This con​ troller-based ar​ chi​ tec​ ture also makes pos​ si​ ble a state​ less con​ fig​ ur​ a​ tion model that de​ cou​ ples the hard​ ware from the con​ fig​ ur​ a​ tion run​ ning on it. The de​ vice can also be sub​ sti​ tuted in a state​ less fash​ ion. Each node re​ ceives a unique node iden​ ti​ fier. which mod​ if​ ies the con​ tents of a syn​ chro​ nized data​ base that is repli​ cated across APICs in a clus​ ter and pro​ vides an ab​ strac​ tion layer be​ tween all of the in​ ter​ faces. with no risk of in​ con​ sis​ tency be​ tween the var​ io ​us data in​ ter​ faces. This re​ sults in a clean and pre​ dictable tran​ si​ tion be​ tween the in​ ter​ faces. This trans​ lates to an APIC clus​ ter that man​ ages in​ di​ vid​ ual fab​ ric nodes of leaf and spine switches that de​ rive their iden​ tity from what the con​ troller de​ fines as being the de​ sired in​ tent. based on a con​ cept known as the promise . mon​ it​ or​ ing. nor from a con​ fig​ ur​ a​ tion file re​ sid​ ing on the de​ vices. This con​ troller pro​ vides ac​ cess to all con​ fig​ ur​ a​ tion. The de​ sired state model for con​ fig​ ur​ a​ tion fur​ ther com​ ple​ ments these con​ cepts of con​ troller-based man​ age​ ment and state​ less​ ness by tak​ ing ad​ van​ tage of a con​ cept known as de​ clar​ at​ ive con​ trol-based man​ age​ ment. The un​ der​ ly​ ing in​ ter​ face for all ac​ cess meth​ ods is pro​ vided through a REST-based API. and ap​ pli​ ca​ tion pro​ gram​ ming in​ ter​ face (API). These changes in​ clude: • Single point of management controller-based architecture • Stateless hardware • Desired state-driven eventual consistency model The sin​ gle point of man​ age​ ment within the ACI ar​ chi​ tec​ ture is known as the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). and health func​ tions. re​ duce the num​ ber of touch​ points.Management 27 APIC Overview There are a num​ ber of fun​ da​ men​ tal dif​ fer​ ences be​ tween the op​ er​ at​ ions of tra​ di​ tional net​ work​ ing hard​ ware and an ACI fab​ ric. Hav​ ing a cen​ tral​ ized con​ troller with an ap​ pli​ ca​ tion pro​ gram​ ming in​ ter​ face (API) means that all func​ tions con​ fig​ ured or ac​ cessed through the fab​ ric can be uni​ formly ap​ proached through a graph​ ic ​al user in​ ter​ face (GUI). and not from the se​ ri​ al num​ ber of the chas​ sis. man​ age​ ment. topol​ ogy changes are less im​ pact​ ful. which al​ lows for the de​ vice to down​ load the cor​ rect con​ fig​ ur​ a​ tion at​ trib​ utes from the con​ troller. com​ mand line in​ ter​ face (CLI). mean​ ing that hard​ ware swaps can be faster. These dif​ fer​ ences serve to sim​ plify the man​ age​ ment greatly. and de​ cou​ ple the switch​ ing hard​ ware from the de​ sired con​ fig​ ur​ a​ tion in​ tent. and net​ work man​ age​ ment is sim​ pli​ fied.

28 Management the​ ory. be told how to do it. . A sys​ tem based on de​ clar​ at​ ive con​ trol is able to scale much more ef​ fi​ ciently than an im​ per​ at​ ive-based sys​ tem. since each en​ tity within the do​ main is re​ spon​ si​ ble for know​ ing its cur​ rent state and the steps re​ quired to get to the de​ sired state. This stands in con​ trast with the tra​ di​ tional model of im​ per​ at​ ive con​ trol. dic​ tated by the man​ ag​ ing con​ troller. where each man​ aged el​ e​ ment must be told pre​ cisely what to do. De​ clar​ at​ ive con​ trol dic​ tates that each ob​ ject is asked to achieve a de​ sired state and makes a "promise" to reach this state with​ out being told pre​ cisely how to do so. and take into ac​ count the spe​ cific sit​ ua ​t​ ional as​ pects that will im​ pact its abil​ ity to get from its cur​ rent state to the con​ fig​ ured state.

Maintaining accurate time across the application logs and infrastructure components is extremely important for being able to correlate events. as well as a “CDP-DIS​ ABLED” pol​ icy with CDP dis​ abled that can be ref​ er​ enced through​ out all in​ ter​ face pol​ icy con​ fig​ ur​ a​ tions. man​ ua ​lly cre​ ate a “CDP-EN​ ABLED" pol​ icy with CDP en​ abled. and storage arrays. With ACI. ACME has existing NTP servers and will leverage them for maintaining time in their ACI fabric deployment.These policies are primarily consumed by the network team. the ACI fab​ ric pro​ vides a de​ fault pol​ icy where CDP is dis​ abled. • Domain Name Services (DNS) . LLDP is a standards-based protocol for discovering topology relationships. as CDP is the standard for their existing network equipment.Discovery protocols are no longer limited to just network devices. As a rec​ om​ mended prac​ tice. these policies will be available for consumption by all of the teams. Cisco Discovery Protocol Cisco Dis​ cov​ ery Pro​ to​ col (CDP) is a valu​ able source of in​ for​ ma​ tion in trou​ bleshoot​ ing phys​ ic ​al and data-link layer is​ sues. the net​ work team cre​ ated the fol​ low​ ing man​ age​ ment pro​ to​ col poli​ cies: • Cisco Discovery Protocol (CDP) . the important ACI components are also registered with ACME's DNS server so teams can easily access them in their management tasks. such as when to cre​ ate switch pro​ files to con​ nect to Cisco UCS B Se​ ries servers. • Link Layer Discovery Protocol (LLDP) . maintaining accurate time is of increased importance as accurate time is a prerequisite for features such as atomic counters. Layer 4 to Layer 7 services devices. It is also im​ por​ tant to note that CDP might be re​ quired for cer​ tain con​ fig​ ur​ a​ tions. ACME is deploying LLDP on servers. .Management 29 Configuring Management Protocols For the op​ ti​ miza​ tion of ACME's Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. By de​ fault. there may be times when you must pro​ vi​ sion a par​ tic​ ul​ ar host-fac​ ing port man​ ua ​lly with a CDP on or off pol​ icy. In the course of ACI op​ er​ at​ ions. Therefore. • Network Time Protocol (NTP) .providing name resolution to fabric components consistent with the application and server teams. In addition to be able to resolve names within the fabric.

the LLDP fea​ ture is en​ abled on the fab​ ric. choose Actions > Create CDP Interface Policy. There might be times in the op​ er​ at​ ion of the ACI fab​ ric in which you must man​ ua ​lly ad​ just the LLDP pro​ to​ col to con​ form with in​ ter​ op​ er​ abil​ ity re​ quire​ ments of end-host de​ vices. Click Submit. choose Fabric > Access Policies. 4 In the Create CDP Interface Policy dialog box. choose Actions > Create LLDP Interface Policy. For ex​ am​ ple. you must dis​ able LLDP. In the Name field enter the name of the policy. 2 In the Navigation pane. c. click Disabled. Cisco rec​ om​ mends that you pre-pro​ vi​ sion LLDP en​ able and dis​ able pro​ to​ col poli​ cies to make fu​ ture in​ ter​ face pol​ icy de​ ploy​ ment de​ ci​ sions stream​ lined and eas​ ily con​ fig​ ured. 3 In the Work pane. 2 In the Navigation pane. when con​ nect​ ing to Cisco Uni​ fied Com​ put​ ing Sys​ tem (UCS) Blade servers. In the course of ACI op​ er​ at​ ions. To cre​ ate an LLDP pol​ icy: 1 On the menu bar. such as "CDP-ENABLED". such as "CDP-DISABLED". c. perform the following actions: . Click Submit. In the Name field enter the name of the policy. choose Actions > Create CDP Interface Policy. For the Admin State radio buttons. choose Fabric > Access Policies.Management 30 To cre​ ate CDP poli​ cies: 1 On the menu bar. by de​ fault. Your CDP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric in​ ter​ faces. b. In the Create CDP Interface Policy dialog box. Link Layer Discovery Protocol Link Layer Dis​ cov​ ery Pro​ to​ col (LLDP) is a valu​ able source of in​ for​ ma​ tion for trou​ bleshoot​ ing phys​ ic ​al and data-link layer is​ sues. choose Interface Policies > Policies > CDP Interface. perform the following actions: a. 5 6 In the Work pane. 4 In the Create LLDP Interface Policy dialog box. 3 In the Work pane. click Enabled. perform the following actions: a. b. choose Interface Policies > Policies > LLDP Interface. For the Admin State radio buttons.

enter "LLDP-TX-OFF-RX-ON". perform the following actions: a. 5 6 In the Work pane. b. d. d. In the Create LLDP Interface Policy dialog box. d. Your LLDP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric in​ ter​ faces. time syn​ chro​ niza​ tion is a cru​ cial ca​ pa​ bil​ ity upon which many of the mon​ it​ or​ ing. For the Receive State radio buttons. enter "LLDP-TX-OFF-RX-OFF". b. In ad​ di​ tion. In the Name field. b. In the Name field. In the Create LLDP Interface Policy dialog box. click Enabled. choose Actions > Create LLDP Interface Policy. Nonex​ is​ tent or im​ proper con​ fig​ ur​ a​ tion of time syn​ chro​ niza​ tion does not nec​ es​ sar​ ily trig​ ger a fault or a low health score. c. click Enabled. For the Receive State radio buttons. b. click Disabled. For the Transmit State radio buttons. perform the following actions: a. enter "LLDP-TX-ON-RX-OFF". In the Name field. Click Submit. click Enabled. c. For the Transmit State radio buttons.Management 31 a. For the Receive State radio buttons. 9 In the Create LLDP Interface Policy dialog box. For the Transmit State radio buttons. click Disabled. click Disabled. 7 8 In the Work pane. choose Actions > Create LLDP Interface Policy. clock syn​ chro​ niza​ tion al​ lows for the full uti​ liza​ tion of the atomic counter ca​ pa​ bil​ ity that is built into the ACI upon which the ap​ pli​ ca​ tion health scores de​ pend. perform the following actions: a. c. d. Time Synchronization and NTP Within the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. click Disabled. Click Submit. click Enabled. and trou​ bleshoot​ ing tasks de​ pend. You . For the Receive State radio buttons. For the Transmit State radio buttons. c. In the Name field. op​ er​ at​ ional. Clock syn​ chro​ niza​ tion is im​ por​ tant for proper analy​ sis of traf​ fic flows as well as for cor​ re​ lat​ ing debug and fault time​ stamps across mul​ ti​ ple fab​ ric nodes. Click Submit. enter "LLDP-TX-ON-RX-ON". Click Submit. An off​ set pre​ sent on one or more de​ vices can ham​ per the abil​ ity to prop​ erly di​ ag​ nose and re​ solve many com​ mon op​ er​ at​ ional is​ sues.

As of the pub​ lish​ ing of this doc​ um ​ent. in​ clu​ sive of spines. including the following fields: Name. d. each node of the fab​ ric. The re​ li​ ab ​il​ ity of the source must be care​ fully con​ sid​ ered when de​ ter​ min​ ing if you will use a pri​ vate in​ ter​ nal clock or an ex​ ter​ nal pub​ lic clock. De​ pend​ ing on which man​ age​ ment op​ tion was cho​ sen for the fab​ ric. In the Create Providers dialog box. The most widely adapted method for syn​ chro​ niz​ ing a de​ vice clock is to use Net​ work Time Pro​ to​ col (NTP). . only one pod per ACI fab​ ric is al​ lowed. An​ other con​ sid​ er​ at​ ion in de​ ploy​ ing time syn​ chro​ niza​ tion where the time source is lo​ cated. 2 In the Navigation pane. such as "Production-NTP". a Date and Time pol​ icy must be cre​ ated that ref​ er​ ences an out-ofband man​ age​ ment end​ point group. choose Fabric > Fabric Policies. 4 In the Create Date and Time Policy dialog box.Management 32 should con​ fig​ ure time syn​ chro​ niza​ tion be​ fore de​ ploy​ ing a full fab​ ric or ap​ pli​ ca​ tions so as to en​ able proper usage of these fea​ tures. Prior to con​ fig​ ur​ ing NTP. There are two op​ tions for con​ fig​ ur​ ing man​ age​ ment of all ACI nodes and Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ trollers (APICs). b. is man​ aged from out​ side the ACI fab​ ric. choose Actions > Create Date and Time Policy. leaves and all mem​ bers of the APIC clus​ ter. 3 In the Work pane. and Maximum Polling Intervals. con​ sider what man​ age​ ment IP ad​ dress scheme is in place within the ACI fab​ ric. con​ fig​ ur​ a​ tion of NTP will vary. perform the following actions: a. Provide a name for the policy to easily distinguish between your environment's different NTP configurations. c. Click Next. enter all relevant information. This IP reach​ ab ​il​ ity will be lever​ aged so that each node can in​ di​ vid​ u​ ally query the same NTP server as a con​ sis​ tent clock source. Date and Time poli​ cies are con​ fined to a sin​ gle pod and must be de​ ployed across all pods pro​ vi​ sioned in the ACI fab​ ric. Out-of-Band Management NTP When an ACI fab​ ric is de​ ployed with out-of-band man​ age​ ment. To con​ fig​ ure NTP. in-band man​ age​ ment and/or out-of-band man​ age​ ment. and Minimum Polling Intervals. 1 On the menu bar. choose Pod Policies > Policies. see the Re​ ac​ tive Mon​ it​ or​ ing Tools chap​ ter. For more in​ for​ ma​ tion on atomic coun​ ters. Click the + sign to specify the NTP server information (provider) to be used. Description.

choose Fabric > Fabric Policies. In-Band Management NTP When an ACI Fab​ ric is de​ ployed with in-band man​ age​ ment. con​ sider the reach​ ab ​il​ ity of the NTP server from within the ACI in-band man​ age​ ment net​ work. see the "In-Band Management NTP" section. you can cre​ ate the NTP pol​ icy as es​ tab​ lished in the "Out-of-Band Man​ age​ ment NTP" sec​ tion. Once you have es​ tab​ lished a pol​ icy to allow com​ mu​ ni​ ca​ tion. In-band IP ad​ dress​ ing used within the ACI fab​ ric is.Management 33 i. Your NTP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric nodes. e. verify the details of the server. The steps used to con​ fig​ ure in-band man​ age​ ment poli​ cies are iden​ ti​ cal to those used to es​ tab​ lish an out-of-band man​ age​ ment pol​ icy: the dis​ tinc​ tion is around how to allow the fab​ ric to con​ nect to the NTP server. you must con​ struct a pol​ icy to en​ able this com​ mu​ ni​ ca​ tion. Pol​ icy con​ fig​ ur​ a​ tion can be ver​ if​ ied by these steps: 1 On the menu bar. if the NTP server is reachable by all nodes on the fabric through out-of-band management. Click OK. To lever​ age an NTP server ex​ ter​ nal to the fab​ ric with in-band man​ age​ ment. choose Pod Policies > Policies > Date and Time > ntp_policy > server_name. ii. 3 In the Work pane. 2 In the Navigation pane. If you have deployed in-band management. click the Preferred check box for the most reliable NTP source. by de​ f​ in ​i​ t​ ion. Start ver​ if​ y​ ing time syn​ chro​ niza​ tion con​ fig​ ur​ a​ tion by en​ sur​ ing that all poli​ cies are con​ fig​ ured prop​ erly in​ side of the APIC GUI. In the Management EPG drop-down list. The ntp_policy is the previously created policy. choose Out-ofBand. . Verifying NTP Operation Full ver​ if​ i​ ca​ tion of NTP func​ tion​ al​ ity is best ac​ com​ plished by lever​ ag​ ing both the ACI CLI and the APIC GUI. not reach​ able from any​ where out​ side the fab​ ric. Repeat steps c and d for each provider that you want to create. If you are creating multiple providers. iii.

admin@apic1:~> attach node_name 4 View the NTP peer status: leaf-1# show ntp peer-status A reach​ able NTP server has its IP ad​ dress pre​ fixed by an as​ ter​ isk (*). From this prompt. choose Fabric > Fabric Policies. 3 In the Work pane. Domain Name Services (DNS) Set​ ting up a DNS server al​ lows the APIC to re​ solve var​ io ​us host​ names to IP ad​ dresses. To ac​ cess the NX-OS Vir​ tual Shell.Management 34 Verifying that the NTP Policy Deployed to Each Node To ver​ ify that the pol​ icy has been suc​ cess​ fully de​ ployed to each node. 1 On the menu bar. 2 In the Navigation pane. and the delay is a non-zero value. 1 SSH to an APIC in the fabric. you should use the NX-OS Vir​ tual Shell that is pre​ sent in the APIC. 5 Repeat steps 3 and 4 for each node on the fabric. . ex​ e​ cute the "ver​ sion" com​ mand to ob​ tain a con​ sol​ id ​ated list of node host​ names. This is use​ ful when in​ te​ grat​ ing VMM do​ mains or other Layer 4 to Layer 7 de​ vices and the host​ name is ref​ er​ enced. choose Global Policies > DNS Profiles > default. in the Management EPG drop-down list. 2 Press the Tab key twice after the entering the attach command to list all of the available node names: admin@apic1:~> attach <Tab> <Tab> 3 Log in to one of the nodes using the same password that you used to access the APIC. open an SSH ses​ sion to the out-of-band man​ age​ ment IP in​ ter​ face of any of the APIC nodes. choose the appropriate management EPG.

Click Update. Note: You can have only one domain name as the default. b. enter the domain name. 8 Click Submit. Verifying DNS Operation 1 Check the resolv.168.conf # Generated by IFC search cisco. In the Address field. click the check box to make this domain the default domain. 5 Repeat step 4 for each additional DNS provider.183 nameserver 173. In the Default field . click the check box if you want to have this address as the preferred provider. enter the provider address.131. Click Update. 7 Repeat step 6 for each additional DNS Domain suffix.conf file from the APIC CLI: admin@apic1:~> cat /etc/resolv.70. b. such as "cisco. In the Preferred field. c. In the Name field. Note: You can have only one preferred provider.10 .com".Management 35 Note: The default is default (Out-of-Band). c. a. a.com nameserver 171. 6 Click + next to DNS Domains to add a DNS domain. 4 Click + next to DNS Providers to add a DNS provider.36.

163.038/43.com ping statistics --3 packets transmitted.cisco. 3 received.cisco.com (72.163.619 ms admin@apic1:~> .com (72.7 ms 64 bytes from www1.cisco.903/0.cisco.4.9 ms ^C --. 0% packet loss. time 2102ms rtt min/avg/max/mdev = 42.com (72.4. admin@apic1:~> ping cisco.163.com (72.161) 56(84) bytes of data.4.161): icmp_seq=2 ttl=241 time=42.161): icmp_seq=1 ttl=241 time=42.4.161): icmp_seq=3 ttl=241 time=43.Management 36 2 Ping a host by DNS name that will be reachable from the APIC out-of-band management.485/43.com PING cisco.163.4 ms 64 bytes from www1. 64 bytes from www1.

and is un​ able to man​ age other user roles at a fab​ ric wide level. Think of this in terms of Who. whereas a ten​ ant-ad​ min can only have ac​ cess to the com​ po​ nents within the ten​ ant to which they're as​ so​ ci​ ated. privileges and privilege types (which can be "no access". a fab​ ad​ min can have ac​ cess to the en​ tire fab​ ric as well as to as​ sign​ ing other user roles and as​ so​ ci​ ate them to se​ cu​ rity do​ mains. a priv​ il​ ege type.Management 37 Role-Based Access Control Role-based ac​ cess con​ trol (RBAC) and ten​ ancy are im​ por​ tant con​ cepts to mas​ ter when op​ er​ at​ ing a Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. What. or "read-write") information tree (MIT) that a user can access . Role-based ac​ cess and the con​ cept of ten​ ancy are two core foun​ da​ tions upon which the ACI man​ age​ ment and pol​ icy mod​ els are built. and Where. priv​ il​ eges de​ fine What func​ tions a user can per​ form. ricThe role clas​ si​ fi​ ca​ tion is an es​ tab​ lished group​ ing of per​ mis​ sions. The APIC pro​ vides ac​ cess ac​ cord​ ing to a user’s role through RBAC. When com​ bin​ ing these ob​ jects. "read- • One or more security domain tags that identify the portions of the management only". Each user can be as​ signed to a set of roles. ACME's IT or​ ga​ ni​ za​ tion fits per​ fectly into this ac​ cess con​ trol model. and a se​ cu​ rity do​ main. the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) grants ac​ cess to all ob​ jects based on a user's es​ tab​ lished role in the con​ fig​ ur​ a​ tion. In an ACI fab​ ric. you can im​ ple​ ment very gran​ ul​ ar ac​ cess con​ trol in the sys​ tem. For ex​ am​ ple. and the se​ cu​ rity do​ main de​ fines Where the user can per​ form these ac​ tions. The role is the Who (log​ ic ​al col​ lec​ tion of priv​ il​ eges). An ACI fab​ ric user is as​ so​ ci​ ated with the fol​ low​ ing: • A set of roles • For each role.

Read/write re​ stric​ tions pre​ vent any ten​ ant from see​ ing any other ten​ ant’s con​ fig​ u​ ra​ tion. fab​ ric-equip​ ment is a priv​ i​ lege flag. roles.38 Management Authenticated user creation lifecycle The ACI fab​ ric man​ ages ac​ cess priv​ i​ leges at the man​ aged ob​ ject (MO) level. The com​ bi​ na​ tion of user priv​ i​ leges. and se​ cu​ rity do​ mains with ac​ cess rights in​ her​ i​ tance en​ ables ad​ min​ is​ tra​ tors to con​ fig​ ure AAA func​ tions at the man​ aged ob​ ject level in a very gran​ u​ lar fash​ ion. or GUI. For ex​ am​ ple. . This flag is set by the APIC on all ob​ jects that cor​ re​ spond to equip​ ment in the phys​ i​ cal fab​ ric. A priv​ i​ lege is an MO that en​ ables or re​ stricts ac​ cess to a par​ tic​ u​ lar func​ tion within the sys​ tem. and ac​ count​ ing (AAA) func​ tions of the Cisco ACI fab​ ric. au​ then​ ti​ ca​ tion. Multiple Tenant Support A core APIC in​ ter​ nal data ac​ cess con​ trol sys​ tem pro​ vides multi-ten​ ant iso​ la​ tion and pre​ vents in​ for​ ma​ tion pri​ vacy from being com​ pro​ mised across ten​ ants. APIC poli​ cies man​ age the ac​ cess. CLI. sta​ tis​ tics. These con​ fig​ u​ ra​ tions can be im​ ple​ mented using the REST API.

1 On the menu bar. it will be pos​ si​ ble to com​ pletely sep​ ar​ ate con​ fig​ ur​ a​ tion of ap​ pli​ ca​ tion work​ loads while only pro​ vid​ ing ac​ cess to those who need it.Management 39 faults. Cisco does not rec​ om​ mend that you pro​ vide user ac​ cess con​ fig​ u-​ ra​ tion by mod​ if​ ing per​ mis​ sions of the "all" do​ main. or event data. or events. poli​ cies. you can see each built-in role and the associated privileges. Security Domains The se​ cu​ rity do​ main con​ cept is cru​ cial to proper op​ er​ at​ ion of the ACI fab​ ric. 2 In the Navigation pane. . Additionally. ten​ ants are re​ stricted from read​ ing fab​ ric con​ fig​ ur​ a​ tion. choose Admin > AAA. if a ten​ ant named solar is tagged with the se​ cu​ rity do​ main called sun and a VMM do​ main is also tagged with the se​ cu​ rity do​ main called sun. the users con​ tained in the se​ cu​ rity do​ main can ac​ cess the cor​ re​ spond​ ing VMM do​ main. Changes in the "all" do​ main will af​ fect ac​ cess per​ mis​ sions for all users. Note: From here. If you need to make se​ lec​ tive changes to allow ac​ cess out​ side of a user's se​ cu​ rity do​ main. Un​ less the ad​ min​ is​ tra​ tor as​ signs per​ mis​ sions to do so. faults. The poli​ cies can​ not cur​ rently be ap​ plied to in​ di​ vid​ ual ob​ jects de​ spite ref​ er​ ences at the ob​ ject level in​ side of the RBAC screen in the APIC. most com​ monly ap​ plied to ten​ ants. Cisco rec​ om​ mends that you con​ fig​ ure se​ cu​ rity do​ mains and ac​ cess lev​ els prior to de​ ploy​ ment of ten​ ants. Using the ten​ ancy ca​ pa​ bil​ it​ ies of the ACI fab​ ric in con​ junc​ tion with prop​ erly con​ fig​ ured se​ cu​ rity do​ mains. sta​ tis​ tics. A key con​ cept to keep in mind when con​ fig​ ur​ ing se​ cu​ rity do​ mains is that the con​ fig​ u-​ ra​ tion only ap​ plies at a ten​ ant level. For ex​ am​ ple. users can be or​ ga​ nized into var​ io ​us per​ mis​ sion struc​ tures. By using se​ cu​ rity do​ mains. choose Security Management > Roles. If a vir​ tual ma​ chine man​ age​ ment (VMM) do​ main is tagged as a se​ cu​ rity do​ main. User Roles You can view the built-in user roles as well as cre​ ate cus​ tom roles to meet spe​ cific re​ quire​ ments. be sure to set up a dis​ crete user ac​ cess pol​ icy just for that com​ mu​ ni​ ca​ tion. then users in the solar ten​ ant can ac​ cess the VMM do​ main ac​ cord​ ing to their ac​ cess rights. you can create a custom role from the Actions menu.

​ html To cre​ ate a se​ cu​ rity do​ main: 1 On the menu bar. Give the security domain a name and an optional description. Con​ fig​ ure Local Users: 1 On the menu bar. see the doc​ um ​ent at the fol​ low​ ing URL: http://​ www. 2 In the Navigation pane. choose Security Management > Security Management. Adding Users You might need to add new users within the ACI fab​ ric. a se​ cu​ rity do​ main would be cre​ ated for each ten​ ant. . Ac​ tive Di​ rec​ tory. You can add se​ cu​ rity do​ mains as nec​ es​ sary. choose AAA Authentication. and "mgmt". or use some​ thing such as LDAP. 2 In the Navigation pane. perform the following actions: a.​ cisco. choose Admin > AAA. for a multi-ten​ ant en​ vi​ ron​ ment. 3 In the Work pane. The "all" se​ cu​ rity do​ main usu​ ally in​ cludes ac​ cess to every​ thing within the man​ age​ ment in​ for​ ma​ tion tree (MIT). or ten​ ants. choose Actions > Create a Security Domain.​ com/​ c/​ en/​ us/​ products/​ collateral/​ cloud-systems-management/​ aci-fabric-controller/​ white-paper-c11-729586. in the Realm drop-down list. For ex​ am​ ple. "com​ mon".Management 40 Creation of a Security Domain There are three se​ cu​ rity do​ mains that are pro​ vi​ sioned by de​ fault on the ACI fab​ ric: "all". "Com​ mon" is usu​ ally used when there is a need for shared re​ sources be​ tween ten​ ants. 4 In the Create a Security Domain dialog box. such as DNS or di​ rec​ tory ser​ vices. For more in​ for​ ma​ tion about the MIT. RBAC poli​ cies are con​ fig​ ured under the "Admin" tab of the APIC GUI. The "mgmt" se​ cu​ rity do​ main is for man​ age​ ment traf​ fic poli​ cies. or TACACS+ to spec​ ify users that will be al​ lowed to man​ age cer​ tain parts of the ACI net​ work. RA​ DIUS. Users are then cre​ ated and as​ signed to cer​ tain se​ cu​ rity do​ mains. choose Admin > AAA. choose Local and click Submit if Local is not already chosen. ACI can have local users man​ u-​ ally en​ tered by an admin. 3 In the Work pane.

such as Read/Write for admin or tenant admin. and click Next. If cre​ at​ ing an LDAP ac​ count. de​ tail in the Cisco ACI Fun​ . along with the port it uses to com​ mu​ ni​ cate as well as any other rel​ e​ vant in​ for​ ma​ tion that will allow con​ nec​ tion to that server. The same is true for RA​ DIUS and TACACS+. 6 41 In the Create Local User dialog box. The next op​ tion is to con​ fig​ ure groups from which it is al​ lowed to read data and grab it for the pur​ poses of se​ lect​ ing re​ mote users granted ac​ cess to the ACI net​ work. perform the following actions: a. The IP ad​ dress or host name of the LDAP server is needed. c. b. Re​ mote au​ then​ ti​ ca​ tion is cov​ ered in da​ men​ tals Guide. choose Actions > Create Local User. Remote Authentication Cre​ at​ ing re​ mote user ac​ counts in the ACI is sim​ il​ ar to most other data cen​ ter sys​ tems. choose Security Management. 5 In the Work pane. Select the roles to be given to this user. then the LDAP provider must be con​ fig​ ured first.Management 4 In the Navigation pane. Specify login information for the user and 7 Click Finish. Specify any security information that is necessary and click Next.

.

Pro​ vid​ ing a tech​ ni​ cal sup​ port bun​ dle will give Cisco sup​ port en​ gi​ neers the in​ for​ ma​ tion needed to help iden​ tify and sug​ gest re​ me​ di​ at​ ion to is​ sues. in the case where an ex​ ter​ nal SCP/FTP server is not avail​ able. and so the bun​ dles will not be cov​ ered in de​ tail bleshoot​ here. The backup and re​ store process does not re​ quire backup of in​ di​ vid​ ual com​ po​ nents. and of​ fline analy​ sis. you might need to con​ tact the Cisco Tech​ ni​ cal As​ sis​ tance Cen​ ter (TAC). This can be done from any ac​ tive and fully fit APIC within the ACI fab​ ric. ACI is a good choice on this front as well. For de​ tails on tech​ ni​ cal sup​ port ex​ port poli​ cies. all poli​ cies and ten​ ants are backed up. a pol​ icy needs to be con​ fig​ ured to ex​ port or backup the con​ fig​ ur​ a​ tion poli​ cies. This is use​ ful in multi-ten​ ant de​ ploy​ ments where in​ di​ vid​ ual ten​ ants wish to backup or re​ store their re​ spec​ tive poli​ cies. trou​ bleshoot​ ing. As with all things APIC. but the ad​ min​ is​ tra​ tor can op​ tion​ ally spec​ ify only a spe​ cific sub​ tree of the man​ age​ ment in​ for​ ma​ tion tree. Tech​ ni​ cal sup​ port ex​ port poli​ cies can be con​ fig​ ured to run on-de​ mand or sched​ uled for re​ cur​ ring pur​ poses.Management 43 Import and Export Policies All of the stake​ hold​ ers at ACME in​ volved in the de​ ploy​ ment and ad​ min​ is​ tra​ tion of the new mo​ bile ap​ pli​ ca​ tion plat​ form need to know that they will be able to eas​ ily re​ cover from any loss of con​ fig​ ur​ a​ tion quickly and eas​ ily. Cisco rec​ om​ mends that a cur​ rent Backup be per​ formed be​ fore mak​ ing any major sys​ tem con​ fig​ ur​ a​ tion changes or ap​ ply​ ing soft​ ware up​ dates. An​ other Ex​ port pol​ icy re​ quired on oc​ ca​ sion is for gath​ er​ ing tech​ ni​ cal sup​ port in​ for​ ma​ tion. Back​ ups are con​ fig​ urable through an ex​ port pol​ icy that al​ lows ei​ ther sched​ uled or im​ me​ di​ ate back​ ups to a re​ mote server (pre​ ferred) or. Tech​ ni​ cal sup​ port bun​ dles are con​ fig​ ured very sim​ i-​ lar to con​ fig​ ur​ a​ tion ex​ port poli​ cies. Backup/ex​ port poli​ cies can be con​ fig​ ured to be run on-de​ mand or based on a re​ cur​ ring sched​ ule. back​ ups to be writ​ ten to the local APIC file sys​ tem. Configuration Export (Backup) . see the Cisco APIC Trou​ ing Guide that is ref​ er​ enced in the Ap​ pen​ dix of this book. By de​ fault. Con​ fig​ ur​ a​ tion Im​ port poli​ cies (pol​ icy re​ store) will be dis​ cussed later in this sec​ tion. If is​ sues are en​ coun​ tered within the sys​ tem. Since all APIC poli​ cies and con​ fig​ ur​ a​ tion data can be ex​ ported to cre​ ate back​ ups and tech sup​ port files for Dis​ as​ ter Re​ cov​ ery.

The pro​ ce​ dure below de​ tails how to cre​ ate a re​ mote lo​ ca​ tion and ex​ port pol​ icy in ACI to trans​ fer the con​ fig​ u​ ra​ tion file bun​ dle to an SCP server. a re​ mote lo​ ca​ tion is cre​ ated (the ex​ ter​ nal server to which you want to ex​ port in​ for​ ma​ tion). Choose a Management EPG Note: default (Out-of-Band) 5 Click Submit. Choose a Protocol d. Enter a Remote path e. 2 In the Navigation pane. The in​ di​ vid​ ual XML files can be ex​ tracted and viewed after the con​ fig​ u​ ra​ tion bun​ dle is trans​ ferred to the desk​ top.Management 44 Configuration Export (Backup) Since ACI is pol​ icy dri​ ven. An ex​ trac​ tion util​ ity is needed to de​ com​ press the tar. Enter a Remote port f. 3 In the Work pane. Enter a Remote location name b. then an Ex​ port pol​ icy task. Configuration Export Policy Add a Remote Location (SCP) 1 On the menu bar. choose Admin > Import/Export. .​ gz file that is cre​ ated. Enter a Hostname/IP address c. more than a backup job is re​ quired. choose Remote Locations. perform the following actions: a. Enter a Password h. Enter a Username g. choose Actions > Create Remote Location 4 In the Create Remote Location dialog box. First. and op​ tion​ ally a Sched​ uler (for re​ cur​ ring tasks).

3 In the Work pane. choose the Operational tab. (Optional) Confirm on the SCP server that the backup filename exists. 1 On the menu bar. . choose Export Policies > Configuration > Export_Name. Name = Export_Policy_Name b. choose Export Policies > Configuration. Start Now = Yes d. choose Actions > Create Configuration Export Policy 4 In the Create Configuration Export Policy dialog box. Verify Export Policy was Successful 1 On the menu bar. perform the following actions: a. but the pro​ ce​ dure for a tech​ ni​ cal sup​ port ex​ port pol​ icy is very sim​ il​ ar. 3 In the Work pane. 2 In the Navigation pane.Management 45 Create a One Time Export Policy The pro​ ce​ dure below de​ tails a con​ fig​ ur​ a​ tion ex​ port pol​ icy. b. Two op​ tional con​ fig​ ur​ a​ tions are ap​ ply​ ing a Sched​ uler pol​ icy if you want to setup a re​ cur​ ring op​ er​ at​ ion. choose Admin > Import/Export. Export Destination = Choose_the_Remote_location_created_above 5 Click Submit. choose Admin > Import/Export. a. and spec​ if​ y​ ing a spe​ cific Dis​ tin​ guished Name (DN) if you want to backup only a sub​ set of the Man​ age​ ment In​ for​ ma​ tion Tree (MIT). The State should change from "pending" to "success" when the export completes correctly. Format = XML c. 2 In the Navigation pane.

2 In the Navigation pane. 1 From the workstation where the exported bundle has been copied. 4 In the Create Import Configuration Policy dialog box. 3 Double-click one of the XML files to view the contents in a browser. and then select Extract. select the archive utility of choice.Management 46 Extract and View Configuration Files A con​ fig​ ur​ a​ tion ex​ port task ex​ ports a com​ pressed bun​ dle of in​ di​ vid​ ual XML files. These XML con​ fig​ ur​ a​ tion files can be ex​ tracted and viewed on an​ other work​ sta​ tion. choose Actions > Create Import Configuration Policy. choose Admin > Import/Export.gz).Will replace all configurations with those only from the backup file. Enter a name and the filename for the import policy. Will not overrite any existing policies. Choose the import type: Merge . choose Import Policies > Configuration. Configuration Import (Restore/Merge) To re​ store a con​ fig​ ur​ a​ tion from a pre​ vi​ ous backup: If the re​ mote lo​ ca​ tion does not exist. Do not include the file path. 3 In the Work pane.tar.Will merge backup configuration with existing running configurations. b.gz". . cre​ ate a re​ mote lo​ ca​ tion per the "Adding a Re​ mote Lo​ ca​ tion (SCP)" sec​ tion. perform the following actions: a. 4 Examine the various XML configuration files for parameters that have been configured. Replace . such as "backupfile. 1 On the menu bar. highlight the file. 2 Navigate to the folder where the config export is located (tar.

Attempts to import each shard. The default is best-effort. skipping any invalid objects atomic . The import mode must be specified when you attempt to perform a Merge import. skipping those which contain an ​ invalid configuration.Management 47 c. Choose Start Now. e. A shard's entire configuration must be valid to be imported in this mode. The Replace Import Mode can be either: best-effort . . Choose the Remote Location that was previously created in the "Adding a Remote Location (SCP)" section.Each shard is imported. d. The configuration data is imported per shard with each shard holding a certain part of the system configuration objects.

.

49 Upgrading and Downgrading Firmware .

.

Upgrading and Downgrading Firmware Section Content • Firmware Management Firmware Versions Firmware Components Firmware Policies Firmware Groups Maintenance Groups Controller Firmware Catalogue Firmware • Upgrading and Downgrading Considerations • Upgrading the Fabric Downloading the Firmware Images Upgrading the APIC Controller Software Upgrading the Switch Software Using the GUI Upgrading the APIC Controller Software Using the CLI Upgrading the Switch Software Using the CLI Verifying Cluster Convergence Troubleshooting Failures During the Upgrade Process 51 .

.

Re​ lease notes for the APIC ver​ sions ref​ er​ ence the cor​ re​ spond​ ing switch ver​ sions and vice-versa. or features content.minor(mntnc) • major .Represents bug fixes to a feature release of APIC.Upgrading and Downgrading Firmware 53 Firmware Management ACME Inc. they have put a proac​ tive plan in place to re​ visit this de​ ci​ sion pe​ ri​ od​ ic ​ally to de​ ter​ mine if fu​ ture up​ grades are re​ quired. • minor .Represents a minor release with new software features. Firmware Versions The soft​ ware ver​ sions for ACI are listed in the fol​ low​ ing for​ mat: major. and spine switches. has eval​ ua ​ted the re​ quire​ ments for their de​ ploy​ ment based on the soft​ ware fea​ tures re​ quired. APIC re​ lease 1. Ad​ di​ tion​ ally. leaf switches.0(2j) cor​ re​ sponds to switch soft​ ware 11. • mntnc . and the ma​ tu​ rity of the soft​ ware re​ leases. in part​ ner​ ship with Cisco. platform. For ex​ am​ ple.0(2j)..1(1d) Both the soft​ ware for the APIC and the fab​ ric nodes are de​ noted by the same ver​ sion​ ing scheme. the sup​ port for the hard​ ware plat​ forms they have se​ lected.Represents major changes in the product architecture. All com​ po​ nents of the ACI in​ fra​ struc​ ture in​ clud​ ing the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). Ex​ am​ ple: APIC version: 1. They have se​ lected a tar​ get ver​ sion of soft​ ware for their de​ ploy​ ment. This changes when there are fixes for product defects in the software. but no additional new features. should be on the same .

1. will have less im​ pact than a major/minor ver​ sion up​ grade. Firmware Components There are three main com​ po​ nents that can be up​ graded: • Switches (leaf and spine) • Application Policy Infrastructure Controller (APIC) • Catalog firmware Firmware Policies Firmware Groups Firmware Group poli​ cies on the APIC de​ fine which group of nodes on which firmware will be up​ graded. mak​ ing it pos​ si​ ble to defer an up​ grade task to a busi​ ness main​ te​ nance win​ dow. as there will be only bug fixes and no new fea​ tures added. Maintenance Groups Main​ te​ nance Group poli​ cies de​ fine a group of switches that will be jointly up​ graded to the as​ so​ ci​ ated firmware set.1(1d). a sin​ gle firmware group is ad​ e​ quate. For most de​ ploy​ ments. While at the time of up​ grad​ ing. Con​ trol over which switches up​ grade to the new ver​ sion can be de​ ter​ mined through main​ te​ nance groups. dis​ parate ver​ sions may exist be​ tween APIC and the switches. Controller Firmware The APIC con​ troller firmware pol​ icy al​ ways ap​ plies to all con​ trollers in the clus​ ter. When con​ sid​ er​ ing the im​ pact and risk of up​ grad​ ing.(1d) => 1. but the up​ grade is al​ ways done se​ quen​ tially. do not op​ er​ ate the fab​ ric for ex​ tended pe​ ri​ ods of time in this state. Main​ te​ nance groups can be up​ graded on de​ mand or ac​ cord​ ing to a sched​ ule.Upgrading and Downgrading Firmware 54 ver​ sion. The APIC GUI pro​ vides real-time sta​ tus in​ for​ - . such as 1. you can as​ sume that a main​ te​ nance ver​ sion up​ grade.

The APIC main​ tains a cat​ a​ log of the firmware im​ ages. and mod​ els that are al​ lowed to use that firmware image. has an image repos​ i​ tory for com​ pat​ i​ bil​ ity cat​ a​ logs. Con​ troller Firmware poli​ cies can be up​ graded on de​ mand or also ac​ cord​ ing to a sched​ ule.Upgrading and Downgrading Firmware 55 ma​ tion about firmware up​ grades. which per​ forms image man​ age​ ment. The APIC. APIC con​ troller firmware im​ ages. Catalogue Firmware Each firmware image in​ cludes a com​ pat​ i​ bil​ ity cat​ a​ log that iden​ ti​ fies sup​ ported switch mod​ els. Firmware upgrade policy relationships . and switch im​ ages. switch types.

.

the controllers should be upgraded first. On the menu bar. always export your configuration to an external source. Confirm that the /firmware partition is not filled beyond 75%. and allocate enough time to troubleshoot or perform a rollback. To resolve issues for controllers that are not fully fit see the Troubleshooting Cisco Application Centric Infrastructure document:https://datacenter. see the Import and Export Policies chapter. • Upgrade order. you should set up at least two separate maintenance groups. your controllers should be in good health. In the navigation pane. Assuming that your hosts are dualconnected to at least one odd and one even leaf node. there should not be any . • Permissions. To help minimize the impact to hosts during an upgrade. Verify that the health state of all of the controllers in the cluster are Fully Fit before you proceed.Upgrading and Downgrading Firmware 57 Upgrading and Downgrading Considerations Be​ fore start​ ing the up​ grade or down​ grade process. Although it is possible to upgrade the fabric without impacting the dataplane. choose Controllers > apic_controller_name > Storage.io/acitroubleshooting-book/ • Configuration backup. • Maintenance groups. For information about exporting configurations. you should perform an upgrade during a scheduled maintenance window according to your change control policy. On the menu bar. choose System > Controllers. • Verify free space.github. A common separation is by odd and even node IDs. If the partition is filled beyond 75%. Typically. you might be required to remove some unused firmware files from the repository to accommodate the compressed image as well as provide adequate space to extract the image. Always refer to the relevant release notes of the destination firmware version for any changes to this order. Before starting any upgrade. choose Controllers > apic_controller_name > Cluster. • Maintenance windows. This window should account for any unforseen issues that might arise during the upgrade. con​ sider the fol​ low​ ing things: • Before starting the upgrade process. followed by the switch nodes. The APIC automatically extracts the image. choose System > Controllers. A user must have the fabric administrator role to perform firmware upgrade tasks. In the navigation pane.

Maintenance group creation is covered in detail later in the chapter. Device packages are not always tied to the APIC software.html .Upgrading and Downgrading Firmware 58 impact to your hosts. software is not specifically tied to the APIC or switch software version. You can confirm the device compatability for Layer 4 to Layer 7 devices using the online Application Centric Infrastructure (ACI) Compatability tool: http://www.com/web/techdoc/aci/acimatrix/matrix. Another consideration is that your leaf VPC pairs should contain one odd and one even node.cisco. • Upgrading a fabric with the Application Virtual Switch (AVS) deployed. The AVS • Device packages.

For the Protocol radio buttons. d. perform the following actions: a.iso • SCP Example: 192. Click Submit. choose the Operational tab to view the download status of the images. choose Actions > Create Outside Firmware Source. choose Fabric Node Firmware. choose Download Tasks. • HTTP Example: http://192. .​ com. 6 In the Navigation pane.1. 2 In the Navigation pane. In the Source Name field.3d". 3 In the Navigation pane. b. enter a name for the switch image. 4 In the Work pane. To down​ load the firmware im​ ages using the APIC GUI: 1 On the menu bar. 8 Repeat the steps for the switch image.Upgrading and Downgrading Firmware 59 Upgrading the Fabric Downloading the Firmware Images You must down​ load both the con​ troller soft​ ware pack​ age and switch soft​ ware pack​ age for the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) from Cisco.1. the list of all switches in the fabric and the status of when the firmware was last upgraded are displayed.1. c. enter the URL from where the image must be downloaded. 5 In the Create Outside Firmware Source dialog box.168. in the Navigation pane.0. choose Download Tasks. enter your username and password.1.50:/tmp/aci-firmware/aci-apicdk9.3d. 9 Once the download reaches 100%.1.iso For SCP. Note: In the Work pane. click the Secure copy or HTTP radio button. 7 In the Work pane.3d.0.50/aci-apic-dk9. such as "apic_1. choose Admin > Firmware. In the URL field.168. choose Firmware Repository.

4 Verify the software has been added to the respository: admin@ifc1:~> firmware list Name : aci-apic-dk9. choose Actions > Upgrade Controller Firmware Policy.iso &#10! The firmware image ver_no.1(1d) Upgrading the APIC Controller Software The cat​ al​ og firmware image is up​ graded when an APIC con​ troller image is up​ graded.Upgrading and Downgrading Firmware 60 10 In the Work pane. To up​ grade the APIC con​ troller soft​ ware using the GUI: 1 On the menu bar. choose Admin > Firmware. 2 Pull the software using SCP: admin@apic1:~> scp username@IP_address_with_the_image:/absolute_path_to_image_plus_image_filename &#10! 3 Place the image into the image repository: admin@apic1:~> firmware add ver_no. the downloaded version numbers and image sizes are displayed.bin Type : controller Version : 1. 3 In the Work pane.1. 4 In the Upgrade Controller Firmware Policy dialog box. click Controller Firmware. You do not need to up​ grade the cat​ al​ og firmware image sep​ ar​ ately.1. perform the following actions: . 2 In the Navigation pane. To down​ load the firmware im​ ages using the APIC CLI: 1 SSH to an APIC and log in as "admin".iso is added to the repos​ it​ ory.1d.

choose Admin > Firmware. To up​ grade the switch soft​ ware using the GUI: 1 On the menu bar. Each APIC con​ troller takes about 10 min​ utes to up​ grade. from the drop-down list. Once a con​ troller image is up​ graded. the APICs must have com​ pleted up​ grad​ ing and have a health state of Fully Fit. click the Apply now radio button. choose the image version to which you want to upgrade. In the Target Firmware Version field. The Sta​ tus di​ al​ og box dis​ plays the Changes Saved Suc​ cess​ fully mes​ sage. The con​ trollers up​ grade in ran​ dom order. it drops from the clus​ ter. Upgrading the Switch Software Using the GUI Be​ fore you up​ grade the switches. and it re​ boots with the newer ver​ sion while the other APIC con​ trollers in the clus​ ter are still op​ er​ at​ ional. and the next con​ troller image starts to up​ grade. In the Apply Policy field. The APIC con​ trollers are up​ graded se​ ri​ ally so that the con​ troller clus​ ter is avail​ able dur​ ing the up​ grade. and is not fully fit. 6 In the browser URL field. the up​ grade will wait until the clus​ ter con​ verges and is fully fit. you can apply a schedule policy if you wish to defer the task to a specific date/time. and the up​ grade process be​ gins. Dur​ ing this pe​ riod. it joins the clus​ ter again. Then the clus​ ter con​ verges. Once the con​ troller re​ boots. When the APIC con​ troller that the browser is con​ nected to is up​ graded and it re​ boots. b. and sign in to the APIC controller as prompted. If the clus​ ter does not im​ me​ di​ ately con​ verge. 5 Verify the status of the upgrade in the Work pane by clicking Controller Firmware in the Navigation pane.Upgrading and Downgrading Firmware 61 a. . a Wait​ ing for Clus​ ter Con​ ver​ gence mes​ sage is dis​ played in the Sta​ tus col​ umn for each APIC as it up​ grades. c. enter the URL for the APIC controller that has already been upgraded. Alternately. the browser dis​ plays an error mes​ sage. Click Submit to complete the task.

In the Group Node IDs field. For example. "101. perform the following substeps: a. iii. In the Navigation pane. 108". v. To verify that the firmware group was created. create one group with the even-numbered devices and the other group with the odd-numbered devices. b. you can choose to create a schedule for upgrading or leave the drop-down list blank so that you can upgrade on demand. in the Navigation pane. b. choose Action > Create Maintenance Group. In the Create Firmware Group dialog box. click Maintenance Groups.Upgrading and Downgrading Firmware 62 2 In the Navigation pane. perform the following actions: i. c. c. Click Submit. 103-105. perform the following substeps: a. perform the following actions: i. For example. In the Work pane. The Work pane displays details about the firmware policy that was created earlier. choose the firmware version to which you will upgrade. In the Work pane. iv. 3 If you have not created a firmware group. click the Pause Upon Upgrade Failure radio button. enter a comma-separated list or a range of node IDs to include in the group. In the Group Name field. iv. "Even-Nodes". 106. Note: Cisco recommends that you create two maintenance groups for all of the switches. enter the name of the firmware group. In the Target Firmware Version drop-down list. Click Submit. choose Fabric Node Firmware. choose the Policy tab. d. the switches that are operating in the fabric are displayed. For example. For example. ii. In the Scheduler drop-down list. 104. In the Create Maintenance Group dialog box. which is the default mode. Note: In the Work pane. In the Group Name field. choose Fabric Node Firmware > Firmware Groups > new_firmware_group. In the Group Node IDs field. "102. 4 If you have not created maintenance groups. . enter the name of the maintenance group. 108. 110". enter a comma-separated list or a range of node IDs to include in the group. Choose Actions > Create Firmware Group. ii. For the Run Mode radio buttons. iii.

The default concurrency in a group is set at 20. Therefore. . view all of the switches that are listed. you will see all the switches listed under the controller node. When you use the GUI. for Do you want to upgrade the maintenance group policy now?. the Status displays that all the switches in the group are being upgraded simultaneously. a group named "Odd-Nodes". Cisco rec​ om​ mends that you to per​ form the firmware up​ grade from the GUI. Note: In the Work pane. To verify that the maintenance group was created. in the Navigation pane. Upgrading the APIC Controller Software Using the CLI The cat​ al​ og firmware image is up​ graded when an APIC con​ troller image is up​ graded. up to 20 switches at a time will get upgraded. connectivity drops. You do not need to up​ grade the cat​ al​ og firmware image sep​ ar​ ately. Note: The Work pane displays details about the maintenance policy. vii. and click the name of the maintenance group that you created. and then the next set of 20 switches are upgraded. view the upgrade image details listed against each switch. If there are any VPC configurations in the cluster. Once the switches rejoin the cluster after rebooting. For example. click Yes. 8 In the Navigation pane. In case of any failures. click Fabric Node Firmware. The switch upgrade takes up to 12 minutes for each group.Upgrading and Downgrading Firmware 63 vi. The switches will reboot when they upgrade. choose Fabric Node Firmware > Maintenance Groups > new_maintenance_group. 6 In the Upgrade Now dialog box. the scheduler pauses and manual intervention is required by the APIC administrator. the APIC per​ forms ad​ di​ tional ver​ if​ i​ ca​ tion and in​ tegrity checks on the soft​ ware image. Repeat this step for the second maintenance group. and the controllers in the cluster will not communicate for some time with the switches in the group. Verify that the switches in the fabric are upgraded to the new image. 5 Right-click one of the maintenance groups that you created and choose Upgrade Now. In the Current Firmware column. Note: In the Work pane. the upgrade process will upgrade only one switch at a time out of the two switches in a VPC domain. 7 Click OK.

200) apic-1. The up​ grade oc​ curs in the back​ ground.0(1.0(1.---------- 1 controller 1. When the APIC con​ troller to which you have con​ nected com​ pletes up​ grad​ ing and re​ boots.202a) inqueue 0 3 controller 1. you can close the SSH win​ dow.200) apic-1.1. Ex​ am​ ple: admin@apic1:~> firmware upgrade controllers ver_no. 3 Check the status of the upgrade.0(1.202a) inqueue -----------------0 2 controller 1. If you see "un​ known" in this field.202a) inprogress 0 The Up​ grade-Sta​ tus field will show "in​ queue".bin The APIC con​ trollers are up​ graded se​ ri​ ally so that the con​ troller clus​ ter is avail​ able dur​ ing the up​ grade.1. Upgrading the Switch Software Using the CLI .0(1.200) apic-1.0(1. or "com​ ple​ teok". Ex​ am​ ple: admin@apic1:~> firmware list Name : aci-apic-dk9.1(1d) 2 Upgrade the firmware on the controllers. Ex​ am​ ple: admin@apic1:~> firmware upgrade status Node-Id Role Current- Target- Upgrade- Progress-Percent Firmware Firmware Status (if inprogress) --------- ----------- ------------ -----------------. "in​ progress".bin Type : controller Version : 1.Upgrading and Downgrading Firmware 64 To up​ grade the APIC con​ troller soft​ ware using the CLI: 1 List the current software in the respository that was previously downloaded.1d.0(1. that con​ troller has prob​ a​ bly up​ graded and is re​ boot​ ing.

iso" to ".1d.1.869S1) n9000-11.bin Type : switch Version : 11. 3 Check the upgrade status for the switch. the APICs must have com​ pleted up​ grad​ ing and have a health state of Fully Fit.1(1d) completeok -----------------100 .bin".1(1d) The name changes from ". Ex​ am​ ple: admin@apic1:~> firmware upgrade switch node 101 ver_no.bin Firmware Installation on Switch Scheduled You must up​ grade each switch sep​ ar​ ately. To up​ grade the switch soft​ ware using the CLI 1 Check that the output of the following command appears like the output shown below.Upgrading and Downgrading Firmware 65 Upgrading the Switch Software Using the CLI Be​ fore you up​ grade the switches. with the correct version number: Ex​ am​ ple: admin@apic1:~> firmware list Name : aci-n9000-dk9. 2 Upgrade the switches.0(1.---------- 1017 leaf n9000-11. The output that appears from the following command will appear like the following sample: Ex​ am​ ple: admin@apic1:~> firmware upgrade status node node_id Node-Id Role Current- Target- Upgrade- Progress-Percent Firmware Firmware Status (if inprogress) --------- ----------- ------------------- -----------------.11.

you must re​ sume the paused sched​ uler. 4 Repeat Steps 2 and 3 for each additional switch. The fol​ low​ ing are the pos​ si​ ble up​ grade states for a node: • NotScheduled: No upgrade is currently scheduled for this node. . before declaring failure). when an up​ grade fail​ ure is de​ tected. • Queued: There is a currently active window (schedule) and the node is • Inprogress: Upgrade is currently in progress on this node.Upgrading and Downgrading Firmware 66 You can check the sta​ tus of all nodes at once. you will see "No" in the Wait​ ing for Clus​ ter Con​ ver​ gence field of the Con​ troller Firmware screen. • CompleteNOK: Upgrade failed on this node. The sched​ uler ex​ pects man​ ual in​ ter​ ven​ tion to debug any up​ grade fail​ ures. As the con​ troller and switches move through the up​ grade. you will see mes​ sages about the num​ ber of nodes queued and the num​ ber in the process of up​ grad​ ing. as well as how many have up​ graded suc​ cess​ fully. by en​ ter​ ing the firmware up​ grade sta​ tus com​ mand. When all the clus​ ters have con​ verged suc​ cess​ fully. • Scheduled: Upgrade is scheduled for this node. You view the Con​ troller Firmware screen on the GUI. This may take a while. Verifying Cluster Convergence You can mon​ it​ or the progress of the clus​ ter con​ ver​ gence after a sched​ uled main​ te​ nance. By de​ fault. the sched​ uler pauses. These mes​ sages are dis​ played in the Sta​ tus field. Once man​ ual in​ ter​ ven​ tion is com​ plete. Troubleshooting Failures During the Upgrade Process There is one sched​ uler per main​ te​ nance pol​ icy. which pre​ sents you with a se​ ries of mes​ sages dur​ ing the process of one clus​ ter con​ verg​ ing and then the next clus​ ter. • CompleteOK: Upgrade completed successfully. • Inretryqueue: Node is queued again for upgrade retry (5 attempts are made requesting permission to upgrade. and no more nodes in that group begin to up​ grade.

Until the controller cluster is healthy. If the sys​ tem takes longer than about 60 min​ utes for a switch to dis​ play “wait​ ing​ For​ Clus​ ter​ Health = no” in the API or "Wait​ ing for Clus​ ter Con​ ver​ gence" show​ ing "No" in the GUI. see the Trou​ bleshoot​ ing Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture doc​ um ​ent at the fol​ low​ ing URL: https://​ datacenter. then check the fol​ low​ ing: • Is the controller cluster healthy? The controller cluster must be healthy. that means the controller cluster is not healthy. If you see “waitingForClusterHealth = yes” in the API or "Waiting for Cluster Convergence" showing "Yes" in the GUI.Upgrading and Downgrading Firmware 67 If you no​ tice that switches are in the “queued” state. • Is the switch maintenance group paused? The group will be paused if any switch fails its upgrade. switches which have not already started their upgrade will be in “queued” state.​ io/​ aci-troubleshooting-book/​ . For ad​ di​ tional trou​ bleshoot​ ing pro​ ce​ dures.​ github. you should work through the steps for ver​ if​ y​ ing a pause in the sched​ uler.

.

69 Fabric Connectivity .

.

Access Policies Domains VLAN Pools AEPs Policy Types Interface Policies Switch Policies Interface Policy Groups Switch Policy Groups Interface Profiles Switch Profiles Best Practices • Adding New Devices to the Fabric Sample Configuration Creating VLAN Pools Create VLAN Pool Create Physical Domain Create an Attachable Access Entity Profile (AEP) Interface Policies Interface Policy Groups Interface Profile​ ​ Switch Profiles Reusability Sample vPC Creation Create VLAN Pool ​ Create a Physical Domain 71 .Fabric Connectivity Section Content • Understanding Fabric Policies • Fabric .

Fabric Connectivity 72 Create Access Entity Profile Interface Policies Switch Profile Create vPC domain Validate Operation of Configured vPC • Server Connectivity Cisco UCS B-Series Servers Standalone Rack Mount Servers or Non-Cisco Servers • Virtual Machine Networking Understanding VM Networking in ACI ACI VM Integration Workflow VMware Integration VMM Policy Model Interaction Publishing EPGS to a VMM Domain Connecting VMs to the EPG Port Groups on vCenter Verifying Virtual Endpoint Learning Verifying VM Endpoint Learning on the APIC from the CLI VMware Integration Use Case • Deploying the Application Virtual Switch Prerequisites Getting Started Install the AVS VIB Manual Installation DHCP Relay Attachable Access Entity Profile (AEP) and AVS VMM Domains for vCenter AVS Switching Modes Create the VMM Domain for AVS Verify AVS Deployment on vCenter Add vSphere Hosts to the AVS .

Fabric Connectivity Verify AVS on ESX VXLAN Load Balancing IGMP Snooping Policy for AVS • External Connectivity ​Extending ACI to External Layer 2 Extending an ACI Bridge Domain Outside of the Fabric Extending Endpoint Groups Outside the ACI Fabric Extending ACI to External Layer 3 Supported Routing Protocols Configure MP-BGP Spine Route Reflectors Layer 3 Integration Through Tenant Network with OSPF NSSA External Layer 3 for Multiple Tenants • Application Migration Use Case Extending the Network to ACI 73 .

.

This chap​ ter will re​ view the key ob​ jects in the ac​ cess poli​ cies sub​ sec​ tion of the fab​ ric tab – many of which are reusable. It is im​ por​ tant that other teams such as server teams un​ der​ stand these con​ cepts as well. and ac​ cess poli​ cies. The fab​ ric ac​ cess poli​ cies pro​ vide the fab​ ric with the base con​ fig​ ur​ a​ tion of the ac​ cess ports on the leaf switches. de​ vice dis​ cov​ ery and in​ ven​ tory man​ age​ ment. While many poli​ cies are reusable.Access Policies Domains EPGs are con​ sid​ ered the “who” in ACI. or whether or not to run pro​ to​ cols like LACP on leaf switch in​ ter​ faces is set. as they will be main​ tain​ ing these poli​ cies for the pur​ poses of in​ ter​ nal con​ nec​ tions be​ tween fab​ ric leaf nodes. as they will be in​ ter​ act​ ing with them. and basic man​ age​ ment poli​ cies. Fabric . For ex​ am​ ple. it is time to start cre​ at​ ing con​ nec​ tiv​ ity poli​ cies within the ACI fab​ ric. like port speed. and switch and port be​ hav​ ior. con​ nec​ tions to ex​ ter​ nal en​ ti​ ties such as servers. The fab​ ric tab in the APIC GUI is used to con​ fig​ ure sys​ tem-level fea​ tures in​ clud​ ing. Do​ mains and AEPs are also cre​ ated in the ac​ cess poli​ cies view. AEPs can be con​ sid​ ered the "where" and do​ mains can be thought . con​ fig​ ur​ ing do​ mains. con​ tracts are con​ sid​ ered the “what/when/why”. The fab​ ric pane is split into three sec​ tions: in​ ven​ tory. di​ ag​ nos​ tic tools. fab​ ric poli​ cies. but not lim​ ited to. and stor​ age ar​ rays. demon​ strate how to add and pre-pro​ vi​ sion switches. par​ tic​ ul​ arly in the case of their build processes for adding ad​ di​ tional ca​ pac​ ity.Fabric Connectivity 75 Understanding Fabric Policies Now that ACME has been pro​ vi​ sioned with ACI fab​ ric and in​ fra​ struc​ ture space has been con​ fig​ ured be​ tween the leaf and spine switches. net​ work​ ing equip​ ment. it is also key to un​ der​ stand the im​ pli​ ca​ tions of delet​ ing poli​ cies on the ACI fab​ ric. Un​ der​ stand​ ing how fab​ ric and ac​ cess poli​ cies con​ fig​ ure the fab​ ric is key for the ACME net​ work teams. The ac​ cess poli​ cies sub​ sec​ tion is split into fold​ ers sep​ ar​ at​ ing out dif​ fer​ ent types of poli​ cies and ob​ jects that af​ fect fab​ ric be​ hav​ ior. and walk through the steps and ob​ jects re​ quired when new de​ vices are added to the fab​ ric to ef​ fec​ tively op​ er​ ate an ACI fab​ ric. ac​ cess priv​ il​ eges. the in​ ter​ face poli​ cies folder is where port be​ hav​ ior is con​ fig​ ured.

AEPs At​ tach​ able Ac​ cess En​ tity Pro​ files (AEPs) can be con​ sid​ ered the "where" of the fab​ ric con​ fig​ ur​ a​ tion. VXLAN and mul​ ti​ cast ad​ dress pools are also con​ fig​ urable. For ex​ am​ ple. The fab​ ric op​ er​ at​ or cre​ ates the do​ mains. For​ ward​ ing de​ ci​ sions are still based on con​ tracts and the pol​ icy model. VLANs are in​ stan​ ti​ ated on leaf switches based on AEP con​ fig​ ur​ a​ tion. not sub​ nets and VLANs. the fab​ ric knows where the var​ io ​us de​ vices in the do​ main live and the APIC can push the VLANs and pol​ icy where it needs to be. and are used to group do​ mains with sim​ il​ ar re​ quire​ ments. an ex​ ter​ nal bridged do​ main could be used to con​ nect an ex​ ist​ ing switch trunked-up to a leaf switch. ex​ ter​ nal bridged do​ mains. AEPs are con​ fig​ ured under global poli​ cies. an ex​ ter​ nal routed do​ main could be used to con​ nect a WAN router to the leaf switch. Ex​ ter​ nal bridged do​ mains are used for Layer 2 con​ nec​ tions. Ex​ ter​ nal routed do​ mains are used for Layer 3 con​ nec​ tions.​ youtube. A do​ main is as​ so​ ci​ ated to a sin​ gle VLAN pool. For an in-depth white​ board ex​ pla​ na​ tion on do​ mains.76 Fabric Connectivity of as the “how” of the fab​ ric. and VMM do​ mains. . ex​ ter​ nal routed do​ mains. By group​ ing do​ mains into AEPs and as​ so​ ci​ at​ ing them. AEPs are tied to in​ ter​ face pol​ icy groups. Dif​ fer​ ent do​ main types are cre​ ated de​ pend​ ing on how a de​ vice is con​ nected to the leaf switch. Phys​ ic ​al do​ mains are gen​ er​ ally used for bare metal servers or servers where hy​ per​ vi​ sor in​ te​ gra​ tion is not an op​ tion. There are four dif​ fer​ ent do​ main types: phys​ ic ​al do​ mains. One or more do​ mains are added to an AEP.​ com/​ watch?​ v=_​ iQvoC9zQ_​ A VLAN Pools VLAN pools con​ tain the VLANs used by the EPGs the do​ main will be tied to. Do​ mains act as the glue be​ tween the con​ fig​ ur​ a​ tion done in the fab​ ric tab to the pol​ icy model and EPG con​ fig​ ur​ a​ tion found in the ten​ ant pane. check out the fol​ low​ ing video ti​ tled "How De​ vices Con​ nect to the Fab​ ric: Un​ der​ stand​ ing Cisco ACI Do​ mains": https://​ www. and the ten​ ant ad​ min​ is​ tra​ tors as​ so​ ci​ ate do​ mains to EPGs. For ex​ am​ ple.

CDPen​ abled. con​ fig​ ur​ ing vPC do​ mains. and pro​ files. Note the in​ ter​ face pol​ icy groups sim​ ply dic​ tate pol​ icy. These are also reusable ob​ jects as many de​ vices are likely to be con​ nected to ports that will re​ quire the same port con​ fig​ ur​ a​ tion. Interface Profiles In​ ter​ face pro​ files help tie the pieces to​ gether.and are also tied to the in​ ter​ face pol​ icy groups de​ scribed in the . and a 1GE link level pol​ icy must be cre​ ated for de​ vices con​ nected at that speed.Fabric Connectivity 77 Policy Types Most of the poli​ cies fold​ ers have sub​ fold​ ers. Ide​ ally. there should be a pol​ icy that dic​ tates CDP is dis​ abled. Max​ im ​iz​ ing reusabil​ ity of pol​ icy and ob​ jects makes day-to-day op​ er​ at​ ions ex​ po​ nen​ tially faster and eas​ ier to make large-scale changes. and vPC. poli​ cies should be cre​ ated once and reused when con​ nect​ ing new de​ vices to the fab​ ric. Pol​ icy groups do not ac​ tu​ ally spec​ ify where the pro​ to​ cols and port be​ hav​ ior should be im​ ple​ mented. For ex​ am​ ple. Switch Policies There are also poli​ cies for switches . and a pol​ icy that dic​ tates CDP is en​ abled . In​ ter​ face pro​ files con​ tain blocks of ports . There are three types of in​ ter​ face pol​ icy groups de​ pend​ ing on link type: in​ di​ vid​ ual. Just like poli​ cies. there should ide​ ally be a set of pol​ icy groups cre​ ated once and reused as new de​ vices are con​ nected to the fab​ ric.these can be reused as new de​ vices are con​ nected to the leaf switches. there may be a pol​ icy that dic​ tates 10GE. In​ ter​ face pol​ icy groups use the poli​ cies de​ scribed in the pre​ vi​ ous para​ graph to spec​ ify how links should be​ have.for ex​ am​ ple. Interface Policy Groups In​ ter​ face pol​ icy groups are tem​ plates to dic​ tate port be​ hav​ ior and are as​ so​ ci​ ated to an AEP. Port Chan​ nel. are are later tied to in​ ter​ face pol​ icy groups. Note that the ports on the leaf switches de​ fault to 10GE. under the in​ ter​ face poli​ cies folder there are fold​ ers for con​ fig​ ur​ a​ tion called poli​ cies. The "where" hap​ pens by as​ so​ ci​ at​ ing one or more in​ ter​ face pro​ files to a switch pro​ file. which are called ex​ plicit vPC pro​ tec​ tion groups in the APIC GUI. cov​ ered in the fol​ low​ ing para​ graphs. For ex​ am​ ple. Interface Policies First. Switch Policy Groups Switch pol​ icy groups allow lever​ ag​ ing of ex​ ist​ ing switch po​ lices like Span​ ning Tree and mon​ it​ or​ ing poli​ cies.in​ ter​ face se​ lec​ tors . pol​ icy groups. in​ ter​ face poli​ cies are cre​ ated to dic​ tate in​ ter​ face be​ hav​ ior. For ex​ am​ ple.

VLAN en​ cap​ su​ la​ tion or VXLAN en​ cap​ su​ la​ tion. switch pro​ files allow the se​ lec​ tion of one or more leaf switches and as​ so​ ci​ ate in​ ter​ face pro​ files to con​ fig​ ure the ports on that spe​ cific node. .1. such as e1/1. switch.78 Fabric Connectivity pre​ vi​ ous para​ graphs. a new con​ fig​ urable In​ ter​ face Pol​ icy was added to allow a per port-VLAN sig​ nif​ i​ cance. and cre​ ates a Port Chan​ nel or vPC if one has been con​ fig​ ured in the in​ ter​ face pol​ icy. To con​ nect de​ vices to the ACI fab​ ric we can use un​ tagged traf​ fic. This as​ so​ ci​ a​ tion pushes the con​ fig​ u​ ra​ tion to the in​ ter​ face. this is just an ar​ bi​ trary port. The fol​ low​ ing fig​ ure high​ lights the re​ la​ tion​ ship be​ tween the var​ i​ ous global. the pro​ file must be as​ so​ ci​ ated to a spe​ cific switch pro​ file (dis​ cussed in the next para​ graph) to con​ fig​ ure the ports. and in​ ter​ face poli​ cies: Relationships to allow a physical interface or interfaces to be attached to an EPG Layer 2 In​ ter​ face Pol​ icy In Cisco ACI ver​ sion 1. Again. Switch Profiles Lastly.

For example. there can be a set of interface policy groups for all VMware ESXi servers connected via 10GE vPCs. a policy that enables LACP in mode active could be called "LACPActive". it can be hard to remember what all the defaults are. However. It is expected that the port will flap when the Layer 2 interface policy changes between global and local. In ACI. • Policies Reuse policies whenever possible. For example. which is why policies should be clearly named to avoid making a mistake when adding new devices to the fabric. When naming policies. a port and VLAN pair (P. traffic will get affected. and 10GE port speed.V) is registered internally instead of just a VLAN encapsulation ID. • Per Port-VLAN limitations and considerations When per port-VLAN is used. For example. and a different set of interface policy groups for all bare metal servers running . but might help sim​ plify day-to-day op​ er​ at​ ion of the ACI fab​ ric. How​ ever. A set of interface policy groups should be created for each type of similar devices connected. These are not re​ quire​ ments and might not work for all en​ vi​ ron​ ments or ap​ pli​ ca​ tions. global con​ fig​ ur​ a​ tion as​ sumes that ten​ ants do not share leaf switches and there​ fore there is no VLAN over​ lap​ ping within the same leaf.Fabric Connectivity 79 In tra​ di​ tional net​ work​ ing one of the lim​ it​ a​ tions re​ lated to VLAN en​ cap​ su​ la​ tion is scal​ a-​ bil​ ity and re-us​ abil​ ity due to the limit of 4096 VLANs in net​ work​ ing de​ vices. There are many "default" policies out of the box. 1GE port speed. That is. use names that clearly describe the setting. there should be policies for LACP active/passive/off. with the de​ fault con​ fig​ ur​ a​ tion (global). This al​ lows ten​ ants to re-use VLAN en​ cap​ su​ la​ tion IDs thor​ ough the fab​ ric with​ out al​ low​ ing com​ mu​ ni​ ca​ tion be​ tween ten​ ants. Two EPGs belonging to a single Bridge Domain cannot share the same encapsulation ID on a given leaf switch. EPGs can use the same VLAN en​ cap​ su​ la​ tion as long as EPGs are bound to sep​ ar​ ate switches. Best Practices Cisco has es​ tab​ lished sev​ eral best prac​ tices for fab​ ric con​ fig​ ur​ a​ tion. This increases the consumption of hardware resources at a per switch level.

create a switch profile for each vPC pair (if using vPC). • Domains Build one physical domain per tenant for bare metal servers or servers without hypervisor integration requiring similar treatment. There are some cases where multiple AEPs may need to be configured to enable the infrastructure VLAN.Fabric Connectivity 80 1GE with CDP disabled. or to limit the scope of the presence of VLANs across the fabric. . Create a switch profile for each leaf switch individually. and additionally. each AEP will have its own set of interface policy groups. If a VMM domain needs to be leveraged across multiple tenants. a single VMM domain can be created and associated with all leaf ports where VMware ESXi servers are connected. such as overlapping VLAN pools. • AEPs Multiple domains can be associated to a single AEP for simplicity's sake. Since interface policy groups are tied to a single AEP. Build one physical domain per tenant for external connectivity.

Fabric Connectivity 81 Adding New Devices to the Fabric This sec​ tion will demon​ strate how to con​ fig​ ure ACI to re-use the fab​ ric ac​ cess poli​ cies. The fol​ low​ ing di​ a​ gram de​ picts how this re-us​ abil​ ity sim​ pli​ fies the op​ er​ a​ tion of the fab​ ric over time. The fol​ low​ ing di​ a​ gram re​ it​ er​ ates the ob​ ject re​ la​ tion​ ships: Object relationships Whereas a tra​ di​ tional com​ mand line in​ ter​ face on a switch gen​ er​ ally re​ quires a port-byport confugu​ ra​ tion. ACI al​ lows de​ f​ i​ n​ i​ tion of ob​ jects and poli​ cies that can be re-used. sim​ pli​ fy​ ing day-to-day op​ er​ a​ tion of the fab​ ric. As out​ lined in the pre​ vi​ ous sec​ tion. . This sec​ tion will walk through set​ ting up pro​ files from scratch. with a focus on how to re-use these pro​ files across the fab​ ric. these var​ i​ ous pro​ files are linked to​ gether and have de​ pen​ den​ cies. The re-us​ abil​ ity of these poli​ cies makes it pos​ si​ ble to repli​ cate the con​ fig​ u​ ra​ tion of a switch very eas​ ily.

an op​ er​ a​ tor can lever​ age pro​ files to stream​ line the op​ er​ a​ tion of adding de​ vices and man​ ag​ ing those de​ vices. In tra​ di​ tional net​ works. or spe​ cial​ ized ap​ pli​ ca​ tions. scripts.82 Fabric Connectivity Policy Re-use In any data cen​ ter. Lever​ ag​ ing the Cisco ACI pol​ icy model. when changes that im​ pact a large set of de​ vices need to be made. au​ toma​ tion be​ comes more and more crit​ i​ cal as it has a di​ rect im​ pact on the cost of busi​ ness op​ er​ a​ tions. Port Chan​ nel-con​ nected de​ vices. and vPC-con​ nected de​ vices from scratch. As the data cen​ ter size in​ creases. These can be net​ work man​ age​ ment tools. and the ver​ i​ fi​ ca​ tion steps to en​ sure proper con​ fig​ u​ ra​ tion. The fol​ low​ ing steps rep​ re​ sent the use case of adding a new bare metal server con​ nected to a leaf switch. This is what is de​ picted as the pol​ icy reuse in​ flec​ tion point in the pre​ vi​ ous di​ a​ gram. the con​ fig​ u​ ra​ tion of a cou​ ple of switches does not re​ quire many processes or au​ toma​ tion. . the op​ er​ a​ tor is faced with the cost of de​ sign​ ing processes to man​ age these de​ vices. and will in​ clude a re​ view of the ob​ jects as they are con​ fig​ ured. Sample Configuration The fol​ low​ ing sec​ tions will walk through sam​ ple con​ fig​ u​ ra​ tion of set​ ting up in​ di​ vid​ u​ ally con​ nected de​ vices. These are the steps to be taken in the APIC GUI when new de​ vices are con​ nected to the leaf switches to en​ sure the ac​ cess ports on the leaf switches have the right switch​ port con​ fig​ u​ ra​ tion.

or a spe​ cial con​ fig​ u​ ra​ tion on the NIC dri​ ver or the ker​ nel that al​ lows it to in​ tel​ li​ gently loadbal​ ance traf​ fic using an al​ go​ rithm. a sin​ gle server is dual homed to two dif​ fer​ ent switches for re​ dun​ dancy. as all links are actively forwarding • Your server configurations are simplified since the configurations simply appears as port-channels without the need for special software. This al​ lows for sev​ eral key ad​ van​ tages from a server de​ ploy​ ment per​ spec​ tive: • You can create resilient Layer 2 topologies based on link aggregation • You do not need STP • You have increased bandwidth. With​ out vPC's. from a driver or kernel-tuning standpoint . we pro​ vide a log​ i​ cal topol​ ogy that greatly sim​ pli​ fies server pro​ vi​ sion​ ing and man​ age​ ment. the ben​ e​ fits they pro​ vide and how vPC's in the ACI fab​ ric dif​ fer from how they are de​ ployed on Cisco Nexus switches run​ ning NX-OS soft​ ware. By con​ fig​ ur​ ing ports on two dif​ fer​ ent switches as the same port-chan​ nel and using an in​ ter-switch mes​ sag​ ing chan​ nel (such as the in​ ter-switch port-chan​ nel in the green box on the left hand side) to cover re​ dun​ dancy sce​ nar​ ios. vPC ex​ tends link ag​ gre​ ga​ tion to two sep​ a​ rate phys​ i​ cal switches. At a high level. This sec​ tion of the chap​ ter at​ tempts to clar​ ify at a high level what vPC's are. the server will likely use an ac​ tive-standby con​ fig​ u​ ra​ tion. which are a pop​ u​ lar server con​ nec​ tiv​ ity method​ ol​ ogy.Fabric Connectivity 83 Be​ fore get​ ting into the con​ fig​ u​ ra​ tion of vPC's. vPC Topology In the fig​ ure above. it is im​ por​ tant to un​ der​ stand what vPC's are and how they are dif​ fer​ ent from tra​ di​ tional meth​ ods of server con​ nec​ tiv​ ity.

Legacy connectivity compared to vPC The com​ po​ nents of a tra​ di​ tional vPC do​ main are il​ lus​ trated below: Traditional vPC topology . The fig​ ure below shows a sin​ gle tra​ di​ tional Layer 2 switch con​ nected to a VPC en​ abled Cisco switch pair. to pro​ vide sim​ i​ lar ben​ e​ fits. such as Cisco UCS fab​ ricin​ ter​ con​ nects.84 Fabric Connectivity vPCs can also be used to con​ nect other down​ stream de​ vices.

There is also a peer-keepalive link. The ACI fab​ ric greatly sim​ pli​ fies VPC con​ fig​ u​ ra​ tions.Fabric Connectivity 85 As il​ lus​ trated above. that is used to de​ ter​ mine peer live​ li​ ness to de​ tect a vPC peer-switch fail​ ure. in Cisco switch​ ing prod​ ucts run​ ning NX-OS soft​ ware. there is no re​ quire​ ment for set​ ting up vPC peer-links. The rich in​ ter​ con​ nec​ tiv​ ity be​ tween fab​ ric nodes makes it un​ likely that peers will have an ac​ tive path be​ tween them. vPC con​ fig​ u​ ra​ tions need to be done man​ u​ ally by the op​ er​ a​ tor and re​ quire a pair of ded​ i​ cated "in​ ter-switch" links also called a peer-link. There are also no keepalives being sent on the man​ age​ ment ports. typ​ i​ cally on the out-of-band man​ age​ ment port. Mak​ ing con​ fig​ u​ ra​ tion changes in such sce​ nar​ ios with​ out the con​ fig-sync fea​ ture en​ abled may lead to sce​ nar​ ios where there are mis​ matched vPC pa​ ra​ me​ ters be​ tween the vPC pri​ mary and the vPC sec​ ondary switches that may cause par​ tial con​ nec​ tiv​ ity loss dur​ ing the change it​ self if a type-1 in​ con​ sis​ tency is de​ tected. vPC topology in ACI The key dif​ fer​ ences to note here are that rel​ a​ tive to tra​ di​ tional vPC de​ sign. The fab​ ric it​ self serves as the peer-link. Note that at​ tempt​ ing to cable a leaf switch to an​ other leaf switch will lead to a "wiring mis​ match" fault in the GUI and re​ sult in a black​ listed port that will have to be man​ u​ ally re​ cov​ ered. .

peers assume master-slave roles. it is assumed to have crashed. Hence if the peer switch becomes unreachable. A list of type-1 parameters used for consistency checking for a given vPC domain specific to the ACI fabric are listed below. • Global type-1 parameters: STP • Interface type-1 parameters: STP: Only BPDU Guard is configurable EthPM Port speed Duplex mode Port mode MTU Native VLAN PCM: Channel mode. static vs lacp LACP: Lag ID The fol​ low​ ing di​ ag ​rams il​ lus​ trate how the ACI fab​ ric for​ wards traf​ fic from a vPC do​ main to a non-vPC con​ nected host in the fab​ ric. • In the ACI fabric. • Role election still happens. Slave switch brings down all its vPC ports. . and vice-versa. it is very unlikely that all the redundant paths between vPC peers fail at the same time. The slave switch does not bring down vPC links. the slave switch brings down all its vPC links if the MCT goes down.Fabric Connectivity 86 The fol​ low​ ing are some other key be​ hav​ ioral changes to vPC as it ap​ plies to the ACI fab​ ric rel​ at​ ive to clas​ sic vPC that are im​ por​ tant for op​ er​ at​ ors to un​ der​ stand: • Configurations are automatically synchronized to avoid an error-free configuration by the APIC which is the central point of control for all configurations in the ACI fabric. • In traditional vPC solution. • Role is used in case of vpc type-1 consistency failure.

2 S3 does a table lookup and routes with vPC Virtual IP (VIP). 4 S3 delivers the pkt to locally attached host H2. As men​ tioned in the pre​ vi​ ous sec​ tion. 2 S1 does a table lookup and routes with IP of S3. H1 -> H2 1 H1 sends a pkt towards H2 on one of its PC link (S1 in this case). Creating VLAN Pools In this ex​ am​ ple. con​ fig​ ur​ ing newly-con​ nected bare metal servers first re​ quires cre​ ation of a phys​ i​ cal do​ main and then as​ so​ ci​ a​ tion of the do​ main to a VLAN pool. VLAN pools de​ fine a range of VLAN IDs that will be used by the EPGs.Fabric Connectivity 87 vPC forwarding Uni​ cast packet flow H2 -> H1 1 H2 sends a pkt towards H1 on its link to S3. 3 Spine switch routes to S3. 4 S2 delivers the pkt to locally attached host H1. 3 Spine switch sees multiple routes for VIP and picks one of them (S2 in this case). .

choose Fabric > Access Policies. Each server will be tag​ ging using 802. Note: When dynamic allocation is selected. b. and NVGRE tags. VXLAN. provide a description for the VLAN pool. c. ACI has to know be​ fore​ hand how to clas​ sify pack​ ets into the dif​ fer​ ent EPGs. Encapsulation normalization Create VLAN Pool 1 On the menu bar. choose Pools > VLAN.1Q VLAN tags. Optionally. phys​ i​ cal port IDs. Define a meaningful name for the VLAN pool. it is im​ por​ tant to un​ der​ stand that the de​ f​ i​ n​ i​ tion of VLANs as they per​ tain to the leaf switch ports is uti​ lized only for iden​ ti​ fi​ ca​ tion pur​ poses. vir​ tual port IDs.1Q or VXLAN en​ cap​ su​ la​ tion.Fabric Connectivity 88 The servers are con​ nected to two dif​ fer​ ent leaf nodes in the fab​ ric. the ACI fab​ ric can also act as a gate​ way be​ tween dis​ parate en​ cap​ su​ la​ tion types such as un​ tagged traf​ fic. using iden​ ti​ fiers like VLANs. 4 In the Create VLAN Pool dialog box. the APIC selects VLANs from the pool dynamically. 3 In the Work pane. When a packet ar​ rives ingress to a leaf switch in the fab​ ric. As de​ picted in the fol​ low​ ing fig​ ure. 2 In the Navigation pane. perform the following actions: a. There are two allocation modes: dynamic or static. VXLAN VNIDs. NVGRE. This is common in VMM integration mode where the actual . choose Actions > Create VLAN Pool. The range of VLANs used in the con​ fig​ u​ ra​ tion ex​ am​ ple is 100-199. 802. The leaf switches nor​ mal​ ize the traf​ fic by strip​ ping off tags and reap​ ply​ ing the re​ quired tags on fab​ ric egress. In ACI.

Select the VLAN pool you just created. as the ten​ ant ad​ min​ is​ tra​ tor is the one who as​ so​ ci​ ates do​ mains to EPGs. b.538-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-vlan-pool" ownerKey="" ownerTag="" status="" uid="8131"> <fvnsRtVlanNs childAction="" lcOwn="local" modTs="2015-02-25T11:35:33. XML Ob​ ject <fvnsVlanInstP allocMode="static" childAction="" configIssues="" descr="" dn="uni/infra/vlanns-[bsprint-vlan-pool]-static" lcOwn="local" modTs="2015-02- 23T15:58:33. choose Actions > Create Physical Domain.007-08:00" rn="rtinfraVlanNs-[uni/phys-bsprint-PHY]" status="" tCl="physDomP" tDn="uni/physbsprint-PHY"/> <fvnsEncapBlk childAction="" descr="" from="vlan-100" lcOwn="local" modTs="2015-02- 23T15:58:33. while the do​ mains are cre​ ated under the fab​ ric tab. When con​ fig​ ur​ ing in this order. Note multiple ranges can be added to a single pool. choose Physical and External Domains > Physical Domains. The encap blocks are used to define the range of VLANs in the VLAN pool. The cre​ ation of the AEP and its as​ so​ ci​ at​ ion will be cov​ ered later in this sec​ tion. choose Fabric > Access Policies. 1 On the menu bar. Remember it's the EPG itself which policies are applied to. only the pro​ file name and the VLAN pool are con​ fig​ ured. Define a meaningful name for the profile. In the Create Physical Domain dialog box. Static allocation is typically used when the pool will be referenced from a static source like a static path binding for an EPG for use with bare metal servers. . The do​ main also ties the fab​ ric con​ fig​ ur​ a​ tion to the ten​ ant con​ fig​ ur​ a​ tion. 3 4 In the Work pane. perform the following actions: a.365-08:00" rn="rtinfraVlanNs-[uni/l2dom-JC-L2-Domain]" status="" tCl="l2extDomP" tDn="uni/l2domJC-L2-Domain"/> <fvnsRtVlanNs childAction="" lcOwn="local" modTs="2015-02-23T16:13:22.Fabric Connectivity 89 VLAN ID is not important. d.538-08:00" name="" rn="from-[vlan-100]-to-[vlan-199]" status="" to="vlan199" uid="8131"/> </fvnsVlanInstP> Create Physical Domain A phys​ ic ​al do​ main acts as the link be​ tween the VLAN pool and the Ac​ cess En​ tity Pro​ file (AEP). 2 In the Navigation pane.

4 In the Create Attached Entity Profile dialog box.906-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-PHY" ownerKey="" ownerTag="" status="" uid="8131"> <infraRsVlanNs childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:22. The con​ fig​ ur​ a​ tion for an AEP is straight​ for​ ward. 5 Click Next.Fabric Connectivity 90 XML Ob​ ject <physDomP childAction="" configIssues="" dn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:21. enter a description. choose Actions > Create Attached Entity Profile. c. 6 Click Submit. d. 3 In the Work pane. choose Fabric > Access Policies. choose Global Policies > Attached Acess Entity Profle.945-08:00" rn="rtdomP-[uni/infra/attentp-bsprint-AEP]" status="" tCl="infraAttEntityP" tDn="uni/infra/attentp-bsprint-AEP"/> </physDomP> Create an Attachable Access Entity Profile (AEP) The AEP links the phys​ ic ​al do​ main and its VLAN Pool to the in​ ter​ face poli​ cies. 1 On the menu bar. Click + to associate the domain to the AEP. b. 2 In the Navigation pane. Define the a meaningful name for the profile.065-08:00" rType="mo" rn="rsvlanNsDef" state="formed" stateQual="none" status="" tCl="fvnsAInstP" tDn="uni/infra/vlanns-[bsprint-vlan-pool]-static" tType="mo"/> <infraRtDomP childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.065-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsvlanNs" state="formed" stateQual="none" status="" tCl="fvnsVlanInstP" tDn="uni/infra/vlanns[bsprint-vlan-pool]-static" tType="mo" uid="8131"/> <infraRsVlanNsDef childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:22. perform the following actions: a. Optionally. Select the physical domain that was previously configured. .

980-08:00" rn="rtattEntP-[uni/infra/funcprof/accportgrp-bsprint-AccessPort]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-bsprint-AccessPort"/> <infraRsDomP childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 25T11:35:33.874-08:00" monPolDn="uni/fabric/monfab-default" rn="nscont" status=""> <infraRsToEncapInstDef childAction="" deplSt="" forceResolve="no" lcOwn="local" modTs="2015-02-23T16:13:52.874-08:00" rn="dompcont" status=""> <infraAssocDomP childAction="" dompDn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:52.570-08:00" monPolDn="uni/fabric/monfab-default" profileDn="" rn="source[uni/l2dom-JC-L2-Domain]" status=""/> <fabricCreatedBy childAction="" creatorDn="uni/phys-bsprint-PHY" deplSt="" domainDn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:52.961-08:00" monPolDn="uni/fabric/monfab-default" profileDn="" rn="source-[uni/phys-bsprint-PHY]" status=""/> </infraRsToEncapInstDef> </infraContNS> <infraRtAttEntP childAction="" lcOwn="local" modTs="2015-02-24T11:59:37. de​ fine the in​ ter​ face pro​ files and show​ case the re-us​ abil​ ity of the fab​ ric poli​ - .Fabric Connectivity 91 XML Ob​ ject <infraAttEntityP childAction="" configIssues="" descr="" dn="uni/infra/attentp- bsprint-AEP" lcOwn="local" modTs="2015-02-23T16:13:52.570-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsdomP- [uni/l2dom-JC-L2-Domain]" state="formed" stateQual="none" status="" tCl="l2extDomP" tDn="uni/l2dom-JC-L2-Domain" tType="mo" uid="8754"/> <infraRsDomP childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:52.961-08:00" rn="assocdomp-[uni/phys-bsprint-PHY]" status=""/> <infraAssocDomP childAction="" dompDn="uni/l2dom-JC-L2-Domain" lcOwn="local" modTs="2015-02-25T11:35:33.961-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsdomP[uni/phys-bsprint-PHY]" state="formed" stateQual="none" status="" tCl="physDomP" tDn="uni/phys-bsprint-PHY" tType="mo" uid="8131"/> </infraAttEntityP> Create Interface Policies Next.570-08:00" rn="assocdomp-[uni/l2dom-JC-L2-Domain]" status=""/> </infraContDomP> <infraContNS childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.874-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-AEP" ownerKey="" ownerTag="" status="" uid="8131"> <infraContDomP childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.961-08:00" monPolDn="uni/fabric/monfabdefault" rType="mo" rn="rstoEncapInstDef-[allocencap-[uni/infra]/encapnsdef- [uni/infra/vlanns-[bsprint-vlan-pool]-static]]" state="formed" stateQual="none" status="" tCl="stpEncapInstDef" tDn="allocencap-[uni/infra]/encapnsdef[uni/infra/vlanns-[bsprint-vlan-pool]-static]" tType="mo"> <fabricCreatedBy childAction="" creatorDn="uni/l2dom-JC-L2-Domain" deplSt="" domainDn="uni/l2dom-JC-L2-Domain" lcOwn="local" modTs="2015-02- 25T11:35:33.

In​ ter​ face poli​ cies can be re-used as needed by dif​ fer​ ent in​ ter​ face pro​ file de​ f​ in ​i​ t​ ion re​ quire​ ments. 3 In the Work pane.081- 08:00" rn="rtinfraHIfPol-[uni/infra/funcprof/accportgrp-UCS-1G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-1G-PG"/> <fabricRtHIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. Leaf switch ports default to 10GE. choose Interface Policies > Policies > CDP Interface.693-08:00" name="1G-Auto" ownerKey="" ownerTag="" speed="1G" status="" uid="15374"> <fabricRtHIfPol childAction="" lcOwn="local" modTs="2015-01-14T06:48:48. perform the following actions: a. perform the following actions: a. . Create Link Level Policies Link level poli​ cies dic​ tate con​ fig​ ur​ a​ tion like the speed of ports. Define a meaningful name for the policy such as 'CDP-Enable'. 4 In the Create Link Level Policy dialog box. e. choose Actions > Create CDP Interface Policy. choose Interface Policies > Policies > Link Level. but ide​ ally there are al​ ready poli​ cies in place that can sim​ ply be se​ lected. c.Fabric Connectivity 92 cies. choose Actions > Create Link Level Policy. 3 In the Work pane. Change the de-bounce interval if required. 4 In the Create CDP Interface Policy dialog box. d. Optionally. XML Ob​ ject <fabricHIfPol autoNeg="on" childAction="" descr="" dn="uni/infra/hintfpol-1G-Auto" lcOwn="local" linkDebounce="100" modTs="2015-01-14T06:47:15. choose Fabric > Access Policies. Define the meaningful name for the policy. This sec​ tion will il​ lus​ trate cre​ ation of new pro​ files.331- 08:00" rn="rtinfraHIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example"/> </fabricHIfPol> Create a CDP Interface Policy 1 On the menu bar. 5 Click Submit. provide a description for the policy. choose Fabric > Access Policies. Select the interface speed. 1 On the menu bar. Select the auto negotiation mode for the interface. b. 2 In the Navigation pane. 2 In the Navigation pane.

c. 5 Click Submit. c.154- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" status="" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-01-14T06:48:48. Optionally.470-08:00" name="CDP-Enable" ownerKey="" ownerTag="" status="" uid="15374"> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-01-14T07:23:54. Choose the receive state.957- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-UCS-10G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-10G-PG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. choose Fabric > Access Policies. Choose either admin state enabled or disabled. provide a description for the policy. b. Choose the transmit state.331- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example"/> </cdpIfPol> Create an LLDP Interface Policy 1 On the menu bar. provide a description for the policy. 2 In the Navigation pane. 4 In the Create LLDP Interface Policy dialog box. choose Interface Policies > Policies > LLDP Interface. choose Actions > Create LLDP Interface Policy. XML Ob​ ject <cdpIfPol adminSt="enabled" childAction="" descr="" dn="uni/infra/cdpIfP-CDP-Enable" lcOwn="local" modTs="2015-01-14T06:47:25. 5 Click Submit. Define a meaningful name for the policy.Fabric Connectivity 93 b. d. 3 In the Work pane. perform the following actions: a. Optionally.980- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-bsprint-AccessPort]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-bsprintAccessPort"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11.081- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-UCS-1G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-1G-PG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-24T11:59:37. ​ .

154- 08:00" rn="rtinfraLldpIfPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG" status="" /> <lldpRtLldpIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11.3ad) that al r​ a​ tion re​ quire​ ments an op​ tional ne​ go​ ti​ at​ ion pro​ to​ col to be run on Port Chan​ nel in​ ter pre-emp​ tively de​ fined to be used as needed for the var​ io ​us con​ fig​ u in the data cen​ ter. perform the following actions: a. .Fabric Connectivity 94 XML Object descr="" <lldpIfPol adminRxSt="enabled" adminTxSt="enabled" childAction="" dn="uni/infra/lldpIfP-LLDP-Enable" lcOwn="local" modTs="2015-02-11T07:40:35. choose Actions > Create LACP Policy. These can be Link Ag​ gre​ ga​ tion Con​ trol Pro​ to​ col is part of an IEEE spec​ if​ i​ ca​ tion (802. Optionally. 1 On the menu bar. . c. LACP must also be enabled on the server or other connected device. 2 In the Navigation pane. 5 Click Submit. b. 4 In the Create LACP Policy dialog box. provide a description for the policy. Note if LACP is enabled on the leaf switch. specify the minimum and maximum number of links in the Port Channel.331- 08:00" rn="rtinfraLldpIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example" </lldpIfPol> l​ ows Create an LACP Interface Policy ​ faces. Optionally. d. Select the LACP mode required for the server. choose Interface Policies > Policies > LACP 3 In the Work pane. Define a meaningful name for the policy.664-08:00" status="" name="LLDP-Enable" ownerKey="" ownerTag="" status="" uid="15374"> /> <lldpRtLldpIfPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. choose Fabric > Access Policies.

d.Fabric Connectivity 95 XML Ob​ ject <lacpLagPol childAction="" ctrl="fast-sel-hot-stdby. change the priority. c. It is a com​ mon best prac​ tice to en​ able BPDU guard on in​ ter​ faces con​ nected to servers. choose Fabric > Access Policies. 5 Click Submit. 3 In the Work pane. If required. 1 On the menu bar.susp-individual" descr="" dn="uni/infra/lacplagp-LACP-Active" lcOwn="local" maxLinks="16" minLinks="1" modTs="2015-02-24T11:58:36. . choose Actions > Create Spanning Tree Interface Policy. Optionally.547-08:00" mode="active" name="LACP-Active" ownerKey="" ownerTag="" status="" uid="8131"> <lacpRtLacpPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11.154- 08:00" rn="rtinfraLacpPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" status="" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG"/> </lacpLagPol> Create an LACP Member Profile (optional) Op​ tion​ ally. The Span​ ning Tree in​ ter​ face pol​ icy sim​ ply de​ fines the port be​ hav​ ior. the LACP mem​ ber pro​ file pro​ vides the abil​ ity to pro​ vide pri​ or​ ity spec​ if​ i​ ca​ tions to mem​ bers of an LACP group. choose Actions > Create LACP Member Policy. Note: ACI does not run Span​ ning Tree on the fab​ ric be​ tween the leaves and spines. choose Interface Policies > Policies > LACP Member. choose Interface Policies > Policies > Spanning Tree Interface. Create a Spanning Tree Interface Policy (optional) The Span​ ning Tree pol​ icy dic​ tates the be​ hav​ ior of south​ bound leaf port Span​ ning Tree fea​ tures. provide a description for the policy.graceful-conv. 3 In the Work pane. change the transmit rate. If required. 2 In the Navigation pane. 2 In the Navigation pane. 4 In the Create LACP Member Policy dialog box. Define a meaningful name for the policy. b. choose Fabric > Access Policies. 1 On the menu bar. perform the following actions: a.

Fabric Connectivity

96

4

In the Create Spanning Tree Interface Policy dialog box, perform the following
actions:
a. Define a meaningful name for the policy.
b. Optionally, provide a description for the policy.
c. Enable BPDU filter and/or BPDU guard.

5

Click Submit.

Create a Storm Control Policy (optional)

A traf​
fic storm oc​
curs when pack​
ets flood the LAN, cre​
at​
ing ex​
ces​
sive traf​
fic and de​
grad​
ing net​
work per​
for​
mance. The traf​
fic storm con​
trol fea​
ture can be used to pre​
vent dis​
rup​
tions on ports by a broad​
cast, mul​
ti​
cast, or uni​
cast traf​
fic storm on phys​
i​
cal in​
ter​
faces.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policies > Storm Control.

3

In the Work pane, choose Actions > Create Storm Control Policy.

4

In the Create Storm Control Policy dialog box, perform the following actions:
a. Define a meaningful name for the policy.
b. Optionally, provide a description for the policy.
c. Specify how the control policy is to be applied, either through percentage of
the total bandwidth or as a packet per second definition that matches the
requirement for the data center

5

Click Submit.

Creating a Layer 2 Interface Policy to enable per port-VLAN

1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policies > L2 Interface.

3

In the Work pane, choose Actions > Create L2 Interface Policy.

4

In the Create L2 Interface Policy dialog box, perform the following actions:
a. Give the L2 Interface name and an optional description.
b. Select VLAN scope to Port Local scope to enable per port-VLAN.

Create Interface Policy Groups
The in​
ter​
face pol​
icy groups com​
prise the in​
ter​
face poli​
cies as a func​
tional group that is
as​
so​
ci​
ated to an in​
ter​
face. The fol​
low​
ing di​
ag
​ram shows how pre​
vi​
ously cre​
ated items
are grouped under the pol​
icy group.

Fabric Connectivity

97

Policies contained in a policy group
Once all the in​
ter​
face poli​
cies have been de​
fined, the in​
di​
vid​
ual poli​
cies can be brought
to​
gether to form a pol​
icy group that will be linked to the in​
ter​
face pro​
file. The pol​
icy
group is de​
fined from a mas​
ter de​
f​
i​
n​
i​
tion that en​
com​
passes being one of the fol​
low​
ing:

Access Policy Group

Port Channel Policy Group

VPC Policy Group

Create Access Port Policy Group

The ac​
cess port pol​
icy is de​
fined for an in​
di​
vid​
ual link (non-Port Chan​
nel or vPC).
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create Access Policy Group.

4

In the Create Access Policy Group dialog box, perform the following actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Use the profiles created previously that are relevant for this policy group.

5

Click Submit.

Fabric Connectivity

98

Create Port Channel Interface Policy Group

Port Chan​
nel​
ing also load-bal​
ances traf​
fic across the phys​
ic
​al in​
ter​
faces that are mem​
bers of the chan​
nel group. For every group of in​
ter​
faces that needs to be con​
fig​
ured
into a port chan​
nel, a dif​
fer​
ent pol​
icy group has to be cre​
ated. This pol​
icy group de​
fines
the be​
hav​
iour. For ex​
am​
ple, if ports 1/1-4 are to be con​
fig​
ured into one port chan​
nel, and ports 1/5-8 into a sep​
ar​
ate port chan​
nel, each of those groups would re​
quire
the cre​
ation of a sep​
ar​
ate pol​
icy group.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create PC Interface Policy Group.

4

In the Create PC Interface Policy Group dialog box, perform the following
actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Select the policies created previously that are relevant for this PC policy
group.

5

Click Submit.

Create VPC Interface Policy Group

Note: This ob​
ject must be unique for each VPC cre​
ated.
A vir​
tual PortChan​
nel (vPC) al​
lows links that are phys​
ic
​ally con​
nected to two dif​
fer​
ent
de​
vices to ap​
pear as a sin​
gle Port Chan​
nel to a third de​
vice. In the world of ACI, pairs of
leaf switches may be con​
fig​
ured in a vPC do​
main so that down​
stream de​
vices can be
ac​
tive-ac​
tive dual-homed.
For every group of in​
ter​
faces that are to be con​
fig​
ured into a vPC, a dif​
fer​
ent in​
ter​
face
pol​
icy group needs to be cre​
ated. The vPC pol​
icy group con​
tains both the de​
f​
in
​i​
t​
ion for
the be​
hav​
iour of the port chan​
nel, and the iden​
ti​
fier. For ex​
am​
ple, if ports 1/1-4 are to
be con​
fig​
ured into one vPC across two switches, and ports 1/5-8 into a sep​
ar​
ate vPC
across two switches, each of those groups would re​
quire the de​
f​
in
​i​
t​
ion of a sep​
ar​
ate
pol​
icy group.
Note: For vPC you will also re​
quire a unique vPC do​
main de​
f​
in
​i​
t​
ion be​
tween the two
paired switches. More de​
tails to fol​
low.
1

On the menu bar, choose Fabric > Access Policies.

Fabric Connectivity

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create VPC Interface Policy Group.

4

In the Create VPC Interface Policy Group dialog box, perform the following

99

actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Choose the policies created previously that are relevant for this vPC policy
group.
5

Click Submit.

Interface Profile
The in​
ter​
face pro​
file in ACI links the pol​
icy groups that de​
fine how the in​
ter​
face is
going to be​
have, and as​
signs them to spe​
cific ports via the con​
cept of in​
ter​
face se​
lec​
tor. In turn, the in​
ter​
face pro​
file is even​
tu​
ally tied to a switch pro​
file to spec​
ify which
leaf switch the ref​
er​
enced ports should be con​
fig​
ured. As we con​
tinue the process of
defin​
ing the port pro​
files, you can ob​
serve how we have started at the bot​
tom of this
ob​
ject tree con​
fig​
ur​
ing the dif​
fer​
ent pro​
files. The pur​
poses for all these in​
di​
vid​
ual poli​
cies that tie to​
gether is to max​
i​
mize pol​
icy re-use.

Interface Profile links to Interface Selector and Interface Policy Group
The di​
a​
gram in the pre​
vi​
ous sec​
tion pro​
vides a vi​
sual de​
scrip​
tion of what can be ac​
com​
plished by group​
ing the poli​
cies that have been de​
fined under the in​
ter​
face pro​
file,
and then as​
signed to ports with in​
ter​
face se​
lec​
tors and the ac​
cess port pol​
icy groups.

100 Fabric Connectivity

Create Interface Profile

The in​
ter​
face pro​
file is com​
posed of two pri​
mary ob​
jects. The in​
ter​
face se​
lec​
tor and
the ac​
cess port pol​
icy group. The in​
ter​
face se​
lec​
tor de​
fines what in​
ter​
faces will apply
the ac​
cess port pol​
icy. The ports that share the same at​
trib​
utes can then be grouped
under the same in​
ter​
face pro​
file.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

Next, add the in​
ter​
face se​
lec​
tors that are as​
so​
ci​
ated to this in​
ter​
face pro​
file.
Create Interface Selector

1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

3
4

In the Work pane, choose Actions > Create Access Port Selector.
In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter the interface IDs.
d. Choose the interface policy group that should be associated to these ports.

5

Click Submit.

Create Interface Profile for Port Channel

If a server has two or more up​
links to a leaf switch, the links can be bun​
dled into a Port
Chan​
nel for re​
siliency and load dis​
tri​
bu
​t​
ion. In order to con​
fig​
ure this in ACI, cre​
ate an
in​
ter​
face pol​
icy group of type Port Chan​
nel to bun​
dle the in​
ter​
faces. Dif​
fer​
ent Port
Chan​
nels re​
quire dif​
fer​
ent pol​
icy groups.

Fabric Connectivity 101

Port Channel Policy Group
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

Next, cre​
ate an in​
ter​
face port se​
lec​
tor. Since you will be con​
fig​
ur​
ing a Port Chan​
nel,
the op​
er​
a​
tor will add all of the in​
ter​
faces re​
quired in the Port Chan​
nel in​
ter​
face. In this
ex​
am​
ple in​
ter​
faces Eth​
er​
net 1/1-2 will be con​
fig​
ured in one Port Chan​
nel and in​
ter​
faces Eth​
er​
net 1/3-4 will be con​
fig​
ured in an​
other Port Chan​
nel.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

3

In the Work pane, choose Actions > Create Access Port Selector.

102 Fabric Connectivity

4

In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter interface IDs for the first port channel.
d. Choose the interface policy group.

5

Click Submit.

6

Repeat this process for the second Port Channel (if you have another Port
Channel to add).

Create an Interface Profile for Virtual Port Channel

A vPC do​
main is al​
ways made up of two leaf switches, and a leaf switch can only be a
mem​
ber of one vPC do​
main. In ACI, that means that the de​
f​
i​
n​
i​
tion of the poli​
cies is sig​
nif​
i​
cant be​
tween the two switches. The same pol​
icy can be reused be​
tweeen the two
switches, and through the vPC do​
main the pair as​
so​
ci​
a​
tion can be de​
fined. vPC Switch
do​
main mem​
bers should be taken into con​
sid​
er​
a​
tion when con​
fig​
ur​
ing firmware main​
te​
nance groups. By keep​
ing this in mind, firmware up​
grades should never im​
pact both
vPC switch peers at the same time. More de​
tails on this can be found in the Up​
grad​
ing
and Down​
grad​
ing Firmware sec​
tion.
For this rea​
son, a switch pro​
file that would rep​
re​
sent two sep​
a​
rate switch IDs needs to
be cre​
ated. The re​
la​
tion​
ship of these switches to the two ports in the same pol​
icy
group is de​
fined through the in​
ter​
face pro​
file.

vPC Policy Group

Fabric Connectivity 103

The same process would have to be re​
peated for every grouped in​
ter​
face on each side
that will be a mem​
ber of the vPC.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

6

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

7
8

In the Work pane, choose Actions > Create Access Port Selector.
In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter interface IDs.
d. Select of the interface policy group to be used for the vPC port behavior.

9

Click Submit.

Create a vPC Domain for Virtual Port Channel

When con​
fig​
ur​
ing a vPC, there is one ad​
di​
tional step to be con​
fig​
ured once to put two
leaf switches into the same vPC do​
main.

Creating a vPC Domain
1

On the menu bar, choose Fabric > Access Policies.

104 Fabric Connectivity

2

In the Navigation pane, choose Switch Policies > VPC Domain > Virual Port
Channel default.

3

In the Work pane, choose Actions > Explicit VPC Protection Group.

4

In the Explicit VPC Protection Group dialog box, perform the following actions:
a. Define a meaningful name for the vPC domain.
b. Provide a unique ID to represent the vPC domain.
c. Choose the first switch you want to be part of the vPC domain.
d. Choose the second switch you want to be part of the vPC domain.

5

Click Submit.

Switch Profiles
A switch pro​
file groups all the in​
ter​
face pro​
files that de​
fine the be​
hav​
ior of its re​
spec​
tive switch ports. A switch pro​
file could be the de​
f​
in
​i​
t​
ion of a sin​
gle switch or it could
be the de​
f​
in
​i​
t​
ion of mul​
ti​
ple switches. As a best prac​
tice, there should be a switch pro​
file for each leaf switch, and an ad​
di​
tional switch pro​
file for each vPC do​
main pair of
leaf switches.
The in​
ter​
face pro​
files that you have cre​
ated can be as​
so​
ci​
ated to the switch through a
sin​
gle switch pro​
file or they can be as​
so​
ci​
ated through dif​
fer​
ent switch pro​
files. If you
have var​
io
​us racks that are iden​
ti​
cal in the way the in​
ter​
face ports are con​
fig​
ured, it
could be ben​
e​
fi​
cial to uti​
lize the same switch pro​
file. This would make it pos​
si​
ble to
mod​
ify the con​
fig​
ur​
a​
tion of many switches dur​
ing op​
er​
at​
ions with​
out hav​
ing to con​
fig​
ure each switch in​
di​
vid​
ua
​lly.

Reusability
The ca​
pa​
bil​
ity of pol​
icy reusabil​
ity is cru​
cial to re-em​
pha​
size from an op​
er​
at​
ional per​
spec​
tive. If a pro​
file has been de​
fined to con​
fig​
ure a port as 1GB speed for ex​
am​
ple, that
pro​
file can be reused for many in​
ter​
face pol​
icy groups. When look​
ing at whole switch
con​
fig​
ur​
a​
tions, the re-us​
abil​
ity of the pro​
file can be ex​
tended to sim​
plify data cen​
ter
op​
er​
at​
ions and en​
sure com​
pli​
ance. The fol​
low​
ing fig​
ure il​
lus​
trates the reusabil​
ity of
pro​
files across racks of switches.

each of the top of rack switches is based on the same switch pro​ file. If a pro​ file has been reused across many de​ vices. It is also im​ por​ tant to be aware of the im​ pli​ ca​ tion of delet​ ing pro​ files. Sample vPC Creation The fol​ low​ ing pro​ ce​ dure demon​ strates what a vPC bringup looks like and also API POST con​ fig​ u​ ra​ tion as​ s​ es​ ment of the vPC. make sure to check where it is being used be​ fore you delete the pro​ file or pol​ icy.Fabric Connectivity 105 Policy re-use at scale In the pre​ vi​ ous di​ a​ gram. If all these racks are con​ fig​ ured in the same fash​ ion (mean​ ing they are wired in the same way) the same poli​ cies could be reused by sim​ ply as​ sign​ ing the switches to the same switch pro​ file. It would then in​ herit the pro​ file tree and be con​ fig​ ured the exact same way as the other racks. The fol​ low​ ing topol​ ogy will be con​ fig​ ured: .

choose Actions > Create VLAN Pool. perform the following actions: a. provide a description for the pool. Note: For this example the pool will be from VLAN 100 to VLAN 199. b. 4 In the Create VLAN Pool dialog box. perform the following actions: a. Optionally. choose Fabric > Access Policies. 3 In the Work pane. . choose Actions > Create Physical Domain. 2 In the Navigation pane. Click Static Allocation for the allocation mode.xml CLI :: moquery -c fvnsVlanInstP Create a Physical Domain 1 On the menu bar. choose Pools > VLAN. 3 4 In the Work pane. c. b.106 Fabric Connectivity Sample Topology Create VLAN Pools 1 On the menu bar. REST :: /api/node/class/fvnsVlanInstP. 2 In the Navigation pane. choose Physical and External Domains > Physical Domains. Define a meaningful name for the pool. Choose the VLAN pool that you previously created. choose Fabric > Access Policies. Define a meaningful name for the domain. In the Create Physical Domain dialog box.

c. Define a meaningful name for the pool. . c. 5 Click Submit.xml CLI :: moquery -c physDomP Create Access Entity Profile 1 On the menu bar. choose Actions > Create Link Level Policy. perform the following actions: a. Provide a unique ID to represent the AEP. perform the following actions: a. choose Interface Policies > Policies > Link Level. Choose the auto negotiation for the interface. choose Actions > Create Attached Entity Profile.xml CLI :: moquery -c infraAttEntityP Interface Policies Create Link Level Policy 1 On the menu bar. d.Fabric Connectivity 107 REST :: /api/node/class/physDomP. provide a description for the policy. Optionally. Define a meaningful name for the AEP. 2 In the Navigation pane. choose Fabric > Access Policies. choose Fabric > Access Policies. 6 Click Submit. 5 Click Next. 2 In the Navigation pane. b. 3 4 In the Work pane. In the Create Attached Entity Profile dialog box. REST :: /api/node/class/infraAttEntityP. 3 In the Work pane. Choose the physical domain that you previously created. 4 In the Create Link Level Policy dialog box. Choose the speed to match the interface requirement. choose Global Policies > Attachable Access Entity Profles. Click + to add a domain that will be associated to the interfaces. b. d.

choose Interface Policies > Policy Groups. However. provide a description for the pool. provide a description for the policy group. c. 2 In the Navigation pane. c.108 Fabric Connectivity REST :: /api/node/class/fabricHIfPol. perform the following actions: a. In mode click on Active. choose Interface Policies > Policies > LACP. choose Actions > Create LACP Policy. choose Fabric > Access Policies. Note: LACP is recommended for vPCs. 5 Click Submit. ensure LACP is configured on the device connected to the leaf switch. choose Actions > Create vPC Interface Policy Group.xml CLI :: moquery -c infraAccBndlGrp . Optionally. 3 In the Work pane. Choose a Link Level Policy. REST :: /api/node/class/lacpLagPol. e. 2 In the Navigation pane. d. Optionally. 4 In the Create LACP Policy dialog box. 4 In the Create vPC Interface Policy Group dialog box. Define a meaningful name for the policy group. Choose an AEP to associate the policy group to. REST :: /api/node/class/infraAccBndlGrp. Choose an LACP Policy. b. 3 In the Work pane. 5 Click Submit. choose Fabric > Access Policies. Define a meaningful name for the policy. b.xml CLI :: moquery -c fabricHIfPol Create LACP Policy 1 On the menu bar.xml CLI :: moquery -c lacpLagPol Create vPC Interface Policy Group 1 On the menu bar. perform the following actions: a.

choose Interface Policies > Profiles > Profiles > ACIVPC-int-profile. c. b. In the Create Access Port Selector dialog box. choose Switch Policies > Profiles. Select an interface policy group. ii. 4 In the Create Interface Profile dialog box. 3 In the Work pane. perform the following actions: a. provide a description for the policy group. provide a description for the profile. Define a meaningful name for the profile. i. Optionally. perform the following actions: a. Define a meaningful name for the policy group. provide a description. b. perform the following actions: a. Optionally. In switch selectors click on the + symbol. choose Interface Policies > Profiles. choose Fabric > Access Policies. 7 8 In the Work pane.Fabric Connectivity 109 Create an Interface Profile 1 On the menu bar. 5 Click Submit. choose Actions > Create Interface Profile. choose Actions > Create Access Port Selector.xml CLI :: moquery -c infraAccPortP Create a Switch Profile 1 On the menu bar. choose Fabric > Access Policies. Define a meaningful name for the profile. Optionally. 3 In the Work pane. b. . Select the proper interfaces. 2 In the Navigation pane. choose Actions > Create Switch Profile. d. 4 In the Create Switch Profile dialog box. 2 In the Navigation pane. REST :: /api/node/class/infraAccPortP. c. 9 Click Submit. Name: 103-104 (example node numbers). 6 In the Navigation pane. Blocks: Select switch 103 and switch 104.

c. 3 In the Work pane. REST :: /api/node/class/infraNodeP. 2 In the Navigation pane. Provide a unique ID for the vPC domain.110 Fabric Connectivity 5 Click Next. 3 In the Work pane. 6 Select the previously created interface selector. 2 In the Navigation pane. choose Fabric > Access Policies.xml CLI :: moquery -c fabricExplicitGEp Validate Operation of Configured vPC 1 On the menu bar. 7 Click Finish. If configured correctly. perform the following actions: b. e. choose Actions > Explicit VPC Protection Group. Select the second switch ID that is part of this vPC pair: 104. a. f.xml CLI :: moquery -c infraNodeP Create vPC domain 1 On the menu bar. 4 Click Submit. choose Switch Policies > VPC Domain > Virual Port Channel default. REST :: /api/node/class/fabricExplicitGEp. d. In the Explicit VPC Protection Groups section. Select the first switch ID that is part of this vPC pair: 103. Define a meaningful name for the vPC domain. choose POD 1 > Interfaces > vPC Interfaces. In the Explicit VPC Protection Group dialog box. click + to create a vPC protection group. there will be a table that shows the status of the vPC interface. choose Fabric > Inventory. . You can also val​ id ​ate the op​ er​ at​ ion of the vPC di​ rectly from the CLI of the switch it​ self. the status should be displayed and you should see successful establishment of the vPC domain.

----------.Fabric Connectivity 111 If you con​ nect to the con​ sole or the out of band man​ age​ ment in​ ter​ face of the leaf node you should be able to see the sta​ tus with the com​ mand show vpc.-------------------------------------------------up - vPC status ---------------------------------------------------------------------id Port Status Consistency Reason Active vlans 1 Po1 up - -- ---- -----.local vPC is down. Leaf-3# show vpc Legend: (*) . forwarding via vPC peer-link vPC domain id : 100 vPC keep-alive status : Disabled Peer status : peer adjacency formed ok Configuration consistency status Per-vlan consistency status Type-2 consistency status : success : success : success vPC role : primary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------id -1 Port ---- Status Active vlans -----.-----success success ------------ .

xml <polUni> <fvTenant descr="" dn="uni/tn-Cisco" name="Cisco" ownerKey="" ownerTag=""> <fvAp descr="" name="CCO" ownerKey="" ownerTag="" prio="unspecified"> .Interface Policy Group --> <infraFuncP> <infraAccBndlGrp name="intPolicyGroupforVPC" lagT="node"> <infraRsAttEntP tDn="uni/infra/attentp-AttEntityProfileforCisco"/> <infraRsCdpIfPol tnCdpIfPolName="CDP_ON" /> <infraRsLacpPol tnLacpLagPolName="LACP_ACTIVE" /> <infraRsHIfPol tnFabricHIfPolName="10GigAuto" /> </infraAccBndlGrp> </infraFuncP> </infraInfra> </polUni> https://{{hostName}}/api/node/mo/uni. URL: https://{{apic-ip}}/api/policymgr/mo/.Switch Selector --> <infraNodeP name="switchProfileforVPC_201"> <infraLeafS name="switchProfileforVPC_201" type="range"> <infraNodeBlk name="nodeBlk" from_="201" to_="201"/> </infraLeafS> <infraRsAccPortP tDn="uni/infra/accportprof-intProfileforVPC_201"/> </infraNodeP> <infraNodeP name="switchProfileforVPC_202"> <infraLeafS name="switchProfileforVPC_202" type="range"> <infraNodeBlk name="nodeBlk" from_="202" to_="202"/> </infraLeafS> <infraRsAccPortP tDn="uni/infra/accportprof-intProfileforVPC_202"/> </infraNodeP> <!-.xml <polUni> <infraInfra> <!-.112 Fabric Connectivity The fol​ low​ ing REST API call can be used to build vPCs and at​ tach vPCs to sta​ tic port bind​ ings.Interface Profile --> <infraAccPortP name="intProfileforVPC_201"> <infraHPortS name="vpc201-202" type="range"> <infraPortBlk name="vpcPort1-15" fromCard="1" toCard="1" fromPort="15" toPort="15"/> <infraRsAccBaseGrp tDn="uni/infra/funcprof/accbundle-intPolicyGroupforVPC"/> </infraHPortS> </infraAccPortP> <infraAccPortP name="intProfileforVPC_202"> <infraHPortS name="vpc201-202" type="range"> <infraPortBlk name="vpcPort1-1" fromCard="1" toCard="1" fromPort="1" toPort="1"/> <infraRsAccBaseGrp tDn="uni/infra/funcprof/accbundle-intPolicyGroupforVPC"/> </infraHPortS> </infraAccPortP> <!-.

Fabric Connectivity 113 <fvAEPg descr="" matchT="AtleastOne" name="Web" prio="unspecified"> <fvRsPathAtt encap="vlan-1201" instrImedcy="immediate" mode="native" tDn="topology/pod-1/protpaths-201-202/pathep-[vpc201-202]” /> </fvAEPg> <fvAEPg descr="" matchT="AtleastOne" name="App" prio="unspecified"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/protpaths-201-202/pathep-[vpc201-202]” /> </fvAEPg> </fvAp> </fvTenant> </polUni> .

.

When being con​ nected to the ACI fab​ ric.Fabric Connectivity 115 Server Connectivity Server con​ nec​ tiv​ ity is nec​ es​ sary for all ap​ pli​ ca​ tion work​ loads to func​ tion prop​ erly on the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. in​ di​ vid​ ual link and fab​ ric switch fail​ ures are mit​ i-​ gated to main​ tain a higher ex​ pected up time. See the Con​ fig​ ur​ ing Man​ age​ ment Pro​ to​ cols chap​ ter for more in​ for​ ma​ tion. the type of Layer 2 con​ nec​ tion needed on the Fab​ ric In​ ter​ con​ nect fac​ ing ports must be de​ ter​ mined first. as well as some third party servers that all need to be con​ nected to the ACI fab​ ric. When con​ nect​ ing UCS to the ACI fab​ ric. If a sup​ ported hy​ - . The fab​ ric con​ nec​ tiv​ ity re​ quire​ ments that are dic​ tated by the server in​ fra​ struc​ ture must be care​ fully con​ sid​ ered. These poli​ cies are all gov​ erned by in​ ter​ face pol​ icy groups. the Link Level Dis​ cov​ ery Pro​ to​ col (LLDP) must be dis​ abled. has sev​ eral dif​ fer​ ent mod​ els of servers in their data cen​ ters. If the work​ load is a bare metal server. fab​ ric ac​ cess poli​ cies must be pro​ vi​ soned to match these re​ quire​ ments. Cisco Dis​ cov​ ery Pro​ to​ col (CDP) must be en​ abled on any ACI fab​ ric in​ ter​ faces that are con​ nect​ ing to UCS Fab​ ric In​ ter​ con​ nects. You can pre-pro​ vi​ sion these poli​ cies as part of the ini​ tial man​ age​ ment tasks. Cisco UCS B-Series Servers If UCS B-se​ ries Fab​ ric In​ ter​ con​ nects are being con​ nected to your ACI fab​ ric and Cisco UCS. In this sce​ nario. A best prac​ tice is to lever​ age a vir​ tual pri​ vate cloud (vPC) to con​ nect the UCS en​ vi​ ron​ ment so as to cre​ ate a multichas​ sis ether​ chan​ nel. see the Adding New De​ vices to the Fab​ ric sec​ tion. the kind of traf​ fic ex​ pected out of the server links needs to be de​ ter​ mined. ACME Inc. such as Cisco UCS B and C se​ ries. Standalone Rack Mount Servers or Non-Cisco Servers Any non-UCS server ar​ chi​ tec​ ture can also be con​ nected di​ rectly to the ACI fab​ ric or to a Cisco Nexus 2000 Fab​ ric Ex​ ten​ der (FEX). traf​ fic can be clas​ si​ fied on a per port basis and as​ so​ ci​ ated AEPs and EPGs can be mapped ap​ pro​ pri​ ately to match the en​ cap​ su​ lated traf​ fic. For more in​ for​ ma​ tion on the process needed to con​ fig​ ure links to UCS as ei​ ther a vPC or a tra​ di​ tional port chan​ nel. In the case of Cisco Uni​ fied Com​ put​ ing Sys​ tem (UCS).

are still true.116 Fabric Connectivity per​ vi​ sor is to be used. The key is to map the ex​ pected traf​ fic clas​ si​ fi​ ca​ tion to the ports that are con​ nected to the server in​ fra​ struc​ ture. Re​ stric​ tions that are pre​ sent in NX-OS mode such that non-host-fac​ ing ports are not sup​ ported. . a Vir​ tual Ma​ chine Man​ ager (VMM) do​ main must be prop​ erly con​ fig​ ured. all host-fac​ ing ports are treated the same way as if they were di​ rectly at​ tached to the ACI fab​ ric. Ports must only be con​ nected to hosts. and con​ nec​ tiv​ ity to any other net​ work de​ vice will not func​ tion prop​ erly. When uti​ liz​ ing a FEX. and then as​ so​ ci​ ated with the cor​ re​ spond​ ing ports on the fab​ ric as a hy​ per​ vi​ sor that is fac​ ing through EPG and AEP map​ ping. Uti​ liz​ ing a FEX is an al​ ter​ na​ tive way to con​ nect host de​ vices into the ACI fab​ ric.

such as VMM and Layer 4 to Layer 7 services. A pool is a shared resource and can be consumed by multiple domains. and Microsoft Systems Center Virtual Machine Manager (SCVMM). VMware vShield. A virtual machine controller administrator provides an APIC administrator with a virtual machine controller authentication credential. The ACI pro​ vides the abil​ ity to man​ age both vir​ tual and phys​ ic ​al end​ points with the same set of poli​ cies. such as VMware vCenter. such as VLAN and VXLAN IDs.The EPG has a relation to the domain. A leaf switch does not support overlapping VLAN pools. • A pool represents a range of traffic encapsulation identifiers. The pool contains a range of encapsulated VLANs and VXLANs. A VMware vCenter domain can associate only to a dynamic pool. Multiple controllers can use the same credentials. The Application Policy Infrastructure Controller (APIC) communicates with the VMM to publish network policies that are applied to virtual workloads. multiple controllers of the same type can use the same credential. The encapsulation must be within the range of a pool that is associated with a domain with which the EPG is associated. the user defines the interface and the encapsulation. and multicast addresses. • The two types of VLAN-based pools are as follows: Dynamic pools . For static EPG deployment. Static pools .Managed internally by the APIC to allocate VLANs for endpoint groups (EPGs). . Different overlapping VLAN pools must not be associated with the same attachable entity profile (AEP).Fabric Connectivity 117 Virtual Machine Networking Understanding VM Networking in ACI One of the most com​ mon uses of the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) will be to help man​ age and de​ ploy ap​ pli​ ca​ tions in vir​ tual en​ vi​ ron​ ments. The fol​ low​ ing list de​ scribes some vir​ tual ma​ chine man​ ager (VMM) sys​ tem terms: • A virtual machine controller is an external VMM entity. • Credentials represent the authentication credentials to communicate with virtual machine controllers. and the domain has a relation to the pool. This is the pool type that is required for VMM integration. This chap​ ter will look at var​ io ​us op​ er​ at​ ional tasks that will be per​ formed through​ out the daily op​ er​ at​ ions.

VMware do​ mains can be de​ ployed. ACI VM Integration Workflow ACI VM Integration Workflow For de​ tailed in​ for​ ma​ tion on how to de​ ploy the VMware vSphere Dis​ trib​ uted Switch with the Cisco APIC. see the fol​ low​ ing doc​ u​ ments: • Cisco APIC Getting Started Guide VMware Integration When in​ te​ grat​ ing ACI into your VMware in​ fra​ struc​ ture you have two op​ tions for de​ ploy​ ing net​ work​ ing. lever​ ag​ ing the VMware vSphere Dis​ trib​ uted Vir​ tual Switch (DVS) or the Cisco Ap​ pli​ ca​ tion Vir​ tual Switch (AVS). This in​ cludes cre​ at​ ing the VLANs on Uni​ fied Com​ put​ ing Sys​ tem (UCS). Both pro​ vide sim​ i​ lar basic vir​ tual net​ work​ ing func​ tion​ al​ ity. the VLAN range must also be cre​ ated on any in​ ter​ me​ di​ ate de​ vices. such as tra​ di​ tional switches or blade switches. the AVS pro​ vides . how​ ever.118 Fabric Connectivity When cre​ at​ ing dy​ namic VLAN pools for VMM in​ te​ gra​ tion.

​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ getting-started/​ video/​ cisco_​ apic_​ create_​ vcenter_​ domain_​ profile_​ using_​ gui. This serves as a ref​ er​ ence for the ways the var​ i​ ous poli​ cies are re​ lated to each other.Fabric Connectivity 119 ad​ di​ tional ca​ pa​ bil​ i​ ties.​ cisco. such as VXLAN and mi​ croseg​ men​ ta​ tion sup​ port. has cho​ sen to lever​ age the ad​ di​ tional fea​ tures pro​ vided by AVS. please refer to the fol​ low​ ing doc​ u​ ments: http://​ www. ACME Inc. VMM Policy Model Interaction . For or​ ga​ ni​ za​ tions in​ ter​ ested in using the stan​ dard DVS pro​ vided by VMware.​ html VMM Policy Model Interaction Shown below are some of the var​ i​ ous ACI poli​ cies which are in​ volved with set​ ting up VM In​ te​ gra​ tion.

4 In the Work pane. 3 In the Navigation pane. a do​ main bind​ ing within the ten​ ant EPG must be cre​ ated. 1 On the menu bar. choose the Tenant_Name. This provides the best resource usage in the fabric by only deploying policies to Leaf nodes when endpoints assigned to this EPG are connected. and in the Network Connection dropdown box select the Port Group which corresponds to your EPG. Cisco recommends keeping the default option of On Demand. There is no communication delay or traffic loss by keeping the default selections. It should display in the format of [TENANT|APPLICATION_PROFILE|EPG|VMM_DOMAIN_PROFILE] . For the Deployment & Resolution Immediacy. choose Actions > Add VM Domain Association. see the Ten​ ants sec​ tion. Note: The EPG will now be available as a Port Group to your VMM.120 Fabric Connectivity Publishing EPGs to a VMM Domain This sec​ tion will de​ tail how to pub​ lish an ex​ ist​ ing end​ point group (EPG) to a Vir​ tual Ma​ chine Man​ ager (VMM) do​ main. choose the VMM Domain Profile that you previously created. For de​ tails on how to cre​ ate EPGs. For an EPG to be pushed to a VMM do​ main. 3 Click on the Network Adapter. a. choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > Application_EPG_Name > Domains (VMs and Bare-Metals). 2 From the Host and Clusters view. right click on your Virtual Machine and select "Edit Settings". 6 Click Submit. Connecting VMs to the EPG Port Groups on vCenter 1 Connect to your vCenter using the VI Client. choose Tenants > ALL TENANTS 2 In the Work pane. 5 In the Add VM Domain Association dialog box.

se​ lect​ ing "Save As" and look​ ing at the XML ob​ ject. you should ver​ ify the APIC has learned your vir​ tual end point. 2 In the Work pane. and one for the Class Name of 'fvCEp' (Fab​ ric Vec​ tor Client End​ point) moquery -c fvCEp --dn uni/tn-<TENANT_NAME>/ap-<APP_PROFILE_NAME>/epg-<EPG_NAME> You can de​ ter​ mine the DN of your EPG by right click​ ing on the EPG in the GUI. All endpoints either virtual or physical will be displayed. 3 In the Navigation pane. 4 In the Work pane.Fabric Connectivity 121 If you do not see your ACI EPG in the Net​ work Con​ nec​ tion list. choose Tenants > ALL TENANTS. From here you should be able to find your Virtual Machine by filtering the "Learning Source" column for rows with values of "Learned VMM". From this file you will see the DN entry for the par​ tic​ ul​ ar EPG: . Verifying Virtual Endpoint Learning Once the VMs are con​ nected to the ap​ pro​ pri​ ate port group/EPG. One for the Dis​ tin​ guished Name (DN) name of your EPG. it means one of the fol​ low​ ing: • The VM is running on a host which is not attached to the Distributed Switch • There may be a communication between your APIC and vCenter either through managed by the APIC. Verifying VM Endpoint Learning on the APIC from the CLI You can ver​ ify the same info as above from the CLI by using the 'mo​ query' (Man​ aged Ob​ ject Query) com​ mand and adding two fil​ ters. 1 On the menu bar. choose the Tenant_Name. Note: The current tab should display CLIENT ENDPOINTS. choose the Operational tab. choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > Application_EPG_Name. the OOB or INB management network.

as well as pro​ vid​ ing un​ lim​ ited Layer 2 mo​ bil​ ity for all the VM hosts within the ACI fab​ ric.122 Fabric Connectivity <imdata totalCount="1"><fvAEPg uid="15374" triggerSt="triggerable" status="" scope="2588672" prio="unspecified" pcTag="49159" name="epg-od" monPolDn="uni/tn-common/monepg-default" modTs="2015-02-06T06:46:24.vmm lcOwn : local mac : 00:50:56:BB:8C:6A mcastAddr : not-applicable modTs : 2015-02-06T06:48:52.729+11:00" matchT="AtleastOne" lcOwn="local" dn="uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od" descr="" configSt="applied" configIssues="" childAction=""/></imdata> Next.CEp name : 00:50:56:BB:8C:6A childAction : dn : uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od/cep-00:50:56:BB:8C:6A encap : vlan-211 id : 0 idepdn : ip : 10. Rather than trunk​ ing VLANs on a per server basis. . use this DN with the mo​ query to re​ turn the list of client En​ points for this EPG: admin@apic1:~> moquery -c fvCEp --dn uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od Total Objects shown: 1 # fv.10.10 lcC : learned. the net​ work team de​ cides to lever​ age a new method​ ol​ ogy to be more agile and lever​ age the on-de​ mand pro​ vi​ sion​ ing of re​ sources where and when they are needed.10.229+11:00 rn : cep-00:50:56:BB:8C:6A status : uid : 0 uuid : VMware Integration Use Case A VMWare ad​ min​ is​ tra​ tor in ACME re​ quests the net​ work team to trunk a set of VLANs down to the ESX hosts to pro​ vide con​ nec​ tiv​ ity to their DVS switches.

VMs are al​ lowed to move any​ where within the ACI fab​ ric with no re​ stric​ tions other than those im​ posed by vCen​ ter. How​ ever. and get​ ting en​ hanced vis​ i​ bil​ ity and teleme​ try into both the vir​ tual and phys​ i​ cal en​ vi​ ron​ ments. Note: The APIC does not au​ to​ mat​ ic ​ally move VM​ NICs into the port-group.Fabric Connectivity 123 To do so. the APIC ad​ min​ is​ tra​ tor pro​ ceeds to con​ fig​ ure VMM in​ te​ gra​ tion in the APIC GUI by pro​ vid​ ing the vCen​ ter cre​ den​ tials to APIC. and still reap the ad​ van​ tages of the Layer 2-any​ where ACI fab​ ric. This al​ lows VMware ad​ mins to main​ tain con​ trol and move vir​ tual NICs into these port-groups on de​ mand. APIC is au​ to​ mat​ ic ​ally in​ formed of the VM move and then up​ dates the end​ point track​ ing table to allow seam​ less com​ mu​ ni​ ca​ tion.800). The APIC also con​ fig​ ures VLAN IDs on the leaf-switches as needed. As the VMware admin pro​ vi​ sions ESX hosts and se​ lects the ap​ pro​ pri​ ate port-groups for VMs. Dur​ ing a vMo​ tion event. the APIC dy​ nam​ ic ​ally com​ mu​ ni​ cates with vCen​ ter to make EPGs avail​ able through port groups. ACME chose VMM in​ te​ gra​ tion as the pre​ ferred de​ ploy​ ment model as it is the most ef​ fec​ tive method of break​ ing down or​ ga​ ni​ za​ tional chal​ lenges. doing on-de​ mand re​ source al​ lo​ ca​ tion. Once this is de​ cided. APIC dy​ nam​ ic ​ally pro​ vi​ sions all EPGs and makes them avail​ able to the ESX hosts as a port-group. This is their dy​ namic VLAN pool. It is im​ por​ tant to note that ACME can still choose to de​ ploy tra​ di​ tional VLAN trunk​ ing down to VMware DVS switches by sta​ t​ i​ cally pro​ vi​ sion​ ing EPGs on a per-port basis. the net​ work ad​ mins work with the VMware ad​ mins to de​ cide on a range of VLANs that will be pro​ vided dy​ nam​ ic ​ally by APIC to the ESX hosts that need them. . They de​ cide on an un​ used VLAN range of (600 .

.

new ver​ sions of the AVS will be re​ leased to in​ clude new fea​ tures and im​ prove​ ments. As of this pub​ li​ ca​ tion there are two VIBs to sup​ port vSphere ver​ sions 5. • One or more vSphere hosts are available for deployment to the AVS.5 (vSphere 5. These can be down​ loaded from CCO at the fol​ low​ ing lo​ ca​ tion: Down​ loads Home > Prod​ ucts > Switches > Vir​ tual Net​ work​ ing > Ap​ pli​ ca​ tion Vir​ tual Switch . • A dynamic VLAN pool has been created with enough VLAN IDs to accommodate one VLAN per EPG you plan on deploying to each VMM domain. and available.2. • A DNS server policy has been configured to enable connection to a VMM using a hostname.2.1. Al​ ways refer to the AVS re​ lease notes to con​ firm if any spe​ cial con​ sid​ er​ at​ ions may exist. The AVS pack​ age for ei​ ther ver​ sion will in​ clude vSphere In​ stal​ la​ tion Bun​ dles (VIBs). Just like any soft​ ware. The ini​ tial AVS soft​ ware re​ leased was ver​ sion 4.1 and 5. Getting Started The AVS soft​ ware was de​ signed to op​ er​ ate in​ de​ pen​ dently of the APIC soft​ ware ver​ sion. • VMware vCenter is installed.0 is not sup​ ported). Refer to the ACI Ecosys​ de​ sired ver​ sion of AVS is com​ pat​ ib ​le with the APIC and vSphere ver​ sions being run.1. Each ver​ sion of AVS soft​ ware in​ cludes the VIB files for all sup​ ported vSphere ver​ sions.Fabric Connectivity 125 Deploying the Application Virtual Switch Prerequisites • All switch nodes have been discovered by the fabric • INB or OOB management connectivity is configured. configured. This al​ lows ei​ ther de​ vice to be up​ graded in​ de​ pen​ dently. fol​ lowed by tem Com​ pat​ i​ bil​ ity List doc​ um ​ent to en​ sure your ver​ sion 5.

2.2. 2 SSH into the vSphere host on which the AVS VIB is to be installed.3.3.1.0-3.2.2.2. Right click on the desired datastore and select Browse. the AVS soft​ ware must be in​ stalled in vSphere.2.1.1.0-3.1.3.2.1. 3 Install or upgrade the VIB using the esxcli command: To in​ stall the AVS VIB: esxcli software vib install -v /<path>/<vibname> --maintenance-mode --no-sig-check .1.1 Bundle cross_cisco-vem-v172-5. all of which are dis​ cussed in Cisco Ap​ pli​ ca​ tion Vir​ tual stal​ la​ tion Guide. The easiest way to copy the VIB to the host is to leverage the VMware VI Client.3.vib cross_cisco-vem-v165-4. From here.1.2.1.126 Fabric Connectivity The VIBs can be iden​ ti​ fied as fol​ lows: AVS 4.3.vib AVS 5. re​ ferred to as the Vir​ tual Eth​ er​ net Mod​ ule (VEM).0-3.1.1.1 Bundle cross_cisco-vem-v165-4. For a few hosts.2.2.0-3.1.2.1.vib cross_cisco-vem-v172-5. This can be achieved in a va​ ri​ ety of ways.VIB can be uploaded directly to the host's datastore.vib Install the AVS VIB Be​ fore set​ ting up the AVS con​ fig​ ur​ a​ tion on the APIC. but for 10+ Switch In​ hosts it may be eas​ ier to lever​ age the Vir​ tual Switch Up​ date Man​ ager (VSUM) to help au​ to​ mate the in​ stal​ la​ tion process. Manual Installation 1 Copy the VIB file to a host.3.2. navigate to the Host > Configuration > Storage > Datastore_X. this can eas​ ily be done man​ ua ​lly.

2.2.1.0-3.0 Releasebuild-1623387 4 Confirm the VEM is loaded and running.5. This VTEP in​ ter​ face will be used for the Open​ Flex con​ trol chan​ nel and/or the VXLAN tun​ nel source be​ tween the VTEP and ACI fab​ ric.Fabric Connectivity 127 To Up​ grade an ex​ ist​ ing AVS VIB: esxcli software vib update -v /<path>/<vibname> --maintenance-mode --no-sig-check A sam​ ple out​ put is shown below: # esxcli software vib install -v /vmfs/volumes/datastore1/cross_cisco-vem-v1725.0-3.1.3.2.1. This will allow the AVS to cre​ ate a vmk Vir​ tual Tun​ nel End​ point (VTEP) in​ ter​ face on each host.3. .2.1. # vem status VEM modules are loaded Switch Name Num Ports Used Ports Configured Ports MTU Uplinks vSwitch0 3072 6 128 1500 VMNIC0 VEM Agent (vemdpa) is running DHCP Relay The first task is to cre​ ate a DHCP relay pol​ icy in the infra ten​ ant on the APIC.1 VIBs Removed: VIBs Skipped: /vmfs/volumes/53cab6da-55209af3-0ef2-24e9b391de3e # vem version Running esx version -1623387 x86_64 VEM Version: 5.0-3.vib --maintenance-mode --no-sig-check Installation Result Message: Operation finished successfully.1. Reboot Required: false VIBs Installed: Cisco_bootbank_cisco-vem-v172-5.2.1.3.1 VSM Version: System Version: VMware ESXi 5.3.3.2.3.1.

choose infra > Networking > Protocol Policies > DHCP > Relay Policies. 4 5 In the Work pane. choose Tenants > ALL TENANTS. . 3 In the Navigation pane. The way APIC would do it is through over the infra vlan. 3 In the Navigation pane. such as "avs-dhcp-relay-pol". b. perform the following actions: a. For this rea​ son it is crit​ ic ​al to ex​ tend the Infra VLAN from a leaf through to each vSphere host that will host the AVS. choose infra > Networking > Bridge Domains > default > DHCP Relay Labels. choose Actions > Create DHCP Relay Policy. b. choose the infra.​ 0/​ 16). Create the Infra DHCP Relay Policy Cre​ ate a DHCP relay pol​ icy that will setup the DHCP server (the APIC in this case) and a label using the Infra Ten​ ant and de​ fault EPG. perform the following actions: a. 6 Click Submit. In the Create DHCP Relay Policy dialog box. choose Actions > Create DHCP Relay Policy. Provide a name for the Relay policy. 2 In the Work pane.​ 0. Change the Scope to TENANT. Create the DHCP Relay Label 1 On the menu bar. STEPS 1 On the menu bar. In the Create DHCP Relay Policy dialog box. 2 In the Work pane. choose the infra. choose Tenants > ALL TENANTS. 6 Click Submit. 4 5 In the Work pane.​ 0. choose the DHCP Relay Policy that you created previously.128 Fabric Connectivity If the de​ fault TEP Ad​ dress pool was used dur​ ing ini​ tial setup (10. From the Name drop-down list. Leave other fields blank and click OK. The APIC IP ad​ dresses au​ to​ mat​ ic ​ally func​ tion as the DHCP providers and DO NOT need to be ex​ plic​ itly added. this will allow the AVS VTEP in​ ter​ faces to pull an ad​ dress from this pool.

From the next page of the wizard. but this guide will de​ tail cre​ at​ ing the AEP sep​ ar​ ately first. such as speed/ne​ go​ ti​ at​ ion. and CDP. 3 In the Work pane. Es​ sen​ tially the In​ ter​ face Pol​ icy Group is a col​ lec​ tion of In​ ter​ face poli​ cies which de​ fine In​ ter​ faces Se​ lec​ tors and prop​ er​ ties. Fill in the AEP wizard information then click Next. 2 In the Navigation pane. choose Actions > Create Attachable Access Entity Profile. . Re​ fer​ ring back to the "VMM Pol​ icy Model In​ ter​ ac​ tion" di​ ag ​ram from the "VM Net​ work​ ing Overview" chap​ ter. This can be com​ pared to a "switch​ port trunk al​ lowed VLAN xxx" com​ mand in tra​ di​ tional net​ work​ ing. 4 In the Create Attachable Access Entity Profile dialog box. Click the All Interfaces radio button for the desired Interface Policy Group. select the Interface Policy Group your AEP will be associated to. This will be covered later in the Publishing EPGs to VMM Domains chapter. ii. choose Global Policies > Attachable Access Entity Profile. such as "AVS-AEP". Create a New AEP 1 On the menu bar.Fabric Connectivity 129 Attachable Access Entity Profile (AEP) and AVS An im​ por​ tant com​ po​ nent used by the AVS is the At​ tach​ able En​ tity Pro​ file (AEP). perform the following actions: a. The AEP de​ fines which VLANs will be per​ mit​ ted on a host fac​ ing in​ ter​ face. i. See the "Adding New De​ vices to the Fab​ ric" chap​ ter for more de​ tail on cre​ at​ ing the in​ ter​ face pol​ icy group and in​ ter​ face pro​ files. This procedure assumes your Interface Policy Group has already been created. Domains (VMs or Baremetal): Leave blank. Re​ gard​ less of using an ex​ ist​ ing AEP or cre​ at​ ing a new one. the En​ able In​ fra​ struc​ ture VLAN check box must be checked for the AEP pol​ icy. b. Name: Provide any name to identify the AEP. The AEP can be cre​ ated on-the-fly dur​ ing the cre​ ation of the VMM do​ main it​ self. iii. Note: In​ ter​ face Pol​ icy Group cre​ ation is cov​ ered else​ where in this guide. the AEP is what ties the VMM do​ main to the phys​ ic ​al in​ ter​ faces where the vSphere hosts are con​ nected. Enable Infrastructure VLAN: Check this box. choose Fabric > Access Policies. This is to en​ sure that the traf​ fic of in​ ter​ est (DHCP re​ quest/offer can flow through the in​ fra​ struc​ ture VLAN to the AVS). LLDP.

130 Fabric Connectivity Modify an Exisiting AEP 1 On the menu bar. choose Global Policies > Attachable Access Entity Profile. This al​ lows the same poli​ cies ap​ plied to phys​ ic ​al end​ points. A new VMM Do​ main will be cre​ ated from scratch to sup​ port AVS de​ ploy​ ment. VMM Domains for vCenter A Vir​ tual Ma​ chine Man​ ager (VMM) do​ main de​ fines a vir​ tual in​ fra​ struc​ ture that will be in​ te​ grated into ACI. . 2 In the Navigation pane. In the Navigation pane. choose Fabric > Access Policies. to also be ap​ plied to vir​ tual end​ points. vCen​ ter VMM Do​ mains are cre​ ated using ei​ ther the VMware DVS or Cisco AVS. check the Enable Infrastructure VLAN check box. • No Local Switch only supports VLAN encapsulation. This switching mode sends all traffic (Inter-EPG included) to the Leaf switch. This switching mode allows Inter-EPG traffic to be switched locally on the AVS. the In​ fra​ struc​ ture VLAN is re​ quired for AVS com​ mu​ ni​ ca​ tion to the fab​ ric using the Open​ Flex con​ trol chan​ nel. Note: As men​ tioned early in this chap​ ter. In the Work pane. a. AVS Switching Modes The AVS can op​ er​ ate in the fol​ low​ ing switch​ ing modes: • Local Switching: Supports VXLAN encapsulation or VLAN encapsulation. You can​ not change from one to the other. choose the existing AEP b.

Create the VMM Domain for AVS Now that the DHCP server pol​ icy has been cre​ ated and AEP cre​ ated/mod​ i​ fied. If VLAN en​ cap​ su​ la​ tion is pre​ ferred. choose the Policies tab. you can cre​ ate the VMM do​ main for the AVS. . 1 On the menu bar. When using VXLAN en​ cap​ su​ la​ tion. 3 In the Work pane. choose VM NETWORKING. only the infra VLAN is re​ quired to be ex​ tended to the AVS hosts. you will need to en​ sure every VLAN in the VM Do​ main VLAN pool has been ex​ tended be​ tween the fab​ ric and AVS host. 2 In the Navigation pane.Fabric Connectivity 131 ​ AVS Switching Modes: Non-Local and Local switching mode The de​ ci​ sion be​ tween using VLAN or VXLAN en​ cap​ su​ la​ tion will man​ date dif​ fer​ ent VLAN ex​ ten​ sion re​ quire​ ments out​ side of the fab​ ric. choose Actions > Create VCenter Domain. This in​ cludes cre​ at​ ing the VLANs on in​ ter​ me​ di​ ate de​ vices such as UCS and the vNICs for any AVS vSphere hosts. All traf​ fic be​ tween the AVS up​ links and ACI fab​ ric will be en​ cap​ su​ lated by VXLAN and trans​ fered using the in​ fra​ struc​ ture VLAN.

b. vCenter Credentials: Create a credential set with administrator/root acces to vCenter f. . Name: This value will be used as the AVS "Switchname" displayed in vCenter. Switching Preference: <Choose Local or No Local Switching> • For No Local Switching mode: Multicast Address: <Assign a multicast address to represent your AVS> Multicast Address Pool: <Create a unique Multicast Address Pool large enough to include each AVS vSphere host. vCenter: Add the vCenter details. Virtual Switch: Cisco AVS c.> d. perform the following actions: a.> • For Local Switching mode: Encapsulation: <Choose VLAN or VXLAN based on preference> For VLAN Encapsulation VLAN Pool: <Choose/Create a VLAN pool> For VXLAN Encapsulation Multicast Address: <Assign a multicast address to represent your AVS> Multicast Address Pool: <Create a unique Multicast Address Pool large enough to include each AVS vSphere host. Click Submit.132 Fabric Connectivity 4 In the Create vCenter Domain dialog box. 5 • Name: Friendly name for this vCenter • Hostname/IP Address: <DNS or IP Address of vCenter> • DVS Version: vCenter Default • Datacenter: <Enter the exact Datacenter name displayed in vCenter> • Management EPG: <Set to oob or inb Management EPG> • Associated Credentials: <Choose the Credential set previously created> • Click OK to complete the creation of the vCenter. Attachable Access Entity Profile: <Choose the AEP previously created/modified> e.

3 In the Add Host dialog box. 2 Right click on the newly created AVS switch (not the folder) and choose Add Host. Add vSphere Hosts to the AVS After the AVS has been cre​ ated in vCen​ ter. 4 Click Next until the wizard completes. and a few default Port Groups including "uplink" and "vtep". navigate to HOME > INVENTORY > NETWORKING.Fabric Connectivity 133 Verify AVS Deployment on vCenter 1 In the vCenter client. the VMNIC in​ ter​ face used must have all nec​ es​ sary VLANs al​ lowed on the in​ ter​ face. you then need to at​ tach hosts to it. this re​ quires the vNIC within the ser​ vice pro​ file to have all rel​ e​ vant VLANs ac​ tive on the vNICs. skipping the migration of any virtual adapters or virtual machine networking at this time. navigate to HOME > INVENTORY > NETWORKING and confirm a new Distributed Virtual Switch folder has been created. 2 Expand this folder to find your AVS. and select an unassigned VMNIC uplink. 1 From the vCenter client. Note: For blade switch sys​ tems such as UCS. To do this you will need at least one un​ used phys​ ic ​al in​ ter​ face (VMNIC) to act as the up​ link on each host. In UCS terms... choose any vSphere hosts to add to the AVS.. . AVS up​ links can not be shared with any other ex​ ist​ ing vSwitch or vDS.

16.255.176.99.168.255.95 00:50:56:65:3d:b3 1500 65535 .255.0. As can be seen from the screenshot below.255 IP Family IP Address MAC Address IPv4 MTU TSO MSS Enabled Type 65535 true STATIC true STATIC true DHCP 172.54 00:25:b5:00:00:29 1500 IPv4 192.0.176.255 9 10.134 Fabric Connectivity Adding Virtual host physical NICs to participate in Virtual Switch 5 Assuming the ACI fabric can reach the vSphere host over the infra VLAN.0 vmk2 255. This implies that we are ready for Opflex communication in between the AVS and the APIC.0 172.54 00:50:56:61:1c:92 1500 IPv4 65535 10.0.255.0 vmk1 255.255.0.16.255 vmotion 192. This vmk is your Virtual Tunnel Endpoint (vtep) interface and should have pulled a DHCP address from the APIC from the TEP subnet.255. you should see a new vmk interface created on your distributed switch within vCenter and assigned to the 'vtep' port group. ~ # esxcfg-VMKNIC -l Interface Port Group/DVPort Netmask Broadcast vmk0 Management Network 255. we see that the VMkernel port has received the IP address from the APIC.0/16 pool that is created during the APIC setup to provision the IP address. The APIC uses the same 10.0.168.99.16.

0.0. VXLAN Load Balancing VXLAN load bal​ anc​ ing is au​ to​ mat​ ic ​ally en​ abled as soon as more than one VMKNIC is con​ nected to the Cisco AVS. Ver​ ify that the ‘sta​ tus: 12 (Ac​ tive)’ is seen as well as the switch​ ing mode. ~ # vemcmd show epp multicast Number of Group Additions 3 Number of Group Deletions 0 Multicast Address EPP Ref Count 225.0.1. Each VMKNIC can use only one up​ link port.76 1 225. A max​ im ​um of eight VMKNICs can be .30 Port: 8000 Infra vlan: 4093 FTEP IP: 10.0. you should have an equal num​ ber of VMKNICs and up​ links.0.0.0. In the out​ put below.0.0.1 Ver​ ify on the AVS host . there are three dif​ fer​ ent Vir​ tual Ma​ chines con​ nected to dif​ fer​ ent EPGs.0.127.58 1 225.32 Switching Mode: LS NS GIPO: 225. ~ # vemcmd show openflex Status: 12 (Active) Dvs name: comp/prov-VMware/ctrlr-[AVS-TEST]-VC/sw-dvs-87 Remote IP: 10.there should be one mul​ ti​ cast group per de​ ployed EPG on the host.92 1 These mul​ ti​ cast ad​ dresses will cor​ re​ spond to the EPG de​ tails found in the APIC GUI antX > Ap​ pli​ ca​ tion Pro​ files > Ap​ pli​ ca​ tion​ Pro​ fileX > End Point under Ten​ ants > Ten​ Groups > End​ Point​ GroupX and click the Op​ er​ a​ tional Tab > Client End Points.Fabric Connectivity 135 Verify AVS on ESX On the ESX com​ mand line. Also ver​ ify that the GIPO ad​ dress is the same as the mul​ ti​ cast ad​ dress that was used while cre​ at​ ing the VMM do​ main. issue the ‘vem​ cmd show open​ flex’ com​ mand.

Distributed Switch configured with APIC port groups . You need to have as many VMKNICs as the host has VM​ NICs. For ex​ am​ ple. if the host has five VM​ NICs. In the screen​ shot below you can see the four VMNIC up​ links to the AVS and the four vmk vir​ tual in​ ter​ faces to pro​ vide equal load bal​ anc​ ing traf​ fic across all up​ links. up to a max​ i​ mum of eight.136 Fabric Connectivity at​ tached to a Cisco AVS switch. In VMware vSphere Client. you will need to cre​ ate an ad​ di​ tional vir​ tual adapter (VMK) for each AVS up​ link. Each of the VMKNICs that you cre​ ate has its own soft​ ware-based MAC ad​ dress. Each vmk in​ ter​ face cre​ ated for the AVS should be at​ tached to the vtep port group and con​ fig​ ured for DHCP. In VXLAN load bal​ anc​ ing. the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) al​ ready cre​ ated one VMKNIC when the host was added to the dis​ trib​ uted vir​ tual switch (DVS). the VMKNICs pro​ vide a unique MAC ad​ dress to pack​ ets of data that can then be di​ rected to use cer​ tain phys​ i​ cal NICs (VM​ NICs). you need to add four VMKNICs to en​ able VXLAN load-bal​ anc​ ing. Note that each VMK in​ ter​ face added is as​ signed a unique DHCP ad​ dress from the fab​ ric TEP pool.

The al​ ter​ nate method would be to cre​ ate an IGMP pol​ icy on UCS to dis​ able IGMP snoop​ ing.Fabric Connectivity 137 IGMP Snooping Policy for AVS Cisco UCS-B Series Considerations with AVS Deployments This sec​ tion of the ar​ ti​ cle will focus the nec​ es​ sary steps to en​ able AVS through the Cisco UCS-B se​ ries. This will cause flood​ ing of the mul​ ti​ cast traf​ fic to all end​ points. USC-B FI has IGMP snoop​ ing en​ abled. Click the Fast Leave check box. . choose infr > Networking > Bridge Domain > default. b. The IGMP snoop​ ing pol​ icy needs to be en​ abled on the infra ten​ ant. a. Note: Verify if IGMP snooping is working properly on the vSphere host CLI using 'vemcmd show epp multicast' as shown above. choose Create IGMP Snooping Policy. then IGMP pol​ icy is not needed since the blade switch will flood the mul​ ti​ cast traf​ fic on all the rel​ e​ vant ports. Click the Enable Querier check box. choose Tenants > ALL TENANTS. Provide a name for the policy. 2 In the Work pane. d. 4 Click Submit. Create an IGMP Snooping Policy for AVS 1 On the menu bar. such as "IGMP_Infra". By de​ fault. 3 In the Navigation pane. we have to con​ fig​ ure an IGMP querier pol​ icy on the APIC. choose infra. In the IGMP Snoop Policy drop-down list. If we dis​ able IGMP snoop​ ing on UCS or other in​ ter​ me​ di​ ate blade switches. c. Due to this.

.

3 In the Navigation pane. to fur​ ther ex​ tend con​ nec​ tiv​ ity to a re​ mote data cen​ ter. After doing so. Deployment immediacy determines when the actual configuration will be applied on the leaf switch hardware. ACME Inc. and as such. perform the following actions: a. 5 In the Deploy Static EPG on PC. ACME Inc. The immediacy also determines when the hardware resource. or sim​ ply to ex​ tend a Layer 2 do​ main out​ side of the fab​ ric to con​ nect in an ex​ ist​ ing Layer 2 net​ work in a non-ACI fab​ ric. choose Action > Deploy Static EPG on PC. is a multi​ na​ tional com​ pany with mul​ ti​ ple data cen​ ters. Click one of the Deployment Immediacy radio buttons. b. must con​ fig​ ure some Layer 2 con​ nec​ tiv​ ity. In the Path field. 2 In the Work pane. 4 In the Work pane. as the traf​ fic clas​ si​ fi​ ca​ tion will be based on the en​ cap​ su​ la​ tion re​ ceived on a port.Fabric Connectivity 139 External Connectivity Extending ACI to External Layer 2 As men​ tioned in the in​ tro​ duc​ tion of this book. vPC or Interface dialog box. The end​ points do not need to be di​ rectly con​ nected to the ACI leaf. To as​ sign a Layer 2 con​ nec​ tion sta​ ti​ c ​ally on an ACI leaf port to an EPG: 1 On the menu bar. There​ fore. Extending Endpoint Groups Outside the ACI Fabric The sim​ plest way to ex​ tend an end​ point group (EPG) out​ side of the ACI fab​ ric is to sta​ t-​ i​ cally as​ sign a leaf port and VLAN ID to an ex​ ist​ ing end​ point group. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). specify a port as well as a VLAN ID. vPC or Interface. choose the Tenant_Name. This is nec​ es​ sary for ex​ tend​ ing Layer 2 con​ nec​ tiv​ ity to a Data Cen​ ter In​ ter​ con​ nect (DCI) plat​ form. the con​ fig​ ured pol​ icy for this EPG will be en​ forced. all of the traf​ fic re​ ceived on this leaf port with the con​ fig​ ured VLAN ID will be mapped to the EPG. choose Tenants > ALL TENANTS. such as a VLAN resource and policy content-addressable memory .

Similar to the switchport access vlan vlan_ID command. i. The option On Demand instructs the leaf switch to program the EPG and its related policy in the hardware only when traffic matching this policy is received for this EPG.1P headers. but (unlike the untagged mode) 802.1P option refers to traffic tagged with 802. Choose this mode if the traffic from the host is tagged with a VLAN ID. with this option you can only assign the interface to one EPG. Tagged . . e. The option Immediate means that the EPG configuration and its related policy configuration will be programmed in the hardware right away. iii. Multiple EPGs can be statically bound to the same interface as long as the encap VLAN/VXLAN ID is unique. Untagged .The tagged option means that the leaf node expects incoming traffic to be tagged with the specified VLAN ID previously established.The untagged option means that the leaf expects untagged traffic without a VLAN ID. 802.1P will allow other 'tagged' EPGs to be statically bound to the same interface. Any traffic received on links with this mode classification will have the following conditions applied to them: d. Create a physical domain and VLAN pool that are associated to this physical domain. This is the default deployment mode.The 802. The mode option specifies whether the ACI leaf expects incoming traffic to be tagged with a VLAN ID or not. ii. will be consumed on the leaf switch. 802.1P .140 Fabric Connectivity (CAM) to support the related contract for this EPG.1P mode is useful when its necessary to handle the traffic on one EPG as untagged to the interface (similar to the switchport trunk native vlan vlan_ID command). This option can be used to connect a leaf port to a bare metal server whose network interface cards (NICs) typically generate untagged traffic. c. A port can have only one EPG statically bound to a port as untagged. Associate the physical domain to the EPG in question. Click one of the Mode radio buttons.

3 In the Navigation pane. Dur​ ing this process. create a Layer 2 domain if one does not already exist. To cre​ ate an ex​ ter​ nal Layer 2 do​ main: 1 On the menu bar. choose the Tenant_Name. Sim​ il​ ar to the pre​ vi​ ous ex​ am​ ple of adding an end​ point to a pre-ex​ ist​ ing EPG. cre​ ate a new ex​ ter​ nal EPG to clas​ sify this ex​ ter​ nal traf​ fic.Fabric Connectivity 141 f. i. 4 5 In the Work pane. This VLAN ID must be in the range of the VLAN pool that is used for the Layer 2 outside connection. Associate the Layer 2 outside connection with the bridge domain and a VLAN ID. you will also need to se​ lect which traf​ fic you would like to tra​ verse be​ tween the two EPGs. Extending an ACI Bridge Domain Outside of the Fabric A Layer 2 out​ side con​ nec​ tion is as​ so​ ci​ ated with a bridge do​ main and is de​ signed to ex​ tend the whole bridge do​ main. 2 In the Work pane. This Layer 2 outside connection will put this VLAN and the bridge domain of the ACI fabric under the same Layer 2 domain. . To ac​ com​ plish an ex​ ten​ sion of the bridge do​ main to the out​ side. This VLAN must be configured on the external Layer 2 network. For the External Bridged Domain drop-down list. choose Tenant_Name > Networking > External Bridged Network. this method will also allow the end​ points to share the same sub​ net and de​ fault gate​ way. choose Tenants > ALL TENANTS. With two sep​ ar​ ate EPGs. Create an attachable access entity profile (AEP) to map the interfaces and policies together. See the Adding New De​ vices to the Fab​ ric sec​ tion for more in​ for​ ma​ tion on how to con​ fig​ ure an AEP and a phys​ ic ​al do​ main. perform the following actions: a. Clas​ sify any out​ side con​ nec​ tions or end​ points into this new ex​ ter​ nal EPG. This new EPG will be part of the ex​ ist​ ing bridge do​ main. In the Create Bridged Outside dialog box. not just an in​ di​ vid​ ual EPG under the bridge do​ main. a Layer 2 out​ side con​ nec​ tion must be cre​ ated for the bridge do​ main. to the out​ side net​ work. choose Action > Create Bridged Outside.

c. Extending ACI to External Layer 3 The most im​ por​ tant com​ po​ nent of any ap​ pli​ ca​ tion is the user or cus​ tomer.142 Fabric Connectivity ii. and there​ fore there must be con​ nected to the ex​ ter​ nal net​ work. see the Adding New Devices to the Fabric section. In the Navigation pane. as well as to the In​ ter​ net to pro​ vide ac​ cess to the mo​ bile ap​ pli​ ca​ tion. This is a means to specify the range of the VLAN IDs that will be used for creating a Layer 2 outside connection. and pro​ vides IP con​ nec​ tiv​ ity be​ tween a ten​ ant pri​ vate net​ work and an ex​ ter​ nal IP net​ work. Add a Layer 2 border leaf node and Layer 2 interface for a Layer 2 outside connection. create a VLAN pool to associate to the VLAN on the Layer 2 outside connection. if it does not already exist. Create an AEP. Layer 3 con​ nec​ tiv​ ity to a de​ vice like a router is known as an Ex​ ter​ nal Routed Net​ work. ACME must be able to con​ nect to both their in​ ter​ nal cor​ po​ rate back​ bone. After specifying this contract as the provided contract for your internal EPG. the communication between this external Layer 2 EPG and your existing internal EPG will be allowed. Configure a contract to allow communication between your existing endpoints in the existing EPG and your new external Layer 2 EPG. After adding a Layer 2 border leaf and Layer 2 interface. Each Layer 3 ex​ ter​ nal con​ nec​ tion is as​ so​ ci​ ated with one ten​ ant . b. All of the traffic entering the ACI fabric with the designated VLAN (the VLAN ID provided in step 1) will be classified into this Layer 2 EPG. Simply provide a name for the Layer 2 EPG. which gen​ er​ ally does not di​ rectly at​ tach to the fab​ ric. d. choose External Bridged Networks > Networks and specify a contract to govern this policy as the consumed contract. This helps avoid any overlapping in the VLAN range between VLANs used for an EPG and those in use for a Layer 2 outside connection. While creating the Layer 2 domain. You should now have the de​ sired reach​ ab ​il​ ity be​ tween the in​ side and out​ side Layer 2 seg​ ments. This in​ te​ gra​ tion is pos​ si​ ble with Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) at the ten​ ant pol​ icy level. click Next to start creating a Layer 2 EPG. e. This is a policy object that tells the APIC to allow certain encap (VLANs) on selected ports. For more information on how to create a Access Attachable Entity Profile.

As the op​ er​ a​ tor of the fab​ ric. In large scale ACI de​ signs it might be pro​ duc​ tive to have ded​ i​ cated ACI leafs as bor​ der leafs. like servers. A bor​ der leaf is sim​ ply ter​ mi​ nol​ ogy to refer to a leaf that hap​ pens to be con​ nected to a Layer 3 de​ vice. of an ap​ pli​ ca​ tion into end​ point groups (EPGs). or tiers. web servers need a con​ nec​ tion to the out​ side world for users to reach them. you can pro​ vide the ten​ ant ad​ min​ is​ tra​ tor with the abil​ ity to in​ ter​ face to an ex​ ter​ nal Layer 3 con​ nec​ tion in var​ i​ ous ways by using a uniquely-de​ fined Layer 3 con​ struct for the ten​ ant ap​ pli​ ca​ tion pro​ file or a shared com​ mon in​ fra​ struc​ ture. the ex​ ter​ nal Layer 3 con​ nec​ tion can be one of the fol​ low​ ing types: 1 Physical Layer 3 interface 2 Subinterface with 8021. These ap​ pli​ ca​ tion com​ po​ nents might have re​ quire​ ments for ex​ ter​ nal con​ nec​ tiv​ ity into them. In the ACI fab​ ric.Fabric Connectivity 143 pri​ vate net​ work.Q tagging 3 Switched Virtual Interface (SVI) . Ex​ ter​ nal Layer 3 con​ nec​ tions are usu​ ally es​ tab​ lished on the bor​ der leaf con​ struct of the ACI. the con​ nec​ tiv​ ity is de​ fined by a con​ tract to a de​ fined ex​ ter​ nal Layer 3 end​ point group. Any ACI leaf can be​ come a bor​ der leaf. An ap​ pli​ ca​ tion pro​ file en​ ables an op​ er​ a​ tor to group the dif​ fer​ ent com​ po​ nents. can still con​ nect to the bor​ der leaves. With ACI. It is im​ por​ tant to note that the spine nodes can​ not have con​ nec​ tions to ex​ ter​ nal routers. Other de​ vices. The re​ quire​ ment of the Layer 3 ex​ ter​ nal net​ work is only needed when a group of de​ vices in the ap​ pli​ ca​ tion pro​ file re​ quire Layer 3 con​ nec​ tiv​ ity to a net​ work out​ side of the ACI fab​ ric. The fol​ low​ ing fig​ ure shows part of a sim​ pli​ fied fab​ ric: A sample application profile for a three-tiered application with contracts between the tiers For ex​ am​ ple.

Indicates that this subnet will be advertised to the external router.1 of Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). One com​ mon mech​ a​ nism is to use sub-in​ ter​ faces on a router to cre​ ate dif​ fer​ ent Layer 3 do​ mains since each ten​ ant will likely not have their own ex​ ter​ nal router. These sub​ nets can be clas​ si​ fied as pri​ vate.​ 1. or shared: • Public . For the pri​ vate net​ works. This be​ hav​ ior is known as a non-tran​ sit fab​ ric. • Private .1 and later. • Shared . the ex​ ter​ nal net​ work has learned of the in​ ter​ nal ACI net​ work 10.​ 0/​ 24. and routes learned from one ex​ ter​ nal Layer 3 con​ nec​ tion can be ad​ ver​ tised out to a dif​ fer​ ent ex​ ter​ nal Layer 3 con​ nec​ tion.Indicates that this subnet will be leaked to one or more private networks inside of the ACI fabric. not just fab​ ric routes. ACI is ca​ pa​ ble of act​ ing as a tran​ sit net​ work.144 Fabric Connectivity Bridge do​ mains con​ tain one or more sub​ nets. In re​ leases prior to ver​ sion 1. The fol​ low​ ing fig​ ure de​ picts the logic of pub​ lic and pri​ vate net​ works: Application profile with external consumers. the fab​ ric only ad​ ver​ tises sub​ nets that are marked pub​ lic in the as​ so​ ci​ ated bridge do​ main. public and private networks annotated With de​ vices con​ nect​ ing through the ex​ ter​ nal Layer 3 con​ nec​ tion. as it is ad​ ver​ tised to the ad​ ja​ cent router through the Layer 3 ex​ ter​ nal con​ nec​ tion. ACI does not ad​ ver​ tise the net​ works through the rout​ ing pro​ to​ col to the ad​ ja​ cent Layer 3 router.Indicates that this subnet will be contained only within the ACI fabric private network. pub​ lic. In re​ lease ver​ sion 1. Routes that are learned ex​ ter​ nally from the fab​ ric are not ad​ ver​ tised through other ports. and the net​ works are not reach​ able to de​ vices ex​ ter​ nal to the fab​ ric.​ 1. . The net​ work team will pro​ vide the ex​ ter​ nal Layer 3 con​ nec​ tiv​ ity for their ten​ ants.

and the ACI leaves that are not part of the border leaf does not have IP connectivity to any outside networks. . or BGP) for the Layer 3 outside connections are not propagated within the ACI fabric. • iBGP peering leaf and external router—With internal Border Gateway Protocol (iBGP). Using static routes reduces the sizing and complexity of the routing tables in the leaf nodes. ACI supports only one autonomous system (AS) number that has to match the one that is used for the internal Multiprotocol Border Gateway Protocol (MP-BGP) route reflector. Configure MP-BGP Spine Route Reflectors The ACI fab​ ric route re​ flec​ tors use mul​ ti​ pro​ to​ col bor​ der gate​ way pro​ to​ col (MP-BGP) to dis​ trib​ ute ex​ ter​ nal routes within the fab​ ric so a full mesh BGP topol​ ogy is not re​ quired. To en​ able route re​ flec​ tors in the ACI fab​ ric. the user must find out the AS number on the router to which the ACI border leaf will connect. OSPF NSSA advertises to the adjacent router the internal public subnets part of the Layer 3 external. the fab​ ric ad​ min​ is​ tra​ tor must se​ lect at least one spine switch that will be a route re​ flec​ tor. Without MP-BGP. and pro​ vide the au​ tonomous sys​ tem (AS) num​ ber for the fab​ ric. Once route re​ flec​ tors are con​ fig​ ured. the router learns only a summarization of routes. the external routes (static. con​ fig​ ure more than one spine as a router re​ flec​ tor node. To con​ nect ex​ ter​ nal Layer 3 de​ vices to the ACI fab​ ric. the fab​ ric in​ fra​ struc​ ture op​ er​ at​ or must con​ fig​ ure a Route Re​ flec​ tor pol​ icy to des​ ig​ nate which spines act as the route re​ flec​ tor(s). Given that the same AS number is used for both cases. With static routes. and use that AS number as the BGP AS number for the ACI fabric. With OSPF NSSA. For re​ dun​ dancy pur​ poses. OSPF.Fabric Connectivity 145 Supported Routing Protocols The fol​ low​ ing rout​ ing pro​ to​ cols are sup​ ported at time of writ​ ing: • Static routes—You can define static routes to the outside world. • OSPF NSSA—Using not-so-stubby area (NSSA) reduces the size of the Open Shortest Path First (OSPF) database and the need to maintain the overhead of the routing protocols with large tables of routes. you must also configure the static path back to the interanal network that you wish to be reachable from the outside world. but increases administrator overhead. including a default path out of the fabric. ad​ min​ is​ tra​ tors can setup con​ nec​ tiv​ ity to ex​ ter​ nal net​ works.

Click Submit. In the BGP Route Reflector Policy drop-down list. After the route re​ flec​ tors are con​ fig​ ured. in the Fabric Policy Group drop-down list. Complete the remainder of the dialog box as appropriate to your setup. perform the following actions: a. choose Actions > Create Pod Policy Group. d. choose default. which pairs the bor​ der leaf node with one of the route re​ flec​ tor nodes as a BGP peer. choose default. f. 5 In the Work pane. b. 3 In the Work pane. In the Navigation pane. c. . 4 In the Navigation pane. they can ad​ ver​ tise routes in the fab​ ric. in the Date Time Policy dropdown list. Each leaf node can store up to 4000 routes at time of writ​ ing. Add the two spine nodes that will be members of this reflector policy and c. 7 Click Submit. 2 In the Navigation pane. To con​ fig​ ure the Route Re​ flec​ tor pol​ icy: 1 On the menu bar. choose Fabric > Fabric Policies. choose Pod Policies > Policies > BGP Route Deflector default. If a WAN router must ad​ ver​ tise more than 4000 routes. In the BGP Route Reflector Policy drop-down list. the in​ fra​ struc​ ture op​ er​ at​ or con​ fig​ ures the leaf node to which the WAN router is being con​ nected as bor​ der leaf node. 6 In the Create Pod Policy Group dialog box. choose Create Pod Policy Group. choose Pod Policies > Profiles > default. The in​ fra​ struc​ ture op​ er​ at​ or con​ fig​ ures each of the paired leaf nodes with the routes (or route pre​ fixes) that the nodes can ad​ ver​ tise. choose Pod Policies. the router should peer with mul​ ti​ ple leaf nodes.146 Fabric Connectivity When a ten​ ant re​ quires a Layer 3 con​ nec​ tion. choose default. In the Create Pod Policy Group dialog box. Change the Autonomous System Number to match the required number for your network. In the Work pane. b. e. perform the following actions: a.

Fabric Connectivity 147 The fol​ low​ ing fig​ ure il​ lus​ trates the ob​ jects and their re​ la​ tion​ ships for ex​ ter​ nal Layer 3 con​ nec​ tions: Object relationships for Layer 3 outside objects Layer 3 Integration Through Tenant Network with OSPF NSSA The fol​ low​ ing fig​ ure shows a sim​ ple in​ te​ gra​ tion of a Layer 3 ex​ ter​ nal into ACI using OSPF: Logical topology for an external OSPF router communicating with two border leafs .

In the Create Node Profile dialog box. In the Select Node dialog box. click the Send redistributed LSAs into NSSA area check box. click the NSSA Area radio button. such as "1". Enter the loopback address. In the OSPF Area Control section. 5. 3 In the Navigation pane. e. 6. enter the router's IP address as the ID. such as Leaf-1. h. iii.254. In the OSPF Area ID field. In the Node ID drop-down list. and click Update. 4 5 In the Work pane. choose the tenant. such as "10. In the Create Routed Outside dialog box. In the Name field. click + to add a loopback address. click + to add a profile.0. Note: See the "Adding New De​ vices to The Fab​ ric" sec​ tion to setup the ac​ cess poli​ cies for the in​ ter​ faces of the leaves that are con​ nected to the router. such as "10. In the OSPF Area Type section. choose a node. Uncheck the Router ID as Loopback Address check box.​ g. click + to add a node.148 Fabric Connectivity The setup in​ cludes a sin​ gle router with two in​ ter​ faces con​ nected to leaf switches. enter a name for the profile. choose Tenant_Name > Networking > External Routed Networks. ii. 4. perform the following actions: a. choose Tenants > ALL TENANTS. In the Router ID field. enter the OSPF area ID. Click the OSPF check box. enter a name for the profile. perform the following actions: 1.1". d. 2. f. b. 2 In the Work pane.1". In the Nodes and Interfaces Protocol Profiles section. In the Loopback Addresses section. In the Nodes section. .1. perform the following actions: i. In the Private Network drop-down list. c. In the Name field. choose the private network for this tenant. Click OK. 3.254. choose Action > Create Routed Outside. To in​ te​ grate Layer 3 through a ten​ ant net​ work with OSPF/NSSA: 1 On the menu bar.

b. 2. k. 3. In the Create OSPF Interface Policy dialog box. click + create an external network. d. In the IP Address field. In the Path drop-down list. enter a name for the profile. In the OSPF Interface Profiles section. When defining the interaction with another OSPF router. enter the IP address of the path that is attached to the layer 3 outside profile. This document does not explain the different OSPF parameters. Click Next. Click the + sign to select a routed interface. 4.1/24". perform the following actions: . enter the maximum MTU of the external network. Click OK. In the Interfaces section. such as Point to Point.0. In the Create Interface Profile dialog box. 6. 5. 7. In the Create External Network dialog box. you must specify the policy interaction. such as "OSPFPoint2Point". In the Name field. In the OSPF Policy drop-down list. In the External EPG Networks section. j. vi. Click OK. perform the following actions: a. b. click on the Routed Intefaces tab. Complete the remainder of the dialog box as appropriate to your setup and click OK. choose Create OSPF Interface Policy. v. click the radio button that matches the adjacent router. c.Fabric Connectivity 149 iv. c. choose the interface on the leaf. click + to create an OSPF interface profile. such as e1/9 on Leaf-1. such as "1500" to match the example peering router. d. In the Select Routed Interface dialog box. Complete the remainder of the dialog box as appropriate to your setup. perform the following actions: 1.1. In the Network Type section. enter a name for the OSPF policy. In the Name field. i. Click Submit. perform the following actions: a. such as "10. In the MTU (bytes) field.

In the IP Address field. you can use var​ io ​us mech​ an ​isms to reuse the same ex​ ter​ nal Layer 3 router for mul​ ti​ ple ten​ ants.0/0 to permit the learning of any subnet and click OK. l. The fab​ ric op​ er​ at​ or can con​ fig​ ure mul​ ti​ ple ex​ ter​ nal Layer 3 con​ nec​ tions using ei​ ther sub-in​ ter​ face or SVI and pro​ vide that to each ten​ ant.0. . For routers ca​ pa​ ble of using sub-in​ ter​ faces. you must configure the external network EPG.150 Fabric Connectivity i. enter 0. the ex​ ter​ nal Layer 3 con​ nec​ tion can be con​ fig​ ured to route via SVI.0. those can be used to pro​ vide mul​ ti​ ple ex​ ter​ nal Layer 3 con​ nec​ tion for mul​ ti​ ple VRFs and/or ten​ ants. If the ad​ ja​ cent router is a Cisco Nexus Se​ ries Switch with a Layer 2 trunk in​ ter​ face. External Layer 3 for Multiple Tenants In ACI. Next. Click Finish.

Fur​ ther in​ for​ ma​ tion and steps on how to cre​ ate this Layer 2 and Layer 3 con​ nec​ tiv​ ity in​ clud​ ing the pol​ icy.Fabric Connectivity 151 Application Migration Use Case When op​ er​ at​ ing the ACI fab​ ric. can be found in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter of this book. onto the fab​ ric. One com​ mon ex​ am​ ple is when mi​ grat​ ing from a tra​ di​ tional data cen​ ter con​ fig​ ur​ a​ tion over to a pol​ icy-dri​ ven data cen​ ter using ACI. As​ sum​ ing that the SVIs re​ side on the ex​ ter​ nal Layer 2 net​ work. you must start by con​ fig​ ur​ ing the Layer 2 out​ side net​ work to allow traf​ fic from the source VLAN to com​ mu​ ni​ cate with the same VLAN re​ sid​ ing on the ACI fab​ ric. there can be oc​ ca​ sions when you will have to mi​ grate work​ loads. . Once they have suc​ cess​ fuly es​ tab​ lished con​ nec​ tiv​ ity be​ tween the out​ side Layer 2 net​ work (where the work​ load or host is com​ ing from) and the ex​ ist​ ing in​ ter​ nal fab​ ric EPG. ACME would like to per​ form the mi​ gra​ tion in mul​ ti​ ple stages. One key con​ sid​ er​ at​ ion should be when to switch over the SVI in​ ter​ faces from the ex​ ist​ ing en​ vi​ ron​ ment into the ACI fab​ ric and when to start ad​ ver​ tis​ ing routes to this SVI net​ work. Cisco rec​ om​ mends that you move the SVIs over to the ACI fab​ ric once a ma​ jor​ ity of the hosts have been mi​ grated over. it will be come nec​ es​ sary to per​ form these mi​ gra​ tions. At a high level. servers or vir​ tu​ al​ iza​ tion hosts from out​ side the ACI fab​ ric. ACME must man​ age the mi​ gra​ tion of SVI in​ ter​ faces as well as the pol​ icy al​ low​ ing traf​ fic to tra​ verse a Layer 2 out​ side net​ work and then on to the ACI fab​ ric. They would like to mi​ grate with min​ im ​al ser​ vice in​ ter​ rup​ tion while tak​ ing ad​ van​ tage of ACI in​ no​ va​ tions where ap​ plic​ ab ​le. As ACME starts to use ACI in more of their data cen​ ters. You will also need to con​ fig​ ure Layer 3 con​ nec​ tiv​ ity from the fab​ ric out to the ex​ ist​ ing Layer 3 net​ works in prepa​ ra​ tion for full con​ nec​ tiv​ ity after SVI mi​ gra​ tion. you can then start the mi​ gra​ tion process of mov​ ing ap​ pli​ ca​ tion work​ loads onto the fab​ ric. Extending the Network to ACI One of the ACME sites would like to mi​ grate from the legacy data cen​ ter ar​ chi​ tec​ ture to the next gen​ er​ at​ ion ACI Fab​ ric. In this ex​ am​ ple.

152 Fabric Connectivity The Legacy data cen​ ter prior to mi​ gra​ tion: Traditional pre-migration data center The ACI data cen​ ter fol​ low​ ing mi​ gra​ tion: Post-migration ACI based data center topology .

The cre​ ated ten​ ant will rep​ re​ sent the legacy data cen​ ter into the ACI fab​ ric.Fabric Connectivity 153 The first stage will pro​ vide con​ nec​ tiv​ ity from the legacy data cen​ ter to the ACI fab​ ric. . 1 2 On the menu bar. Pro​ vide phys​ ic ​al con​ nec​ tiv​ ity from the ex​ ist​ ing ag​ gre​ ga​ tion layer to the ACI bor​ der leafs. In this state. In the Name field enter a name for the tenant. Con​ fig​ ure the ag​ gre​ ga​ tion links as a Layer 2 trunk. Port Chan​ nel. 2 In the Work pane. In the APIC con​ troller you will now con​ fig​ ure a sin​ gle ten​ ant. you log​ ic ​ally map a VLAN=EPG. Con​ fig​ ure a sin​ gle pri​ vate net​ work. the Fab​ ric Ac​ cess Poli​ cies for the ac​ cess ports that you will be used for the DCI must be con​ fig​ ured. This allows for the host VLAN to be extended into the fabric. 1 Provide a physical connection from aggregation switch #1 to the ACI border leaf #1. Note: Be​ fore con​ nect​ ing ex​ ter​ nal phys​ ic ​al con​ nec​ tions into the fab​ ric. choose Tenants > Add Tenant. 1 On the menu bar. b. In the Create Tenant dialog box perform the following actions: a. 3 Click Finish. 1 Trunk the VLAN representing the host connectivity. This con​ nec​ tiv​ ity can be ac​ com​ plished in ei​ ther the form of a Vir​ tual Port Chan​ nel. The in​ ter​ con​ nect from the legacy net​ work to the ACI fab​ ric will be ac​ com​ plished through stan​ dard Layer 2 ex​ ten​ sions (VLAN/VXLAN). 2 Provide a physical connection from aggregation switch #2 to the ACI border leaf #1. or a sin​ gle in​ ter​ face. choose Tenants > ALL TENANTS. For de​ tails on con​ fig​ ur​ ing the ac​ cess poli​ cies please ref​ er​ ence the Fab​ ric Con​ nec​ tivty sec​ tion of this book. choose the Tenant_Name. Click Next.

2 In the Work pane. 5 In the Create Private Network dialog box. choose Tenants > ALL TENANTS. choose the Tenant_Name. the bridge do​ main will then mimic the be​ hav​ ior in the ACI Fab​ ric. e. g. perform the following actions: a. Click the ARP Flooing check box. Con​ fig​ ure a sin​ gle ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. 6 Click Finish. choose Actions > Create Private Network. click Flood. perform the following actions: a. In the Name field enter a name for the Application Profile. click Flood in BD. choose Tenants > ALL TENANTS 2 In the Work pane. b. 5 In the Create Application Profile dialog box. 4 In the Work pane. f. b. By de​ fault BPDU frames are flooded within EPG. For the Multi Destination Flooding radio buttons. 3 In the Navigation pane choose Application Profiles. choose Actions > Create Application Profile. Con​ fig​ ure a sin​ gle end​ point group: 1 On the menu bar. . 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name. In the Name field enter a name for the private network. In the Name field enter a name for the bridge domain. Click Submit. choose Custom. In the Forwarding drop-down list. Note: The con​ cept of flood​ ing the un​ known uni​ cast and arp within the Fab​ ric is to allow for the Layer 2 se​ man​ tics from the legacy data cen​ ter to be ex​ tended into the ACI Fab​ ric. choose the Tenant_Name. Click Next. For the Layer 2 Unknown Unicast radio buttons. c. When a host in the legacy data cen​ ter sends an ARP re​ quest and/or floods an un​ known uni​ cast frame. d.154 Fabric Connectivity 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. 4 In the Work pane.

VPC or Interface dialog box. The en​ cap​ su​ la​ tion VLAN should match the de​ fined legacy data cen​ ter VLAN de​ f​ in ​i​ t​ ion. con​ fig​ ure a sta​ tic trunk bind​ ing using the VPC under the EPG. b. perform the following actions: a. b. Fol​ low​ ing the Stage 1 mi​ gra​ tion. The use of a sin​ gle VLAN per EPG will pro​ vide a path for a net​ work-cen​ tric mi​ gra​ tion— min​ im ​iz​ ing im​ pact while in​ tro​ duc​ ing fab​ ric in​ no​ va​ tions. VPC or Interface. Then. the Legacy host VLAN is now ex​ tended to the ACI fab​ ric and all hosts from the Fab​ ric point of view are in the EPG. choose Actions > Create Application EPG. Enter the encapsulation VLAN. d. Choose the Path Type. See the "Fab​ ric Con​ nec​ tiv​ ity" sec​ tion. 1 On the menu bar. In the Bridge Domain field. choose the appropiate bridge domain. enter a name for the endpoint group. In the Name field. perform the following actions: a. Click Submit. 4 In the Work pane. Note: The EPG within the fab​ ric will map to a sin​ gle VLAN in the legacy data cen​ ter. . 2 In the Work pane. Choose the Path. This is what is known as nor​ mal​ iza​ tion where ACI uses the tagged vlan to map ex​ ter​ nal Layer 2 con​ nec​ tions into ACI End Point Groups. c. c. Click Finish. Con​ fig​ ure a VPC for the con​ nec​ tiv​ ity to the legacy data cen​ ter. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths).Fabric Connectivity 155 4 5 In the Work pane. the local VLAN num​ ber is not sig​ nif​ ic ​ant as it will be mapped to the tagged VLAN on the VPC to​ ward the Legacy data cen​ ter. 5 In the Deploy Static EPG on PC. Acme​ Out​ Side. From the ACI Fab​ ric per​ spec​ tive. In the Create Application EPG dialog box. Acme​ Out​ Side. choose Actions > Deploy Static EPG on PC. choose Tenants > ALL TENANTS.

b. Dur​ ing this stage. Note: En​ sure the ap​ pro​ pri​ ate re​ sources have been cre​ ated to sup​ port the con​ nec​ tiv​ ity.156 Fabric Connectivity The con​ nec​ tiv​ ity noted below rep​ re​ sents a BD=EPG=VLAN. . i. Create the endpoint group "AcmeInSide". c. Create a static binding for each physical host to be migrated. ii. a. An EPG as a VLAN datacenter deployment The sec​ ond stage of the mi​ gra​ tion takes ad​ van​ tage of the fab​ ric pol​ icy model to de​ fine fur​ ther the ap​ pli​ ca​ tion mod​ el​ ing. "AcmeInSide". choose Any-Any. choose Tenant. Configure additional Bridge Domains (optional). Configure additional End Point Groups. AEP. Create a VMM domain to deploy the host within the ACI Fabric (optional). The fol​ low​ ing list is an overview of the steps that you must per​ form: 1 In the APIC: a. For more in​ for​ ma​ tion. 2 Begin migrating the host to the fabric EPG. b. The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter or pro​ vided by the Legacy data cen​ ter. mul​ ti​ ple EPGs will be cre​ ated with APIC con​ tracts that de​ fine reach​ a​ bil​ ity. For the Scope drop-down list. f. Configure contracts for inter-EPG communications. VLAN pools. d. phys​ i​ cal do​ main or VMM Do​ main. For the Allow drop-down list. Configure additional Application Profiles (optional). i. and switch or in​ ter​ face pro​ files. Configure additional Tenants (optional). Configure additional Private Networks (optional). e. see the "Fab​ ric Con​ nec​ tivty" sec​ tion.

iv. ii. . choose Unknown Unicast. iii. The fol​ low​ ing list is an overview of the steps that you must per​ form: 1 In the APIC: a. Datacenter migration with Layer 3 provided by existing DC and Layer 2 extended to new ACI datacenter In the third stage of the mi​ gra​ tion the Layer 3 gate​ way will move from the legacy data cen​ ter to the ACI Fab​ ric. choose AcmeBD. In the ARP drop-down list. Note: The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter are pro​ vided by the legacy data cen​ ter. Note: The con​ cept of uni​ cast rout​ ing within the bridge do​ main al​ lows for the con​ fig​ u​ ra​ tion of the per​ va​ sive gate​ way across the fab​ ric.Fabric Connectivity 157 Fol​ low​ ing the stage 2 mi​ gra​ tion. Configure a Layer 3 out. The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter are pro​ vided by the ACI fab​ ric. Migrate the gateway from the legacy data center to the fabric. choose Routing. choose Flooding. b. the host con​ nec​ tiv​ ity across both the legacy data cen​ ter and ACI fab​ ric are now gov​ erned by the APIC pol​ icy (con​ tracts). In the Unicast drop-down list. i. In the Bridge Domain drop-down list. In the Flood Layer 2 drop-down list.

.

159 Tenants .

.

Tenants 161 Section Content • • ACI Tenancy Models Application Profile Application Profile Configuration Create a New Application Profile Modify Application Profile Remove Application Profile Verify Application Profile • Endpoint Group Endpoint Group Configuration Create a New Endpoint Group Modify Endpoint Group Remove Endpoint Group Verify Endpoint Group • Endpoint Verify Endpoint Group • Private Network Private Network Configuration Parameters Creating a New Private Network Modify Private Network Remove Private Network Verify Private Network .

162 Tenants • Bridge Domain Bridge Domain Configuration Parameters Create a new Bridge Domain Modify a Bridge Domain Remove a Bridge Domain Verify Bridge Domain • Tenant Networking Use Cases Common Private Network for All Tenants Multiple Private Networks with Intra-Tenant Communication Multiple Private Networks with Inter-Tenant Communication .

They will be using ten​ ant con​ structs for the ap​ pli​ ca​ tion life​ cy​ cle of their cur​ rent de​ ploy​ ment. such as web. a ten​ ant is a unique cus​ tomer. security. and pre​ vent changes which would im​ pact other groups. and fi​ nally a pro​ duc​ tion ten​ ant. they are also look​ ing to build an in​ fra​ struc​ ture which can be lever​ aged for sim​ il​ ar ini​ tia​ tives in the fu​ ture. 4 Fault domain size and scope to limit the impact of failures. In the case of a clas​ sic ser​ vice provider. a ten​ ant that will be used for the au​ to​ mated test​ ing. Ad​ di​ tion​ ally. such as different business units. The ten​ ant ob​ ject gives us the abil​ ity to draw a box around the log​ ic ​al and con​ crete ob​ jects that we use to pro​ vide a uni​ fied view of the con​ fig​ ur​ a​ tion de​ pen​ den​ cies for un​ der​ lay and over​ lay net​ works. main​ tain​ ing a sep​ a​ rate ten​ ant for the re​ sources that de​ vel​ op​ ers will be using to build the ap​ pli​ ca​ tion. Ten​ ants will be used to draw vir​ tual bound​ aries for dif​ fer​ ent lines of busi​ ness. mak​ ing a rout​ ing pro​ to​ col change on a router or Layer 3 switch could po​ ten​ tially af​ fect hun​ dreds of unique VLANs/sub​ nets. 3 Separation of duties by domain owner. This means dif​ fer​ ent things to dif​ fer​ ent peo​ ple (much like the term Cloud) based on their per​ spec​ tive. and Production. In tra​ di​ tional net​ work​ ing en​ vi​ ron​ ments. servers. while in a typ​ ic ​al endcus​ tomer en​ vi​ ron​ ment a ten​ ant could be an op​ er​ at​ ing group. This in​ tro​ duces a war​ ranted level of cau​ tion around change con​ trol and ap​ pli​ ca​ tion im​ pact. and so on. Lever​ ag​ ing the ACI pol​ icy model. The in​ for​ ma​ tion se​ cu​ rity team will be able to in​ te​ grate this into the cor​ po​ rate LDAP sys​ tem.Tenants 163 ACI Tenancy Models ACME Inc. and database owners. will be using ten​ ancy for a cou​ ple of use cases. ap​ pli​ ca​ tion owner. the phys​ ic ​al hard​ ware is ab​ stracted from the log​ ic ​al con​ structs. Quality Assurance. ACI has been de​ signed from the be​ gin​ ning to be “multi-ten​ ant”. busi​ ness unit. as men​ tioned in the in​ tro​ duc​ tion. etc. app. The de​ ci​ sion on how to lever​ age ten​ ancy mod​ els is dri​ ven by a num​ ber of fac​ tors: 1 Overall IT operations and support models in your organization to manage application. 2 Separation of environments from a software development lifecycle perspective: Development. networking. .

Al​ though the ten​ ant net​ work​ ing and the ten​ ant poli​ cies are de​ fined sep​ ar​ ately. ap​ pli​ ca​ tion servers and data​ base servers. In ACI. bridge do​ mains and sub​ nets. the ten​ ant poli​ cies are where you de​ fine ap​ pli​ ca​ tions.164 Tenants A Ten​ ant in the ACI Ob​ ject model rep​ re​ sents the high​ est-level ob​ ject. the ap​ pli​ ca​ tion servers com​ mu​ ni​ cate with the data​ base which could also be a clus​ ter of servers. such as Pri​ vate Net​ works. adds each com​ po​ nent. An ap​ pli​ ca​ tion could con​ sist of a com​ bi​ na​ tion of phys​ ic ​al servers or VMs that we will call servers from now on. There​ fore. The ten​ ant net​ work​ ing is used to de​ fine net​ work​ ing poli​ cies and will be ap​ plied to the un​ der​ ly​ ing hard​ ware in a trans​ par​ ent way thanks to the layer of ab​ strac​ tion pro​ vided by ACI using pri​ vate net​ works. The fol​ low​ ing image shows all of the com​ po​ nents that can be con​ fig​ ured within a ten​ ant. End​ points are clas​ si​ fied in ACI to apply poli​ cies. you can dif​ fer​ en​ ti​ ate be​ tween the ob​ jects that de​ fine the ten​ ant net​ work​ ing. an ap​ pli​ ca​ tion can be formed by sev​ eral end​ point groups and they are grouped in an Ap​ pli​ ca​ tion Pro​ file. a web​ site could use a 3-tier ap​ pli​ ca​ tion model. such as with whom are they going to com​ mu​ ni​ cate and what type of com​ mu​ ni​ ca​ tion or re​ stric​ tions are re​ quired. Bridge Do​ mains and sub​ nets. the net​ work​ ing poli​ cies used by an ap​ pli​ ca​ tion are de​ fined with a re​ la​ tion​ ship be​ tween the End​ point Groups and the Bridge Do​ main. The web servers in turn com​ mu​ ni​ cate with core ap​ pli​ ca​ tions that can be di​ vided amongst sev​ eral ap​ pli​ ca​ tions servers for load bal​ anc​ ing or high avail​ abil​ ity pur​ poses. For ex​ am​ ple. . You cre​ ate end​ point groups with end​ points that share the same type of poli​ cies. When a user browses the web site. Fi​ nally. In the next sec​ tions of this chap​ ter these con​ cepts will be cov​ ered in de​ tail. Each server is re​ ferred to as an End​ point in ACI. and the ob​ jects that de​ fine the ten​ ant poli​ cies such as Ap​ pli​ ca​ tion Pro​ files and End​ point Groups. they might ac​ tu​ ally be com​ mu​ ni​ cat​ ing with a vir​ tual IP ad​ dress on a load bal​ ancer that in turn can dis​ trib​ ute the web re​ quest to a num​ ber of dif​ fer​ ent web servers. In​ side. com​ prised of web servers. Below you can find an il​ lus​ tra​ tion with the dif​ fer​ ent ob​ jects that com​ pound a ten​ ant and how they are re​ lated. In the fol​ low​ ing sec​ tions each di​ ag ​ram shows the progress of how ACME Inc.

Tenants 165

Tenant Logical Model
There are 3 Ten​
ants pre​
con​
fig​
ured in the sys​
tem by de​
faut:
1

Common – a special tenant with the purpose of providing “common” services to
other tenants in the ACI fabric. Global reuse is a core principle in the common
tenant. Some examples of common services are:
a. Shared Private Networks
b. Shared Bridge Domains
c. DNS
d. DHCP
e. Active Directory

2

Infra – The Infrastructure tenant that is used for all internal fabric
communications, such as tunnels and policy deployment. This includes switch
to switch (Leaf, Spine, Application Virtual Switch (AVS)) and switch to APIC.
The infra tenant does not get exposed to the user space (tenants) and it has its
own private network space and bridge domains. Fabric discovery, image
management, and DHCP for fabric functions are all handled within this tenant.

3

Mgmt - The management tenant provides convenient means to configure
access policies for fabric nodes. While fabric nodes are accessible and
configurable through the APIC, they can also be accessed directly using in-band
and out-of band connections. In-band and out-of-band policies are configured
under the mgmt tenant:

In-Band Management Access

Out-of-Band Management Access

Tenants 167

Application Profile
An ap​
pli​
ca​
tion pro​
file is a con​
ve​
nient log​
i​
cal con​
tainer for mul​
ti​
ple hosts (phys​
i​
cal or
vir​
tual). You can cre​
ate Ap​
pli​
ca​
tion Pro​
file con​
tain​
ers based on a va​
ri​
ety of cri​
te​
ria,
such as what func​
tion the ap​
pli​
ca​
tion pro​
vides, how the ap​
pli​
ca​
tion looks from the
end-user per​
spec​
tive, where they are lo​
cated within the con​
text of the data cen​
ter, or
any other log​
i​
cal group​
ing rel​
a​
tive to the im​
ple​
men​
ta​
tion. Ap​
pli​
ca​
tion Pro​
file servers
are grouped in EPGs de​
pend​
ing on the use of com​
mon poli​
cies.
Ap​
pli​
ca​
tion Pro​
files pro​
vide a mech​
a​
nism to un​
der​
stand groups of servers as a sin​
gle
ap​
pli​
ca​
tion. This ap​
proach makes an ACI ap​
pli​
ca​
tion aware and al​
lows us to check the
op​
er​
a​
tional state for an ap​
pli​
ca​
tion mon​
i​
tor​
ing all the servers that are part of an ap​
pli​
ca​
tion as a whole and be​
come in​
formed about rel​
e​
vant faults and health sta​
tus for that
par​
tic​
u​
lar ap​
pli​
ca​
tion. Each Ap​
pli​
ca​
tion Pro​
file cre​
ated can have a unique mon​
i​
tor​
ing
pol​
icy and QOS pol​
icy.
An Ap​
pli​
ca​
tion Pro​
file is a child ob​
ject of the Ten​
ant and a sin​
gle Ten​
ant can con​
tain
mul​
ti​
ple Ap​
pli​
ca​
tion Pro​
files.

Adding components to a Tenant - 1. Application Profile

168 Tenants

Application Profile Configuration
Name - The name of the ap​
pli​
ca​
tion pro​
file.
Tags - A tag or meta​
data is a non-hi​
er​
ar​
chi​
cal key​
word or term as​
signed to the fab​
ric
mod​
ule.
Mon​
i​
tor​
ing Pol​
icy - The mon​
it​
or​
ing pol​
icy name for the EPG se​
man​
tic scope (op​
tional).

Create a New Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles.

4

In the Work pane, choose Actions > Create Application Profile.

5

In the Create Application Profile dialog box, perform the following actions:
a. Enter an Application Profile Name.
b. Enter a TAG (optional).
c. Choose a Monitoring Policy (optional).

6

Click Submit.

Modify Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles >
Application_Profile.

4
5

In the Work pane, choose policy.
In the Create Application Profile dialog box, perform the following actions:
a. Enter an Application Profile Name.
b. Enter an appropriate TAG (optional).
c. Choose the Monitoring Policy (optional).

6

Click Submit.

Tenants 169

Remove Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles
> Application_Profile.

4

In the Work pane, choose Actions > Delete.

Verify Application Profile
REST :: /api/node/class/fvAp.xml
CLI :: moquery -c fvAp

Tenants 171

Endpoint Group
End​
point Groups are used to cre​
ate log​
i​
cal group​
ings of hosts or servers that per​
form
sim​
i​
lar func​
tions within the fab​
ric. Each End​
point Group cre​
ated can have a unique
mon​
i​
tor​
ing pol​
icy and QoS pol​
icy and must be as​
so​
ci​
ated with a Bridge Do​
main.
An End​
point group is a child ob​
ject of the Ap​
pli​
ca​
tion Pro​
file and an Ap​
pli​
ca​
tion Pro​
file can con​
tain mul​
ti​
ple End​
point Groups. Each end​
point within an End​
point Group is
sus​
cep​
ti​
ble to the same pol​
icy in the Fab​
ric.

Adding components to a tenant - 2. End Point Group in the Application Profile
All the End​
points in​
side an EPG can com​
mu​
ni​
cate with each other. How com​
mu​
ni​
ca​
tions be​
tween EPGs con​
tracts will be re​
quired is gov​
erned by con​
tracts and not tra​
di​
tional Layer 2/Layer 3 for​
ward​
ing con​
structs. For ex​
am​
ple, Host-A in EPG-A can have
the IP ad​
dress/mask of 10.​
1.​
1.​
10/​
24 and Host B in EPG B can have the IP ad​
dress/mask 10.​
1.​
1.​
20/​
24 (note that both hosts be​
lieve they are "in the same sub​
net"). In
this case they would not be al​
lowed to com​
mu​
ni​
cate un​
less a con​
tract that per​
mit​
ted
con​
nec​
tiv​
ity ex​
isted be​
tween EPG-A and EPG-B. Con​
tracts will be ex​
plained in greater
de​
tail in a fol​
low​
ing sec​
tion.
Note that there are some types of End​
point Groups within the fab​
ric that are not con​
tained under Ap​
pli​
ca​
tion Pro​
files such as, Ap​
pli​
ca​
tion End​
point Group, Ex​
ter​
nal Bridge
Net​
works (aka Lay​
er2 Ex​
ter​
nal), Ex​
ter​
nal Routed Net​
works (aka as Lay​
er3 Ex​
ter​
nal) and
Man​
age​
ment End​
point Groups. These End​
point Groups might have spe​
cial re​
quire​
-

172 Tenants

ments, for ex​
am​
ple, in Ex​
ter​
nal Bridge Net​
works, MAC ad​
dresses of the end​
points are
not learnt by the leaf switches.
End​
point Groups are linked to Bridge Do​
mains but they will re​
ceive a VLAN ID dif​
fer​
ent
from the bridge do​
main, un​
less Bridge Do​
main legacy mode is used.
It is im​
por​
tant to un​
der​
stand that a sin​
gle sub​
net can be ex​
tended across sev​
eral EPGs.
Each EPG is iden​
ti​
fied by an en​
cap​
su​
la​
tion VLAN or VXLAN so that the same sub​
net will
be using dif​
fer​
ent en​
cap​
su​
la​
tion IDs across the fab​
ric. This con​
cept is dif​
fer​
ent from
tra​
di​
tional net​
work​
ing.

Endpoint Group Configuration
Name - The name for the end​
point group.
Tag - A tag or meta​
data is a non-hi​
er​
ar​
chi​
cal key​
word or term as​
signed to the fab​
ric
mod​
ule.
Qos Class - The QoS pri​
or​
ity class iden​
ti​
fier.
The class can be:

Unspecified

Level1-Class 1 Differentiated Services Code Point (DSCP) value.

Level2-Class 2 DSCP value.

Level3-Class 3 DSCP value.

Cus​
tom Qos - The QoS traf​
fic pri​
or​
ity class iden​
ti​
fier. The Cus​
tom class is a user-con​
fig​
urable DSCP value.
Bridge Do​
main - The name of the bridge do​
main as​
so​
ci​
ated with this ob​
ject.
it​
or​
ing pol​
icy name for the EPG se​
man​
tic scope (op​
tional).
Mon​
it​
or​
ing Pol​
icy – The mon​
As​
so​
ci​
ated Do​
main Pro​
file – A source re​
la​
tion to an in​
fra​
struc​
ture do​
main pro​
file as​
so​
ci​
ated with ap​
pli​
ca​
tion end​
point groups.

Tenants 173

Sub​
net - An End​
point Group sub​
net is rel​
e​
vant when and only when con​
fig​
ur​
ing route
leak​
ing be​
tween VRF's/Pri​
vate Net​
work within a Ten​
ant (op​
tional).
Sta​
tic End​
point - The sta​
tic client end​
point rep​
re​
sents a silent client end​
point at​
tached
to the net​
work (op​
tional).

Create a New Endpoint Group
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles >
Application_Profile > Application EPGs.

4
5

In the Work pane, choose Actions > Create Application EPG.
In the Create Application EPG dialog box, perform the following.
a. Enter an Application EPG Name.
b. Enter an Tag (optional).
c. Enter an Qos Class (optional).
d. Enter an Custom Qos (optional).
e. Enter a Bridge Domain Name.
f. Choose a Monitoring Policy (optional).
g. Enter an Associated Domain Profile Name.

6

Click Finish.

Modify Endpoint Group
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles
> Application_Profile > Application EPGs > Application_EPG.

4

In the Work pane, select policy.
a. Enter an Application EPG Name.
b. Enter an Tag (optional).
c. Enter an Qos Class (optional).
d. Enter an Custom Qos (optional).

174 Tenants e. choose Actions > Delete. choose the Tenant_Name. 2 In the Work pane. g. Verify Endpoint Group REST :: /api/node/class/fvAEPg. 5 Click Finish. choose Tenants > ALL TENANTS. Enter an Associated Domain Profile Name. Enter a Bridge Domain Name. Choose the appropriate Monitoring Policy if applicable (optional). 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile > Application EPGs > Application_EPG. f.xml CLI :: moquery -c fvAEPg . Remove Endpoint Group 1 On the menu bar.. 4 In the Work pane.

4 In the Work pane. and can be ei​ ther vir​ tual or phys​ ic ​al.Tenants 175 Endpoint End​ points are de​ vices that are con​ nected to the net​ work ei​ ther di​ rectly or in​ di​ rectly. Below is an ex​ am​ ple of a sta​ tic bind​ ing. 5 In the Deploy Static EPG on PC. You can add Sta​ tic End​ points by cre​ at​ ing Sta​ tic Bind​ ings within the End​ point Group. an en​ cap​ su​ la​ tion. 1 On the menu bar. and a de​ ploy​ ment Im​ me​ di​ acy mode as​ so​ ci​ ated with it. d. An End​ point is a child ob​ ject of the End​ point Group and an End​ point Group con​ struct can con​ tain mul​ ti​ ple End​ points. Choose the Path Type. 2 In the Work pane. and at​ trib​ utes. Click Submit. . c. perform the following actions: a. VPC or Interface. a lo​ ca​ tion. In order to show the end​ points that are con​ nected to the fab​ ric under cer​ tain EPGs: 1 On the menu bar. See the VVM sec​ tion for an ex​ am​ ple of a dy​ namic bind​ ing. choose Tenants > ALL TENANTS. Each end​ point has a path. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). choose the Tenant_Name. The End​ points ref​ er​ enced within the fab​ ric can be ei​ ther sta​ tic (de​ fined within the APIC) or dy​ namic (au​ to​ mated by vCen​ ter/Open​ stack). choose Tenants > ALL TENANTS. choose the Tenant_Name. Choose the Path. 2 In the Work pane. b. choose Actions > Deploy Static EPG on PC. Enter the encapsulation VLAN. VPC or Interface dialog box. End​ points have an ad​ dress (iden​ tity).

xml CLI :: moquery -c fvCEp .176 Tenants 3 In the Navigation Pane choose Tenant_Name > Application Profiles > Application_Profile > Application EPGs > Application_EPG. choose Operational. Verify Endpoint REST :: /api/node/class/fvCEp. 4 In the Work pane.

It is a unique Layer 3 for​ ward​ ing and ap​ pli​ ca​ tion pol​ icy do​ main. . pri​ vate Layer 3 net​ work. When there is a re​ quire​ ment to route traf​ fic be​ tween sep​ a​ rate Pri​ vate Net​ work in​ stances. Pri​ vate Net​ works cre​ ated in the com​ mon ten​ ant are shared glob​ ally within the fab​ ric.3.Tenants 177 Private Network A Pri​ vate Net​ work is also re​ ferred to as a VRF. Private Network as part of the Tenant Logical Model The most com​ mon method to share Pri​ vate Net​ works be​ tween ten​ ants is through the com​ mon ten​ ant. Adding components to a Tenant . This will be dis​ cussed in de​ tail in the Bridge Do​ main and EPG con​ fig​ u​ ra​ tion sec​ tions. a Pri​ vate Net​ work that is in​ tended to be used by mul​ ti​ ple ten​ ants and is not cre​ ated in the com​ mon ten​ ant re​ quires ex​ plicit con​ fig​ u​ ra​ tion to be shared. or con​ text. spe​ cial con​ sid​ er​ a​ tion for sub​ net con​ fig​ u​ ra​ tion is needed. All of the end​ points within the Pri​ vate Net​ work must have unique IP ad​ dresses be​ cause it is pos​ si​ ble to for​ ward pack​ ets di​ rectly be​ tween these de​ vices if the pol​ icy al​ lows it. How​ ever. Pri​ vate net​ works are a child of the Ten​ ant ob​ ject. see the overview sec​ tion of this chap​ ter. One or more bridge do​ mains are as​ so​ ci​ ated with a pri​ vate net​ work. For more in​ for​ ma​ tion about com​ mon ten​ ants.

The pre​ ferred pol​ icy con​ trol. i​ tor​ ing pol​ icy name for the Ten​ ant se​ man​ tic scope (op​ tional). Choose the appropriate DNS Label if applicable (optional). 4 5 In the Work pane. Mon​ i​ tor​ ing Pol​ icy .The mon​ DNS Label .Name of the OSPF timers pol​ icy as​ so​ ci​ ated with this ob​ ject. choose Actions > Create Private Network. Choose the appropriate Monitoring Policy if applicable (optional). Choose an End Point Retention Policy Name (optional). 2 In the Work pane. Enter an Private Network Name. Choose an OSPF Policy Name (optional). The val​ ues can be en​ forced or un​ forced is cho​ sen. Un​ en​ forced al​ lows all traf​ fic within the Pri​ vate Net​ work.The end point re​ ten​ tion pol​ icy name (op​ tional).The name of the Pri​ vate Net​ work. . perform the following actions: a. Pol​ icy En​ force​ ment . 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks.The net​ work do​ main name label. choose the Tenant_Name. choose Tenants > ALL TENANTS. c. BGP Timers . The de​ fault is en​ forced.Name of the BGP timers pol​ OSPF Timers . g. La​ bels en​ able clas​ si​ fy​ ing which ob​ jects can and can​ not com​ mu​ ni​ cate with one an​ other (op​ tional). Creating a New Private Network 1 On the menu bar. In the Create Private Network dialog box. icy as​ so​ ci​ ated with this ob​ ject. d. 6 Click Finish. Choose a Policy Enforcement (optional).178 Tenants Private Network Configuration Parameters Name . b. e. con​ tracts be​ tween EPGs are re​ quired to allow en​ forced. f. End Point Re​ ten​ tion Pol​ icy . Choose a BGP Policy Name (optional). When en​ traf​ fic.

choose Tenants > ALL TENANTS. g.Tenants 179 Modify Private Network 1 On the menu bar. d. b. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network. 5 Click Finish. Verify Private Network REST :: /api/node/class/fvCtx. a. Choose a BGP Policy Name (optional). Choose an End Point Retention Policy Name (optional). 4 In the Work pane. Choosesa Policy Enforcement (optional). f. Remove Private Network 1 On the menu bar.xml CLI :: moquery -c fvCtx . choose Actions > Delete. e. 2 In the Work pane. Choose the appropriate Monitoring Policy if applicable (optional). choose the Tenant_Name. Choose an EIGRP Policy Name (optional). Choose an OSPF Policy Name (optional). select policy. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network. c. choose Tenants > ALL TENANTS. 4 In the Work pane. 2 In the Work pane. Choose the appropriate DNS Label if applicable (optional).

.

Bridge do​ mains will span all switches in which as​ so​ ci​ ated EPG are con​ fig​ ured. . Bridge Domain as part of the Tenant Application Profile The fol​ low​ ing image pro​ vides an ex​ am​ ple of a ten​ ant to show how bridge do​ mains are con​ tained in​ side of Pri​ vate Net​ works and how they are linked to EPGs and the other el​ e​ ments. A Bridge Do​ main is a child of the Ten​ ant ob​ ject and must be linked to a Pri​ vate Net​ work. The bridge do​ main de​ fines the unique Layer 2 MAC ad​ dress space and a Layer 2 flood do​ main if flood​ ing is en​ abled. A bridge do​ main can have mul​ ti​ ple sub​ nets. While a Pri​ vate Net​ work de​ fines a unique IP ad​ dress space. Adding components to a Tenant . that ad​ dress space can con​ sist of mul​ ti​ ple sub​ nets. Those sub​ nets will be spread across one or more bridge do​ mains con​ tained in the Pri​ vate Net​ work.4.Tenants 181 Bridge Domain A Bridge Do​ main is the ab​ stract rep​ re​ sen​ ta​ tion of a Layer 2 for​ ward​ ing do​ main within the fab​ ric. a sub​ net is con​ tained within a sin​ gle bridge do​ main. How​ ever.

. under the Ten​ ant Acme. Each Bridge Do​ main re​ ceives a VLAN ID in the leaf switches. You in​ stead should think of it as a dis​ trib​ uted switch. From a prac​ ti​ cal per​ spec​ tive. which. on a leaf. Each PI VLAN is then linked to a VXLAN ID that will be used for for​ ward​ ing pur​ poses in​ side of the fab​ ric.182 Tenants End Point Group as part of the Tenant Application Profile It is im​ por​ tant to un​ der​ stand that a bridge do​ main is NOT a VLAN. each Bridge Do​ main will exist in a par​ tic​ u​ lar leaf if there is a con​ nected end​ point that be​ longs to that EPG. In the fol​ low​ ing ex​ am​ ple. This VLAN ID is sig​ nif​ i​ cant lo​ cally in the leaf switches and there​ fore it might be dif​ fer from one to other leaf switch. This VLAN con​ cept is dif​ fer​ ent from tra​ di​ tional net​ work​ ing and it is not used to for​ ward traf​ fic but as an iden​ ti​ fier. can be trans​ lated lo​ cally as a VLAN with local sig​ nif​ i​ cance. The VLAN ID used is also called Plat​ form In​ de​ pen​ dent VLAN or PI VLAN. al​ though it can act sim​ i​ lar to a VLAN. the bridge do​ main Acme-Ap​ pli​ ca​ tions-BD was as​ signed the PI VLAN ID 42 in the Leaf-1.

EPGs con​ tained in bridge do​ mains in legacy mode can​ not com​ mu​ ni​ cate with EPGs in ACI bridge do​ mains. for ex​ am​ ple dur​ ing some mi​ gra​ tion processes. When a Sub​ net is de​ fined in a Bridge Do​ main. For more de​ tails refer to the EP sec​ tion in this chap​ ter. sev​ eral VLANs will be used for EPs in​ side on one bridge do​ main. there is a fea​ ture called Bridge do​ main legacy mode. In that way. For this sit​ u​ a​ tion. This VLAN ID is dif​ fer​ ent from the Bridge Do​ main. also know as a per​ va​ sive gate​ way. the leaf switches will be the de​ fault gate​ way for the EPGs using that sub​ net. In some sit​ u​ a​ tions where tra​ di​ tional net​ work​ ing is re​ quired. This means that an SVI will be con​ fig​ ured under the VRF that rep​ re​ sents the pri​ vate . If the EPGs have end​ points on mul​ ti​ ple leaves. There​ fore in ACI. each leaf will con​ fig​ ure the de​ fault gate​ way. It is thought to use tra​ di​ tional net​ work​ ing within ACI but it lim​ its the ACI fea​ tures that can be used. Bridge do​ main in legacy mode is not rec​ om​ mended under ACI best prac​ tices. it might be re​ quired to have only one en​ cap​ su​ la​ tion VLAN per Bridge do​ main across all the leaf switches and EPGs that ref​ er​ ence that Bridge Do​ main. the de​ fault gate​ way for the end​ points will al​ ways be the first switch of the fab​ ric that is reached.Tenants 183 VLAN output from Leaf node EPGs are also as​ signed with a PI VLAN ID that is lo​ cally sig​ nif​ i​ cant in each leaf.

uni​ cast rout​ ing will be per​ formed on the tar​ get IP ad​ dress. there will be only one SVI per Bridge Do​ main but it will use sec​ ondary IP ad​ dresses. ing pol​ icy name.The Layer 3 out​ side in​ ter​ face iden​ ti​ fier con​ trol​ ling con​ nec​ tiv​ ity to out​ side net​ works. .184 Tenants net​ work that the Bridge Do​ main is linked to. L3 Out for Route Pro​ file .Op​ ti​ mize/Cus​ tom L2 Un​ known Uni​ cast . By ex​ am​ in​ ing (snoop​ ing) IGMP IGMP Snoop Pol​ icy .The for​ ward​ ing method for un​ known layer 2 des​ ti​ na​ tions. Every BD by de​ fault takes the fab​ ric wide de​ fault mac ad​ dress. mul​ ti​ cast traf​ fic is lim​ ited to the sub​ set of VLAN in​ ter​ faces on which the hosts re​ side.The for​ te​ ria (IP or MAC ad​ dress). Flood​ ing is dis​ abled by de​ fault but can be en​ abled by click​ ing the check box. Net​ work . erty to spec​ ify whether ARP flood​ ing is en​ abled. Un​ known Mul​ ti​ cast Flood​ ing . If a Bridge Do​ main has sev​ eral sub​ nets.The name of the Layer 3 out​ side in​ ter​ face as​ so​ ci​ ated with this ob​ ject.The node for​ ward​ ing pa​ ra​ me​ ter for un​ known Mul​ ti​ cast des​ ti​ na​ tions.The MAC ad​ dress of the bridge do​ main (BD) or switched vir​ tual in​ ter​ face (SVI).The IGMP Snoop​ mem​ ber​ ship re​ port mes​ sages from in​ ter​ ested hosts. For​ ward​ ing . De​ fault is Proxy. Uni​ cast rout​ ing is en​ abled by de​ fault but can be dis​ abled by click​ ing the check box.The as​ so​ ci​ ated layer 3 con​ text.A prop​ ing is dis​ abled.The name of the Bridge Do​ main. Con​ fig BD MAC Ad​ dress . ward​ ing method based on pre​ de​ fined for​ ward​ ing cri​ Uni​ cast Rout​ ing . As​ so​ ci​ ated L3 Outs . Bridge Domain Configuration Parameters Name . If flood​ ARP Flood​ ing .

The default is Private. configured.Defines subnets under an endpoint group.The con​ trol can be spe​ cific pro​ to​ cols ap​ plied to the sub​ net such as IGMP Snoop​ ing.The default is Unspecified.The mon​ it​ or​ ing pol​ icy name for the Ten​ ant se​ man​ tic scope (op​ tional). c. DHCP La​ bels .Defines subnets under a bridge domain. Choose the Network. • Querier IP .The net​ work do​ main name label. choose Actions > Create Bridge Domain. . to route leak to other Tenants within the Fabric. The scope can be: • Shared . Enter an Bridge Domain Name. Sub​ net Con​ trol . choose the Tenant_Name. perform the following actions: a. choose Tenants > ALL TENANTS.The as​ so​ ci​ ated route pro​ file name. with the Public option configured.Defines subnets under a bridge domain.Enables IGMP Snooping on the subnet. to share with Layer 3 outbound. It will be con​ fig​ ured in the leaf nodes which have EPGs in that bridge do​ main. • Private . Sub​ nets . 4 In the Work pane. The net​ work vis​ ib ​il​ ity of the sub​ net. with the Private option configured. The sub​ net is a por​ tion of a net​ work shar​ ing a par​ tic​ ul​ ar sub​ net ad​ dress. Choose the Forwarding Semantics (optional). b. Create a new Bridge Domain 1 On the menu bar. 2 In the Work pane.The IP ad​ dress and mask of the de​ fault gate​ way. with the Shared option • Public . The con​ trol can be: • Unspecified .Tenants 185 Route Pro​ file . 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains. 5 In the Create Bridge Domain dialog box. to only be used in that Tenant (will not be leaked). Mon​ i​ tor​ ing Pol​ icy .

Choose the L3 Out for Route Profile (optional). 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains > Bridge_Domain_Name. h. g. Choose the Network. Choose the IGMP Snoop Policy (optional). choose the Tenant_Name. 6 Click Submit. 2 In the Work pane. 5 Click Finish. 4 In the Work pane. d. choose the Policy tab and perform the following actions: a. f. Choose the Associated L3 Outs (optional). choose the Tenant_Name. 3 In the Navigation Pane choose Tenant_Name > Networking > Bridge Domain > Bridge Domain_Name. Choose the IGMP Snoop Policy (optional). choose Tenants > ALL TENANTS. b. g. Choose the Route Profile (optional). f. Remove a Bridge Domain 1 On the menu bar. i. Choose the Monitoring Policy if applicable (optional). Choose the DNS Label if applicable (optional). Choose the Forwarding Semantics (optional). Choose the DNS Label if applicable (optional). 4 In the Work pane. Modify a Bridge Domain 1 On the menu bar. Choose the L3 Out for Route Profile (optional). Choose the Subnets (optional). Choose the Monitoring Policy if applicable (optional). choose Actions > Delete. Choose the Route Profile (optional). Choose the Associated L3 Outs (optional). h. e. choose Tenants > ALL TENANTS. i. c. j.186 Tenants d. Choose the Subnets (optional). e. .

Tenants 187 Verify Bridge Domain REST :: /api/node/class/fvBD.xml CLI :: moquery -c fvBD .

.

choose Tenants > ALL TENANTS. . This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages: • Ability to use a single private network for all internal and external fabric • No route leaking needed between EPGs in different VRFs • Single Layer 3 Outside can be used by all tenants connectivity Dis​ ad​ van​ tages: • Changes to routing will impact all tenants From a con​ tain​ ment and re​ la​ tion​ ship per​ spec​ tive. this topol​ ogy looks as fol​ lows: Common Private Network for all Tenants To Con​ fig​ ure the com​ mon Ten​ ant pri​ vate net​ work: 1 On the menu bar. 2 In the Work pane.Tenants 189 Tenant Networking Use Cases Common Private Network for All Tenants This use case may be typ​ i​ cal for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate mul​ ti​ ple ten​ ants. choose the common. but place all within a sin​ gle pri​ vate net​ work in the fab​ ric.

choose Tenants > ALL TENANTS. 4 In the Work pane. Public and Shared. In the Gateway IP field enter the IP address for this subnet. choose the Tenant_Name. 4 In the Work pane. click +. f. In the Scope field you have the option to select Private. d. b. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains. c. 1 On the menu bar. Enter a Bridge Domain Name. Now the net​ work ad​ min​ is​ tra​ tor will have to as​ so​ ci​ ate the com​ mon pri​ vate net​ work to the Ten​ ant by first cre​ at​ ing a bridge do​ main. choose Actions > Create Bridge Domain. c.190 Tenants 3 In the Navigation pane choose common > Networking > Private Networks > default. Note: By default the Private option is selected. 5 Click Finish. 5 In the Create Bridge Domain dialog box. 6 Click OK. b. Choosea BGP Policy Name (optional). e. d. 2 In the Work pane. . Choose a Policy Enforcement (optional). a. In the Subnets field. perform the following actions: a. Choose an OSPF Policy Name (optional). The Ten​ ant has been cre​ ated. e. choose policy. For more information on what to select please reference the External Layer 3 section. Choose network. Choosean End Point Retention Policy Name (optional). 7 Click Finish. Choosethe appropriate DNS Label if applicable (optional). Choose the appropriate Monitoring Policy if applicable (optional).

Tenants 191 The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'default' moconfig commit # subnet cd '/aci/tenants/Cisco/networking/bridge-domains/Cisco/subnets' mocreate '172.0.16.1/24' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit # criterion cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs/EPG1/vm-attributescriteria' mocreate 'default' moconfig commit This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API .

This par​ tic​ ul​ ar use case demon​ strates how a pri​ vate net​ work can be as​ so​ ci​ ated with each ten​ ant.16.1/24" preferred="no" scope="private"/> </fvBD> <fvAp name="App1"> <fvAEPg matchT="AtleastOne" name="EPG1"> <fvRsBd tnFvBDName="Cisco"/> </fvAEPg> </fvAp> <fvRsTenantMonPol tnMonEPGPolName=""/> </fvTenant> For many multi-ten​ ant en​ vi​ ron​ ments it is de​ sir​ able to allow each ten​ ant to man​ age and own their own ad​ dress space and not be con​ cerned with over​ laps be​ tween other ten​ ants.192 Tenants XML : Ten​ ant Cisco <fvTenant name="Cisco"> <fvBD arpFlood="no" multiDstPktAct="bd-flood" name="Cisco" unicastRoute="yes" unkMacUcastAct="proxy" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="default"/> <fvSubnet ctrl="nd" descr="" ip="172. One Pri​ vate Net​ work per Ten​ ant with In​ tra-EPG com​ mu​ ni​ ca​ tions Ad​ van​ tages: • Allow for maximum isolation between tenants • Ability to address hosts in tenants with overlapping IP addresses Dis​ ad​ van​ tages: • Increased complexity when needing to allow EPG communication between different tenants with dedicated VRF .0.

Enter a Tenant Name. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. . 2 In the Create Tenant dialog box. perform the following actions: a. choose Tenants > ALL TENANTS. 3 Click Finish. b. Click Next. Enter Private Network Name. Click Next.Tenants 193 The ob​ ject con​ tain​ ment for this par​ tic​ u​ lar setup can be de​ picted as shown below: Private Network per Tenant To cre​ ate the ten​ ant: 1 On the menu bar. b. choose Actions > Create Private Network. choose Tenants > ADD TENANT. In the Create Private Network dialog box. The Ten​ ant has been cre​ ated. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. 4 5 In the Work pane. 1 On the menu bar. 2 In the Work pane. perform the following actions: a.

f. In the Gateway IP field enter the IP address for this subnet. d. e. In the Subnets field. Public and Shared.1/24' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit . The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # subnet cd '/aci/tenants/Cisco/networking/bridge-domains/Cisco/subnets' mocreate '172. Enter Associated Bridge Domain Name. 7 Click Finish. 6 Click OK.194 Tenants c. click +. For more information on what to select please reference the External Layer 3 section. Note: By default the Private option is selected. In the Scope field you have the option to select Private.16.0.

0.16. This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages: • Ability to have overlapping subnets within a single tenant Dis​ ad​ van​ tages: • EPGs residing in overlapping subnets cannot have policy applied between one another .Tenants 195 This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : Ten​ ant Cisco <fvTenant name="Cisco"> <fvBD arpFlood="no" multiDstPktAct="bd-flood" name="Cisco" unicastRoute="yes" unkMacUcastAct="proxy" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="Cisco"/> <fvSubnet ctrl="nd" ip="172. This may be a re​ sult of need​ ing to pro​ vide multi-ten​ ancy at a net​ work level. It may also be caused by need​ ing to sup​ port over​ lap​ ping sub​ nets within a sin​ gle ten​ ant. due to merg​ ers and ac​ qui​ si​ tions or other busi​ ness changes. but not at a man​ age​ ment level.1/24" name="" preferred="no" scope="private"/> </fvBD> <fvCtx knwMcastAct="permit" name="Cisco" pcEnfPref="enforced"/> <fvAp name="App1" prio="unspecified"> <fvAEPg name="EPG1"> <fvRsBd tnFvBDName="Cisco"/> </fvAEPg> </fvAp> </fvTenant> Multiple Private Networks with Intra-Tenant Communication An​ other use case that may be de​ sir​ able to sup​ port is the op​ tion to have a sin​ gle ten​ ant with mul​ ti​ ple pri​ vate net​ works.

196 Tenants The ob​ ject con​ tain​ ment for this par​ tic​ u​ lar setup can be de​ picted as shown below: Multiple Private Networks with Intra-Tenant communication To cre​ ate the ten​ ant: 1 On the menu bar. 2 In the Work pane. 1 On the menu bar. Click Next. In the Create Private Network dialog box. choose the Tenant_Name. . choose Actions > Create Private Network. In the Name field enter a name for the Private Network. 4 Click Finish. 3 Click Next. perform the following actions: a. b. In the Name field enter a name for the bridge domain. 2 In the Create Tenant dialog box. 4 5 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. Enter a Tenant Name. choose Tenants > ADD TENANT. perform the following actions: a. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. The Ten​ ant has been cre​ ated. c. choose Tenants > ALL TENANTS.

In the Create Application Profile dialog box. choose the Tenant_Name. you can move to the Ap​ pli​ ca​ tion Pro​ file. In the Scope field select Shared. f. In the Subnets field click +. f. 1 On the menu bar. In the Name field enter a name for the second bridge domain. choose Actions > Create Private Network. 4 5 In the Action Tab select Create Application Profile. perform the following actions: a.Tenants 197 d. 9 10 In the Work pane. 2 In the Work pane. In the Gateway IP field enter the IP address for this subnet. 6 Click Submit. c. choose Tenants > ALL TENANTS. perform the following actions: a. . In the Name field enter a name for the Application Profile. Click Next. Now that the two pri​ vate net​ works and bridge do​ mains have been cre​ ated. In the Scope field select Shared. b. d. 8 In the Navigation pane choose Tenant_Name > Networking > Private Networks. 6 Click OK. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles. In the Gateway IP field enter the IP address for this subnet. 7 Click Finish. In the Name field enter a name for the second Private Network. 11 Click OK. In the Create Private Network dialog box. In the Subnets field click +. e. 12 Click Finish. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). e.

choose Tenants > ALL TENANTS. In the Create Application EPG dialog box. As the ten​ ant ad​ min​ is​ tra​ tor. In the IP Protocol column select ICMP. Click +. b. In the Name field enter a name for the Filter. b. you would have the knowl​ edge of the fil​ ters re​ quired to per​ mit traf​ fic across the two EPGs. 2 In the Work pane. 7 Click Submit. perform the following actions: a. In the Bridge Domain field. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Filters. choose Actions > Create Filters. 6 Click Update. 4 5 In the Work pane. 2 In the Work pane. 1 On the menu bar. In the Name field enter a name for the End Point Group. choose Actions > Create Application EPG. In the Create Filter dialog box. The ten​ ant ad​ min​ is​ tra​ tor will have to now cre​ ate a con​ tract and fil​ ter be​ tween the two EPGs. e. c. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name. Re​ peat these steps for the sec​ ond EPG. 6 Click Finish. perform the following actions: a. choose the Tenant_Name. you can re​ peat the process of defin​ ing . choose Tenants > ALL TENANTS. 4 5 In the Work pane. In the Name column enter ICMP. In the fil​ ter. select the appropiate bridge domain. choose the Tenant_Name.198 Tenants To cre​ ate the two end​ point groups: 1 On the menu bar. d. In the Ethertype select IP.

Tenants 199 var​ io ​us dif​ fer​ ent net​ work pro​ to​ cols as re​ quired for your ap​ pli​ ca​ tions. Each EPG that you have cre​ ated will then ei​ ther con​ sume or pro​ vide that con​ tract to es​ tab​ lish a re​ la​ tion​ ship be​ tween both EPGs. a. c. In the Action Tab select Add Consume Contract. 5 In the Add Provided Contract dialog box. f.. Select the created contract . In the Create Contract dialog box. 1 On the menu bar. 6 7 In the Navigation pane under the second EPG select Contracts. 1 On the menu bar. Now you have to de​ fine the con​ tract that will be con​ sumed and pro​ vided by the two EPGs. choose the Tenant_Name. e. 7 Click OK. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contract. perform the following actions: a. choose the Tenant_Name. In the Name field enter a name for the Subject. 4 5 In the Work pane. Select the created contract. perform the following actions: a. Click Subjects field click +. In the Name field enter a name for the Filter. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Contracts. 4 In the Work pane. Click Submit. 2 In the Work pane. 6 Click Update. b. d. As​ sign the Con​ tract be​ tween the EPGs. A con​ tract is as​ signed to an EPG as ei​ ther a con​ sumed or a pro​ vided con​ tract. b. 8 Click Submit. 2 In the Work pane. In the Filter Chain field click +. choose Tenants > ALL TENANTS. choose Actions > Create Contract. Select the Filter you just created. In the Scope field select Global. choose Tenants > ALL TENANTS. choose Actions > Add Provided Contract.

200 Tenants 8 Click Submit. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco1' cd 'Cisco1' moset network 'Cisco1' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco1' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' .

1.1/24' cd '172.1.1:24' moset scope 'private.16.shared' moconfig commit This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : Ten​ ant Cisco <fvTenant dn="uni/tn-Cisco" name="Cisco"> <vzBrCP name="ICMP" scope="tenant"> <vzSubj consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" .Tenants 201 moconfig commit # fv-rscon cd '/aci/tenants/Cisco/application-profiles/App1/applicationepgs/EPG1/contracts/consumed-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs/EPG1/subnets' mocreate '172.1/24' cd '172.16.1:24' moset scope 'private.16.2.shared' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG2' cd 'EPG2' moset bridge-domain 'Cisco1' moconfig commit # fv-rsprov cd '/aci/tenants/Cisco/application-profiles/App/applicationepgs/EPG2/contracts/provided-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/CCO/application-epgs/EPG2/subnets' mocreate '172.2.16.

1/24" scope="private.1.16.1/24" scope="private.202 Tenants revFltPorts="yes"> <vzRsSubjFiltAtt tnVzFilterName="icmp"/> </vzSubj> </vzBrCP> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvCtx knwMcastAct="permit" name="CiscoCtx2" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx2"/> </fvBD> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="Web"> <fvRsCons tnVzBrCPName="ICMP"/> <fvRsPathAtt encap="vlan-1201" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-201/pathep-[eth1/16]"/> <fvSubnet ip="172.16.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD2"/> </fvAEPg> <fvAEPg matchT="AtleastOne" name="App"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-202/pathep-[eth1/2]"/> <fvSubnet ip="172.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsProv matchT="AtleastOne" tnVzBrCPName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> Multiple Private Networks with Inter-Tenant Communication .2.

This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages • Each tenant container can be managed separately • Allows for maximum isolation between tenants Dis​ ad​ van​ tages • Tenant address space must be unique From a con​ tain​ ment and re​ la​ tion​ ship per​ spec​ tive. . 2 In the Create Tenant dialog box perform the following actions: a. this topol​ ogy looks as fol​ lows: Multiple Private Networks with Inter-Tenant Communication To cre​ ate the ten​ ant: 1 In the GUI Navigate to Tenants > ADD TENANT.Tenants 203 Multiple Private Networks with Inter-Tenant Communication This use case may be typ​ i​ cal for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate mul​ ti​ ple ten​ ants with the abil​ ity to sup​ port in​ ter-ten​ ant com​ mu​ ni​ ca​ tions. 3 Click Next. In the Name field enter a name for the first Tenant.

3 In the Navigation Pane choose Tenant_Name > Networking > Private Networks. In the Scope field select Shared. In the Gateway IP field enter the IP address for this subnet. b. 7 Click Finish. d. 6 Click Submit. Click Next. choose Actions > Create Application Profile. perform the following actions: a. 2 In the Work pane. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). 6 Click OK. perform the following actions: a. 3 In the Navigation Pane choose Tenant_Name > Networking > Application Profiles. In the Create Application Profile dialog box. In the Name field enter a name for the Private Network. In the Name field enter a name for the bridge domain. 1 On the menu bar. 5 In the Create Private Network dialog box. 2 In the Work pane. The ten​ ant has been cre​ ated. 4 5 In the Work pane. To cre​ ate the ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. c. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. choose the Tenant_Name. f. choose the Tenant_Name. choose Tenants > ALL TENANTS. choose Actions > Create Private Network. In the Subnets field click +. . e. choose Tenants > ALL TENANTS. In the Name field enter a name for the Application Profile. 4 In the Work pane.204 Tenants 4 Click Finish.

choose Tenants > ALL TENANTS. 6 Click Finish. choose the Tenant_Name. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. 2 In the Create Tenant dialog box perform the following actions: a. 4 Click Finish. e. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name 4 5 In the Work pane. 2 In the Work pane. 5 In the Create Private Network dialog box. In the Create Application EPG dialog box. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. In the Bridge Domain field. b. choose Tenants > ALL TENANTS. c. 3 Click Next. perform the following actions: a. In the Subnets field click +. In the Gateway IP field enter the IP address for this subnet. Click Next. In the Name field enter a name for the Private Network. The ten​ ant has been cre​ ated. To cre​ ate the sec​ ond ten​ ant and ap​ pli​ ca​ tion pro​ file: 1 In the GUI Navigate to Tenants > ADD TENANT. In the Name field enter a name for the bridge domain.Tenants 205 To cre​ ate the end​ point group: 1 On the menu bar. choose the Tenant_Name. choose Actions > Create Private Network. select the appropiate bridge domain. . 4 In the Work pane. b. 1 On the menu bar. In the Name field enter a name for the End Point Group. 2 In the Work pane. In the Name field enter a name for the first Tenant. perform the following actions: a. choose Actions > Create Application EPG. d.

In the Name field enter a name for the Application Profile. choose the Tenant_Name. b. 4 5 In the Work pane.206 Tenants f. 6 Click Submit. 6 Click OK. 2 In the Work pane. 2 In the Work pane. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). In the Bridge Domain field. select the appropiate bridge domain. To cre​ ate the end​ point group: 1 On the menu bar. 6 Click Finish. perform the following actions: a. 7 Click Finish. . choose the Tenant_Name. choose Tenants > ALL TENANTS. choose Tenants > ALL TENANTS. To cre​ ate the ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. perform the following actions: a. In the Name field enter a name for the End Point Group. 1 On the menu bar. In the Create Application Profile dialog box. 2 In the Work pane. In the Create Application EPG dialog box. The ten​ ant ad​ min​ is​ tra​ tor will have to now cre​ ate a con​ tract and fil​ ter be​ tween the two EPGs. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name 4 5 In the Work pane. choose Actions > Create Application EPG. choose Tenants > ALL TENANTS. choose Actions > Create Application Profile. choose the Tenant_Name. In the Scope field select Shared.

Select the filter you just created. In the fil​ ter you can re​ peat the process of defin​ ing var​ io ​us dif​ fer​ ent net​ work pro​ to​ cols as re​ quired for your ap​ pli​ ca​ tions. 4 5 In the Work pane. 7 Click OK. In the Create Contract dialog box. f. . 2 In the Work pane. Now you have to de​ fine the con​ tract that will be con​ sumed and pro​ vided by the two EPGs. d. b. c. c. e. 4 In the Work pane. 6 Click Update. 8 Click Submit. e. Click +. choose Actions > Create Contract. In the Scope field choose Global. As the ten​ ant ad​ min​ is​ tra​ tor you would have the knowl​ edge of the fil​ ters re​ quired to per​ mit traf​ fic across the two EPGs. 6 Click Update. 7 Click Submit. perform the following actions: a. choose the Tenant_Name. 1 On the menu bar. 5 In the Create Filter dialog box. In the Name field enter a name for the Filter. b. Click Subjects field click +. In the Name field enter a name for the Subject. In the Ethertype select IP. In the Filter Chain field click +. perform the following actions: a. choose Actions > Create Filters. In the Name column enter ICMP. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Contracts. In the IP Protocol column select ICMP. choose Tenants > ALL TENANTS.Tenants 207 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Filters. d. In the Name field enter a name for the filter.

4 In the Work pane. 8 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contracts. 9 In Work pane. Each EPG that you have cre​ ated will then ei​ ther con​ sume or pro​ vide that con​ tract to es​ tab​ lish a re​ la​ tion​ ship be​ tween both EPGs. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : TEN​ ANT Cis​ co1 # tenant cd '/aci/tenants' mocreate 'Cisco1' moconfig commit # bridge-domain cd '/aci/tenants/Cisco1/networking/bridge-domains' mocreate 'Cisco1' cd 'Cisco1' moset network 'Cisco1' moconfig commit . 7 Select the second created Tenant. b. perform the following actions: a. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contracts. A con​ tract is as​ signed to an EPG as ei​ ther a con​ sumed or a pro​ vided con​ tract. 10 Click Submit. 6 In the GUI Navigate to Tenants > ALL TENANTS. choose Tenants > ALL TENANTS. 2 In the Work pane. choose Action > Add Consume Contract. Click Submit. a. choose the Tenant_Name. choose Actions > Add Provided Contract. 1 On the menu bar. 5 In the Add Provided Contract dialog box. Select the created Contract. Select the created Contract.208 Tenants As​ sign the Con​ tract be​ tween the EPGs.

16.16.1/24' cd '172.1.1.Tenants 209 # private-network cd '/aci/tenants/Cisco1/networking/private-networks' mocreate 'Cisco1' moconfig commit # application-profile cd '/aci/tenants/Cisco1/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco1/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco1' moconfig commit # fv-rsprov cd '/aci/tenants/Cisco/application-profiles/CCO/applicationepgs/App/contracts/provided-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/CCO/application-epgs/App/subnets' mocreate '172.1:24' moset scope 'private.shared' moconfig commit # contract cd '/aci/tenants/Cisco/security-policies/contracts' mocreate 'ICMP' cd 'ICMP' moset scope 'global' moconfig commit # contract-subject cd '/aci/tenants/Cisco/security-policies/contracts/ICMP/subjects' mocreate 'icmp' moconfig commit # vz-rssubjfiltatt cd '/aci/tenants/Cisco/security-policies/contracts/ICMP/subjects/icmp/common-filters' mocreate 'icmp' moconfig commit .

210 Tenants CLI : TEN​ ANT Cis​ co2 # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco2/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit # fv-rsconsif cd '/aci/tenants/Cisco1/application-profiles/CCO/applicationepgs/Web/contracts/consumed-contract-interfaces' mocreate 'CiscoInterTenantICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco1/application-profiles/CCO/application-epgs/Web/subnets' mocreate '172.16.2.2.1/24' cd '172.16.1:24' moset scope 'shared-subnet' moconfig commit # imported-contract cd '/aci/tenants/Cisco1/security-policies/imported-contracts' mocreate 'CiscoInterTenantICMP' cd 'CiscoInterTenantICMP' moset contract 'tenants/Cisco/security-policies/contracts/ICMP' moconfig commit .

Tenants 211 This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : TEN​ ANT Cis​ co1 <fvTenant dn="uni/tn-Cisco1" name="Cisco1"> <vzBrCP name="ICMP" scope="global"> <vzSubj consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" revFltPorts="yes"> <vzRsSubjFiltAtt tnVzFilterName="icmp"/> </vzSubj> </vzBrCP> <vzCPIf dn="uni/tn-Cisco1/cif-ICMP" name="ICMP"> <vzRsIf consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" revFltPorts="yes"> <vzRsSubjFiltAtt tDn="uni/tn-Cisco2/brc-default"/> </vzRsIf> </vzCPIf> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx2"/> </fvBD> <fvBD arpFlood="yes" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="EPG1"> .

1.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsProv matchT="AtleastOne" tnVzBrCPName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> XML : TEN​ ANT Cis​ co2 <fvTenant dn="uni/tn-Cisco2" name="Cisco2"> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvBD arpFlood="yes" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="EPG2"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-201/pathep-[eth1/2]"/> <fvSubnet ip="172.16.1/24" scope="private.1.1/24" scope="private.16.212 Tenants <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-202/pathep-[eth1/2]"/> <fvSubnet ip="172.shared"/> .

Tenants 213 <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsConsIf matchT="AtleastOne" tnVzBrCPIfName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> .

.

215 Working with Contracts .

.

Working with Contracts 217 Section Content • Contracts Contract Configuration Parameters Create/Modify/Remove Contracts Create Contracts Modify Contracts Remove Contracts Verify Contracts • Apply/Remove EPG Contracts Apply a Contract to an EPG Remove a Contract from EPG Verify Contract on EPG • Apply/Remove External Network Contracts Apply a Contract to an External Network Remove a Contract from an External Network Verify External Network Contracts • Apply/Remove Private Network Contracts Apply a Contract to a Private Network (vzAny) Remove a Contract From a Private Network (vzAny) Verify Private Network Contracts • Filters Filter Entry Configuration Parameters Create Filters Modify Filters Remove Filters Verify Filters .

or Delete Taboo Contracts Create Taboo Contracts Modify Taboo Contracts Delete Taboo Contracts Verify Taboo Contracts • Apply/Remove Taboo Contracts Apply Taboo Contract to an EPG Remove Taboo Contract from EPG Verify Taboo Contracts Applied to EPG • Inter-Tenant Contracts Configuration Parameters • Create/Modify/Remove Export Contracts Export Contract Modify Exported Contracts Remove Exported Contracts Verify Exported Contracts • Contracts Use Cases Inter-Tenant Contracts Tenant Cisco-1/EPG-1 Tenant Cisco-2/EPG-2 • Inter-Private Network Contracts Communication Tenant Cisco-1/EPG-1 Tenant Cisco-1/EPG-2 .218 Working with Contracts • Taboo Contracts Taboo Contract Configuration Parameters Create. Modify.

Working with Contracts 219 • Single Contract Bidirectional Reverse Filter • Single Contract Unidirectional with Multiple Filters • Multiple Contracts Uni-Directional Single Filter .

.

Used optionally to group objects such as subjects and EPGs for the purpose of further defining policy enforcement EPGs can only com​ mu​ ni​ cate with other EPGs based upon the con​ tract rules de​ fined.Permit. and the Ap​ pli​ ca​ tion EPG pro​ vides a con​ tract which the Data​ base EPG would con​ sume. and service graph to perform Ethernet type. The Web EPG pro​ vides a con​ tract which is con​ sumed by the Ap​ pli​ ca​ tion EPG. Con​ tracts are com​ prised of the fol​ low​ ing items: • Subjects . protocol type. deny. There is no con​ tract re​ quired for in​ tra-EPG com​ mu​ ni​ ca​ tion: in​ tra-EPG com​ mu​ ni​ ca​ tion is al​ lowed by de​ fault.Used to classify traffic based upon layer 2 to layer 4 attributes (such as • Actions . copy. EPGs may acts a both a provider and con​ sumer of the same con​ tract eas​ ily by mak​ ing the con​ tract bidi​ rec​ tional.A group of filters for a specific application or service • Filters . TCP flags and ports) matches based upon filters • Labels . The ex​ am​ ple below. shows how con​ tracts would con​ trol traf​ fic flow be​ tween EPGs in a 3-tiered ap​ pli​ ca​ tion. . mark.Working with Contracts 221 Contracts Con​ tracts pro​ vide a way for the ACI ad​ min​ is​ tra​ tor to con​ trol traf​ fic flow within the ACI fab​ ric be​ tween EPGs. These con​ tracts are built using a provider-con​ sumer model where one EPG pro​ vides the ser​ vices it wants to offer and an​ other EPG con​ sumes them. redirect. log.

for example if inband management is configured for the ACI fabric and certain EPGs are to be allowed to access it Contract Configuration Parameters When con​ fig​ ur​ ing con​ tracts you can de​ fine the fol​ low​ ing op​ tions: Con​ tract Scope . • Application-profile .This contract will be applied for endpoint groups within the same tenant.The scope of a ser​ vice con​ tract be​ tween two or more par​ tic​ i​ pat​ ing peer en​ ti​ ties.This contract will be applied for endpoint groups associated with the same private network. The states are: • Context .This contract will be applied for endpoint groups in the • Tenant . same application profile.222 Working with Contracts Contract Policies Between End Point Groups Con​ tracts gov​ ern the fol​ low​ ing types of end​ point group com​ mu​ ni​ ca​ tions: • Between application EPGs • Between application EPGs and external networks • Between application EPGs and in-band management EPG. .

You can as​ sign the same tag name to mul​ ti​ ple ob​ jects and you can as​ sign one or more tag names to an ob​ ject. • Level3 . QoS Class . Tags .Class 3 DSCP value.This contract will be applied for endpoint groups throughout the fabric. The op​ tions are: • AtleastOne • AtmostOne • None • All The de​ fault is Atlea​ st​ O ​ne.Working with Contracts 223 • Global . A tag al​ lows you to group mul​ ti​ ple ob​ jects by a de​ scrip​ tive name. The pri​ or​ ity level can be: • Unspecified • Level1 .Class 1 Differentiated Services Code Point (DSCP) value. . 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts.The sub​ ject match cri​ te​ ria across con​ sumers. Create/Modify/Remove Contracts Create Contracts 1 On the menu bar. 2 In the Work pane. Match . choose the Tenant_Name. The de​ fault is Con​ text.The search key​ word or term that is as​ signed to the ap​ pli​ ca​ tion pro​ file. • Level2 . The de​ fault is Un​ spec​ i​ fied. choose Tenants > ALL TENANTS.Class 2 DSCP value.The pri​ or​ ity level of the ser​ vice con​ tract.

see the "Filters" section. choose the Tenant_Name. i. choose the Policy tab. b. 2 In the Work pane. b. choose Tenants > ALL TENANTS. 8 Click Submit. 2. Click + next to the Subject field. perform the following actions: 1. Modify Contracts 1 On the menu bar. Note: For information regarding filter creation. 7 Click Submit. 6 Click OK. Click + in the Filter Chain field. Click + next to the Subject to add a Contract Subject. Remove Contracts . c. Choose a QoS Class (optional). i. to add a Contract Subject. In the Create Contract Subject dialog box. In the Create Contract Subject dialog box. a. In the Create Contract dialog box. 7 Click OK. Enter a Contract Subject Name. 5 Click Update. 4 In the Work pane. Click + next to Filter Chain. choose Actions > Create Contract. Enter a Contract Name. d.​ For information regarding filter creation.224 Working with Contracts 4 5 In the Work pane. 6 Click Update. Choose a Qos Class (optional). Choose a Contract Scope (optional). 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. Choose a Contract Scope (optional). perform the following actions: 1. Enter a Contract Subject Name. perform the following actions: a. see the "Filters" section. c. 2.

. Verify Contracts REST :: /api/node/class/vzBrCP. perform the following actions: a. Remove a Contract from EPG 1 On the menu bar. b. 2 In the Work pane. choose the Tenant_Name. 4 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. choose Tenants > ALL TENANTS. c.xml CLI :: moquery -c vzBrCP Apply/Remove EPG Contracts Apply a Contract to an EPG 1 On the menu bar. choose Tenants > ALL TENANTS. Choose a QOS policy (optional). choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts.Working with Contracts 225 Remove Contracts 1 On the menu bar. 4 In the Work pane. Enter a Contract_Name. choose Actions > Add Provided Contract or Actions > Add Consumed Contract. choose Tenants > ALL TENANTS. choose Actions > Delete. Choose a Label (optional). 5 In the Add Contract dialog box. 2 In the Work pane. Note: Choose the action depending on how the contract is to be deployed. 6 Click Submit.

b. choose Tenants > ALL TENANTS. Verify Contract on EPG Provider REST :: /api/node/class/fvRsProv. choose the Tenant_Name. 4 In the Work pane. 2 In the Work pane. 5 Click Update. click + next to either Add Provided Contract or Add Consumed Contract.xml CLI :: moquery -c fvRsCons Apply/Remove External Network Contracts Apply a Contract to an External Network 1 On the menu bar. . a. Note: Make a selection depending on how the contract is to be deployed. choose Actions > Delete. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name . Choose a Contract_Name. c. Choose a Match Criteria.226 Working with Contracts 2 In the Work pane. 4 In the Work pane.> Application EPGs > EPG_Name > Contracts > Contract_Name. choose the Tenant_Name. Choose a QOS Type. 3 In the Navigation pane choose Tenant_Name > Networking > External Routed Networks > Routed Outside_Name > Networks > External_Network_Instance_Profile.xml CLI :: moquery -c fvRsProv Consumer ​ REST :: /api/node/class/fvRsCons.

choose Tenants > ALL TENANTS. rather than to each EPG. they can apply the con​ tracts to this one vzAny group under the pri​ vate net​ work. 4 In the Work pane.xml CLI :: moquery -c fvRsCons Apply/Remove Private Network Contracts In order to apply con​ tracts to all end​ point groups within a pri​ vate net​ work. It eases con​ tract man​ age​ ment by al​ low​ ing the con​ tract con​ fig​ ur​ a​ tion for all end​ point groups within a pri​ vate net​ work from a sin​ gle lo​ ca​ tion as well as op​ ti​ miz​ ing hard​ ware re​ source con​ sump​ tion. 3 In the Navigation pane choose Tenant_Name > Networking > External Routed Networks > Routed Outside_Name > Networks > External_Network_Instance_Profile.Working with Contracts 227 Remove a Contract from an External Network 1 On the menu bar. For in​ stance. con​ tracts can be ap​ plied di​ rectly to the pri​ vate net​ work. . Apply a Contract to a Private Network (vzAny) 1 On the menu bar. choose the Contract_Name and click x. This con​ cept is also re​ ferred as "vzAny" end​ point group​ .xml CLI :: moquery -c fvRsProv Consumer REST :: /api/node/class/fvRsCons. Verify External Network Contracts Provider REST :: /api/node/class/fvRsProv. 2 In the Work pane. if an ACI Ad​ min​ is​ tra​ tion has 100 EPGs that are all part of the same pri​ vate net​ work. 2 In the Work pane. choose the Tenant_Name. choose Tenants > ALL TENANTS. choose the Tenant_Name.

click + next to either Add Provided Contract or Add Consumed Contract. 2 In the Work pane. choose the Contract_Name and click x. Note: Make a selection depending on how the contract is to be deployed.xml CLI :: moquery -c vzBrCP . Enter a Contract_Name. Choose a Match Criteria. 4 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network_Name > EPG Collection for Context. Verify Private Network Contracts REST :: /api/node/class/vzBrCP. b. Remove a Contract From a Private Network (​ vzAny) 1 On the menu bar. a.228 Working with Contracts 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network_Name > EPG Collection for Context. Choose a QOS Type. c. Click Update. choose the Tenant_Name. 4 In the Work pane. choose Tenants > ALL TENANTS.

IP Pro​ to​ col . and so on.The IP pro​ to​ col for a fil​ ter entry. Fil​ ters are Layer 2 to Layer 4 fields. Ac​ cord​ ing to its re​ lated con​ tract. Filter Entry Configuration Parameters When con​ fig​ ur​ ing a Fil​ ter. Arp Flag . the fol​ low​ ing op​ tions can be de​ fined: Name . Layer 4 ports. See the Use Cases below for an ex​ am​ ple. The Ether​ Types are: • ARP • FCOE • IP • MAC Security • MPLS Unicast • Trill • Unspecified The de​ fault is ARP.Working with Contracts 229 Filters A fil​ ter pol​ icy is a group of re​ solv​ able fil​ ter en​ tries. Con​ tract sub​ jects con​ tain as​ so​ ci​ a​ tions to the fil​ ters (and their di​ rec​ tions) that are ap​ plied be​ tween EPGs that pro​ duce and con​ sume the con​ tract. TCP/IP header fields such as Layer 3 pro​ to​ col type.The Ad​ dress Res​ ol​ u​ tion Pro​ to​ col flag for a fil​ ter entry.The name of a fil​ ter entry. The fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties.The Ether​ Type of the fil​ ter entry. . Each fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. Ether​ Type . The fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. an EPG provider dic​ tates the pro​ to​ cols and ports in both the in and out di​ rec​ tions.

The port can be for the fol​ low​ ing server types: • Unspecified • ftpData • smtp • DNS • HTTP • POP3 • HTTPS • RTSP ti​ na​ tion port range. The start of the port range is de​ ter​ mined by the server type.The start of the source port range. The range is 0 to 0xffff. The range is 0 to 0xffff. The end of the port Des​ ti​ na​ tion Port: From . The port can be for the fol​ low​ ing server types: Source Port: From . The end of the port range is de​ ter​ mined by the server type.The end of the source port range. The range is 0 to 0xffff.230 Working with Contracts Allow Frag​ ment . The range is 0 to 0xffff. The port state set​ tings are: • Unspecified • ftpData . The port can be for the fol​ low​ ing server types: • Unspecified • ftpData • SMTP • DNS • HTTP • POP3 • HTTPS • RTSP Source Port: To .The start of the source port range. The start of the port range is de​ ter​ mined by the server type.The end of the des​ range is de​ ter​ mined by the server type.

4 In the Work pane. 2 In the Work pane. Create Filters 1 On the menu bar. choose Actions > Create Filter.The TCP ses​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. Click + to add a Filter Entry. b. Enter an Filter Name. The range is 0 to 0xffff. The port range is de​ ter​ mined by the server type. The des​ ti​ na​ tion port can be set for the fol​ low​ ing server types: • unspecified • ftpData • smtp • dns • http • pop3 • https • rtsp The de​ fault is un​ spec​ i​ fied. 5 In the Create Filter dialog box. sion rules for a fil​ ter entry.The start of the des​ ti​ na​ tion port range. choose Tenants > ALL TENANTS. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters.Working with Contracts 231 • SMTP • DNS • HTTP • POP3 • HTTPS • RTSP Des​ ti​ na​ tion Port: To . choose the Tenant_Name. . The fil​ ter entry is a com​ bi​ TCP Ses​ sion Rules . perform the following actions: a.

7 Click Submit. 5.232 Working with Contracts i. 4. c. In the drop-down list select an Ethertype. 8. 4 In the Work pane. click the Allow Fragment check box (optional). In the drop-down list select an IP Protocol (optional). 6 Click Update. 2. In the drop-down list select the TCP Session Rules (optional). e. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters > Filter_Name. In the drop-down list select the Destination Port To (optional). b. 2 In the Work pane. f. i. In the drop-down list select an IP Protocol (optional). In the drop-down list select the Destination Port To (optional). choose the Tenant_Name. double click the filter entry. Modify Filters 1 On the menu bar. In the drop-down list select the Source Port To (optional). In the drop-down list select an Ethertype (optional). In the drop-down list select the Source Port From (optional). In the drop-down list select the TCP Session Rules (optional). Enter a Filter Entry Name. h. If required. In the Filter Entry dialog box. . In the drop-down list select the Destination Port From (optional). 9. choose Tenants > ALL TENANTS. 6 Click Submit. 5 Click Update. 10. a. g. In the drop-down list select an ARP Flag (optional). 3. In the drop-down list select the Source Port From (optional). d. In the drop-down list select the Destination Port From (optional). perform the following actions: 1. check the Allow Fragment check box (optional). 7. In the drop-down list select an ARP Flag (optional). 6. If required. In the drop-down list select the Source Port To (optional).

2 In the Work pane. Verify Filters REST :: /api/node/class/vzFilter.Working with Contracts 233 Remove Filters 1 On the menu bar. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters > Filter_Name. 4 In the Work pane.xml CLI :: moquery -c vzFilter . choose Tenants > ALL TENANTS. choose Actions > Delete.

.

The name of the con​ tract or con​ tract ob​ ject. Create. Taboos can be used to drop traf​ fic match​ ing a pat​ tern (any EPG. a spe​ cific EPG. Modify. La​ bels en​ able clas​ si​ fi​ ca​ tion of the ob​ jects which can and can​ not com​ mu​ ni​ cate with one an​ other (op​ tional). 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts. Taboo con​ tracts are not rec​ om​ mended as part of the ACI best prac​ tices but they can be used to tran​ si​ tion from tra​ di​ tional net​ work​ ing to ACI. choose Tenants > ALL TENANTS. To im​ it​ ate the tra​ di​ tional net​ work​ ing con​ cepts. Sub​ jects . Di​ rec​ tive .Working with Contracts 235 Taboo Contracts There may be times when the ACI ad​ min​ is​ tra​ tor might need to deny traf​ fic that is al​ lowed by an​ other con​ tract. .The net​ work do​ main name label. Taboo rules are ap​ plied in the hard​ ware be​ fore the rules of reg​ ul​ ar con​ tracts are ap​ plied. or Delete Taboo Contracts Create Taboo Contracts 1 On the menu bar. with taboo con​ tracts con​ fig​ ured to re​ strict cer​ tain types of traf​ fic. Taboos are a spe​ cial type of con​ tract that an ACI ad​ min​ is​ tra​ tor can use to deny spe​ cific traf​ fic that would oth​ er​ wise be al​ lowed by an​ other con​ tract.The fil​ ter di​ rec​ tives as​ signed to the taboo con​ tract. choose the Tenant_Name. and so forth). an "al​ low-all-traf​ fic" con​ tract can be ap​ plied. Taboo Contract Configuration Parameters When con​ fig​ ur​ ing Taboo Con​ tracts you can de​ fine the fol​ low​ ing op​ tions: Name . match​ ing a fil​ ter.

Enter a Taboo Contract Subject Name. b. Enter a Filter Name. i. 5 Click Submit. Choose Directives. 4 In the Work pane. a. 1.236 Working with Contracts 4 5 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts > Taboo_Contract_Name. 7 Click OK. Click + in the Filter Chain field. b. choose the Tenant_Name. choose Tenants > ALL TENANTS. Click + to next to the Subject field. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts > Taboo_Contract_Name. perform the following actions: i. ii. In the Create Taboo Contract dialog box. 8 Click Submit. . Enter a Taboo Contract Name. choose policy. 6 Click Update. choose the Tenant_Name. Click + to next to the Subject field to add a Taboo Subject. Choose Directives. ii. perform the following actions: a. Enter a Filter Name. In the Create Taboo Contract Subject dialog box. Modify Taboo Contracts 1 On the menu bar. 2. Delete Taboo Contracts 1 On the menu bar. choose Tenants > ALL TENANTS. choose Action > Create Taboo Contract. 2 In the Work pane. 2 In the Work pane.

choose the Taboo Contract_Name > Actions > Delete. choose Action > Delete. 2 In the Work pane. 4 In the Work pane. 5 In the Add Taboo Contract dialog box. Verify Taboo Contracts REST :: /api/node/class/vzTaboo. choose Actions > Add Taboo Contract. choose the Tenant_Name. 6 Click Submit. Remove Taboo Contract from EPG 1 On the menu bar. Choose the Taboo Contract. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts. choose Tenants > ALL TENANTS.Working with Contracts 237 4 In the Work pane.xml CLI :: moquery -c vzTaboo Apply/Remove Taboo Contracts Apply Taboo Contract to an EPG 1 On the menu bar. 2 In the Work pane. choose Tenants > ALL TENANTS. a. choose the Tenant_Name. 4 In the Work pane. . 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts.

238 Working with Contracts Verify Taboo Contracts Applied to EPG Provider REST :: /api/node/class/fvRsProv.xml CLI :: moquery -c fvRsCons .xml CLI :: moquery -c fvRsProv C ​onsumer REST :: /api/node/class/fvRsCons.

the source EPG will be of type provider. Enter an Export Contract Name. Choose the Global Contract. Sim​ il​ ar to tra​ di​ tional con​ tracts. in the tar​ get ten​ ant. b. The con​ tract in essence is ex​ ported in the source ten​ ant and im​ ported into the tar​ get ten​ ant. the con​ tract is im​ ported as type con​ tract in​ ter​ face. choose the Tenant_Name. Name . Some use case ex​ am​ ples show the com​ plete process in the next chap​ ter.Name of a ser​ vice con​ tract to be shared be​ tween two or more par​ tic​ ip ​at​ ing peer en​ ti​ ties. perform the following actions: a. Ten​ ant . Create/Modify/Remove Export Contracts Export Contract 1 On the menu bar. 5 In the Export Contract dialog box. 2 In the Work pane. How​ ever. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts. choose Tenants > ALL TENANTS.The name of the con​ Global Con​ tract . In​ ter​ face con​ tracts are a spe​ cial type of con​ tract that an ACI ad​ min​ is​ tra​ tor can use to allow spe​ cific traf​ fic through the use of a con​ tract ex​ port. choose Actions > Export Contract. Configuration Parameters When Im​ port​ ing a Con​ tract.The Ten​ ant name of the tar​ geted Ex​ port con​ tract.Working with Contracts 239 Inter-Tenant Contracts There may be times when the ACI ad​ min​ is​ tra​ tor might need to allow traf​ fic be​ tween two ten​ ants. . 4 In the Work pane. the fol​ low​ ing op​ tions can be de​ fined: tract in​ ter​ face.

choose Actions > Delete. 6 Click Finish.xml CLI :: moquery -c vzCPif . choose policy. Enter an Export Contract Name. 5 Click Finish. b. Enter the Tenant Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. c. choose Tenants > ALL TENANTS. 4 In the Work pane. a. 2 In the Work pane. 4 In the Work pane.240 Working with Contracts c. Modify Exported Contracts 1 On the menu bar. Enter the Tenant Name. 2 In the Work pane. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Imported Contracts > Contact_Name. Verify Exported Contracts REST :: /api/node/class/vzCPif. choose Tenants > ALL TENANTS. Choose the Global Contract. choose the Tenant_Name. Remove Exported Contracts 1 On the menu bar.

an end​ point group starts con​ sum​ ing all the sub​ jects rep​ re​ sented by the in​ ter​ face. This is ac​ com​ plished by uti​ liz​ ing con​ tact in​ ter​ faces. In the use case below. These ser​ vices will be used across most of their ten​ ants and so ACME Inc.. it will need to be ex​ ported from the source ten​ ant to the ap​ pro​ pri​ ate des​ ti​ na​ tion ten​ ant.Working with Contracts 241 Contracts Use Cases These use cases all as​ sume the ob​ jec​ tive is for a host in EPG-1 to talk to a host in EPG2. Inter-Tenant Contracts ACME Inc. Note: A con​ tract con​ sump​ tion in​ ter​ face rep​ re​ sents one or more sub​ jects de​ fined under the con​ tract. There are four com​ mon sce​ nar​ ios: 1 Inter-Tenant Contracts. Com​ mu​ ni​ ca​ tion be​ tween EPGs that be​ long to dif​ fer​ ent ten​ ants is only al​ lowed when they share the same con​ tract. and whether the sys​ tem is more fo​ cused on ob​ ject reuse or ten​ ant au​ ton​ omy. as with most com​ pa​ nies. To use the same con​ tract. Re​ view the con​ tracts sec​ tion on Con​ tract Scop​ ing for a more de​ tailed dis​ cus​ sion. 4 Single Contract Unidirectional with multiple Filters. That con​ tract will ap​ pear under the Im​ ported Con​ tract sec​ tion in the Se​ cu​ rity Poli​ cies of the des​ ti​ na​ tion ten​ ant. In ten​ ant Cisco-1 . EPG-1 in ten​ ant Cisco-1 re​ quires com​ mu​ ni​ ca​ tion with EPG-2 in ten​ ant Cisco-2. makes use of shared ser​ vices such as DNS for name res​ o​ lu​ tion and Ac​ tive Di​ rec​ tory for user man​ age​ ment. 2 Inter-Private Network Contracts. By as​ so​ ci​ at​ ing to an in​ ter​ face. 5 Multiple Contracts Unidirectional with single Filter. 3 Single Contract Bidirectional forwarding with reverse filter. tract In​ ter​ face will be used to as​ so​ ci​ ate an EPG from the des​ ti​ na​ tion A Con​ sumed Con​ ten​ ant with the im​ ported con​ tract. achiev​ ing bidi​ rec​ tional traf​ fic. must allow this traf​ fic across the whole fab​ ric. How these sce​ nar​ ios are im​ ple​ mented will de​ pend on the op​ er​ at​ ional model cho​ sen.

In ten​ ant Cisco-1 the user will ex​ port the in​ tended con​ tract and se​ lect provider to pro​ vide the con​ trast to EPG-2.subnet scope shared.242 Working with Contracts the user will ex​ port the in​ tended con​ tract in​ ter​ faces. 4 Create the host subnet (default Gateway IP) under the bridge domain .subnet scope shared.subnet scope private/public. The user will then con​ firm the im​ ported con​ tract in ten​ ant Cisco-2 and se​ lect the con​ tract as con​ sumed. .contract type consumed.subnet scope private/public. Exporting Contracts Between Tenants Tenant Cisco-1/EPG-1 1 Create an Export Contract under security policies. Tenant Cisco-2/EPG-2 1 Confirm the exported contract is listed under Imported Contracts. 2 Create the host subnet (default Gateway IP) under EPG2 . To ad​ ver​ tise the routes from the source VRF to the in​ tended VRF. 3 Add the Contract under EPG1 . 4 Create the host subnet under the bridge domain . 3 Add the Interface Contract under EPG2 . 2 Create the host subnet (default Gateway IP) under EPG1 . the user must cre​ ate the sub​ net within the EPG.contract type provider.

the route will be leaked to the VRF noted within the Ten​ ant scoped con​ tract.subnet scope shared. Single Contract Bidirectional Reverse Filter This use case is use​ ful when im​ ple​ ment​ ing a con​ tract with the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions and with the op​ tion to apply the re​ verse fil​ ter. By cre​ at​ ing the sub​ net under the EPG and se​ lect​ ing shared. 2 Add the Contract under EPG1 . Tenant Cisco-1/EPG-1 1 Create the host subnet (default Gateway IP) under EPG1 .contract scope Tenant.Working with Contracts 243 Inter-Private Network Contracts Communication In the use case below. Tenant Cisco-1/EPG-2 1 Create the host subnet (default Gateway IP) under EPG2 .subnet scope shared.contract type provider. 2 Add the Contract under EPG2 . EPG-1 in VRF Cisco-1 re​ quires com​ mu​ ni​ ca​ tion with EPG-2 in VRF Cisco-2. This is ac​ com​ plished by uti​ liz​ ing the sub​ net field within the EPG. This is . Exporting Contracts Between Private Networks 1 Create the contract under Security Policies .contract type provider.

EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of www and EPG-2 is con​ sum​ ing the con​ tract. Single Filter with Reverse Filtering Ports Enabled Single Contract Unidirectional with Multiple Filters . i. In this ex​ am​ ple. EPG-1 is pro​ vid​ ing a ser​ vice to EPG-2. Default Bi-directional Contract with Reverse Filter Re​ sult: A sin​ gle con​ tract with (1) Sub​ ject and (1) Fil​ ter with a sin​ gle provider and a sin​ gle con​ sumer.e. In the use case below.244 Working with Contracts the most com​ mon of the use cases and al​ lows for a sin​ gle sub​ ject/fil​ ter to be im​ ple​ mented with a sin​ gle Provider/Con​ sumer re​ la​ tion​ ship. This al​ lows the Web Client in EPG-2 to ac​ cess the Web Server in EPG-1. www. Sample Contract using Single Bi-directional Subject.

EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of icmp and EPG-2 is con​ sum​ ing the con​ tract.Working with Contracts 245 Single Contract Unidirectional with Multiple Filters This use case in​ volves im​ ple​ ment​ ing a con​ tract with​ out the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions. Single Unidirectional Subject. In the use case below. This al​ lows the Host in EPG-1 to ac​ cess the Host in EPG-2 via icmp. Multiple Filters Re​ sult: A sin​ gle con​ tract with (1) Sub​ ject (2) Fil​ ters and a sin​ gle provider and a sin​ gle con​ sumer. When se​ lect​ ing this op​ tion the user no longer has the op​ tion to se​ lect the re​ verse fil​ ter op​ tion. one in each di​ rec​ tion. icmp. Single Contract. When uti​ liz​ ing a sin​ gle sub​ ject with​ out the use of “Apply Both Di​ rec​ tions. Single Unidirectional Subject.” the user must then con​ fig​ ure two fil​ ters. Sample Single Contract. In this ex​ am​ ple. Multiple Filters .

Single Filters Re​ sult: Two con​ tracts with (1) Sub​ ject (1) Fil​ ters. This al​ lows the Web Client in EPG-2 to ac​ cess the Web Server in EPG-1. but is also the most com​ pre​ hen​ sive. Unidirectional Subjects. EPG-1 is pro​ vid​ ing a ser​ vice to EPG-2. EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of www and EPG-2 is con​ sum​ ing the con​ tract. and with​ out the op​ tion to apply the re​ verse fil​ ter. This al​ lows the end-user the most gran​ u​ lar​ ity when de​ ploy​ ing con​ tracts. Each con​ tract will have a sin​ gle provider and a sin​ gle con​ sumer ref​ er​ enc​ ing the same con​ tract. Multiple Contracts. In the use case below.246 Working with Contracts Multiple Contracts Uni-Directional Single Filter This use case is use​ ful when im​ ple​ ment​ ing a con​ tract with the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions. . That is. The dif​ fer​ ence here is that the con​ tract is ex​ plic​ itly ap​ plied in BOTH di​ rec​ tions.

Working with Contracts 247 Sample Contract with Single Bi-Directional Subjects and single Filter .

.

249 Layer 4 to Layer 7 Services .

.

Layer 4 to Layer 7 Services 251 Section Content • Understanding Layer 4 to Layer 7 Integration Service Insertion Design Principles Applying Service Graphs to EPG Communications Rendering Service Graphs Integration Support Function • Services Deployment Guide Reference Import Device Package Create a Device Modify a Device Create Layer 4 to Layer 7 Service Graph Template Apply a Service Graph Template to EPGs • Service Graph Monitoring Monitoring a Service Graph Instance Monitoring and Resolving Service Graph Faults Monitoring a Virtual Device • ASAv Sample Configuration Verify ASAv VM configuration Remove GW IP from Relevant Fabric Bridge Domains Create a Layer 4 to Layer 7 Device Create a Layer 4 to Layer 7 Service Graph Template Apply the Service Graph Template Associate the Web and App Bridge domains to the Interfaces Verifying service node configuration Display the Access List configuration pushed-down from APIC Display the Interface configuration pushed-down from APIC Display the current Interface names .

.

Layer 4 to Layer 7 Services 253 Understanding Layer 4 to Layer 7 Integration When ACME Inc. we will dis​ cuss the high level overview of how the process works and how ACME will uti​ lize Ser​ vice Graphs for ef​ fi​ cient man​ age​ ment of Layer 4 to Layer 7 ser​ vices. be​ tween phys​ ic ​al and vir​ tual hosts within the fab​ ric and egress back out of the fab​ ric. Through the rest of this chap​ ter. Some pos​ si​ ble Layer 4 to Layer 7 ser​ vices in​ clude: • Firewalls • Load balancers • Traffic inspection appliances • SSL offload functions • Application flow acceleration functions In​ te​ grat​ ing ser​ vices with Cisco ACI Ser​ vice Graphs will pro​ vide ACME with the fol​ low​ ing ben​ e​ fits: • Policy based configuration management . IDS/IPS. ser​ vice in​ ser​ tion would re​ quire some in​ line de​ vice place​ ment or redi​ rec​ tion in order to get traf​ fic to the ser​ vice de​ vices. or other types of higher-layer ser​ vice de​ vice. such as fire​ walls. One of the key tech​ nol​ ogy in​ no​ va​ tions in Cisco's Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) is pol​ icy-based man​ age​ ment of ser​ vice in​ ser​ tion through ap​ pli​ ca​ tion of a Ser​ vice Graph. Fire​ walls are typ​ ic ​ally con​ fig​ ured per de​ vice by build​ ing up blocks of sta​ tic con​ fig​ ur​ a​ tion. fire​ walls might be placed di​ rectly in​ line as a "bump in the line" or might be an ad​ junct ser​ vice de​ vice near a gate​ way. As an ex​ am​ ple. but can be​ come in​ flex​ ib ​le and frag​ ile. Use​ ful in​ fra​ struc​ ture im​ ple​ men​ ta​ tions also uti​ lize this fast fab​ ric in a smart way to also in​ te​ grate Layer 4 to Layer 7 ser​ vices. they knew they would need to in​ cor​ po​ rate some ser​ vices. de​ signed their new ap​ pli​ ca​ tion. These sta​ tic con​ fig​ ur​ a​ tion blocks build up over time to cre​ ate a sit​ ua ​t​ ion where the con​ fig​ ur​ a​ tion works. In tra​ di​ tional in​ fra​ struc​ ture. The main pur​ pose of data cen​ ter fab​ ric equip​ ment is fast and ef​ fi​ cient for​ ward​ ing of traf​ fic from ingress to the fab​ ric. which can cause change man​ age​ ment to be​ come chal​ leng​ ing. load bal​ ancers.

254 Layer 4 to Layer 7 Services • Flexible configuration state abstraction through the ACI object model • Integrated configuration management using the APIC GUI. phys​ ic ​al ap​ pli​ ances can also be run in mul​ ti​ ple con​ text mode. and can be ex​ tended to in​ clude ser​ vice in​ ser​ tion via at​ tach​ ment of a Ser​ vice Graph to a con​ tract. With vir​ tual ser​ vices ap​ pli​ ances. all based on a consistent ACI object model • Complex topology modeling with logical flow stitching allowing abstracted links • Policy-based provisioning allowing rapid complex topology deployment • Configuration synchronization allowing dynamic workload provisioning and de- between multiple service devices provisioning without manual intervention • Application centric template-based configuration management and object • Infrastructure multi-tenancy within the fabric and the service devices reuse to shorten infrastructure implementation timelines Service Insertion Design Principles With the spine-leaf ar​ chi​ tec​ ture and holis​ tic fab​ ric man​ age​ ment as​ pects of ACI. . This con​ tract may take the form of a spe​ cific con​ sumer/provider re​ la​ tion​ ship de​ fined by spec​ if​ ied pro​ to​ col and port. Ser​ vices ap​ pli​ ances can be phys​ ic ​al or vir​ tual and can be con​ nected to any leaf under man​ age​ ment of the ACI fab​ ric. Where ap​ plic​ ab ​le. Applying Service Graphs to EPG Communications To allow com​ mu​ ni​ ca​ tions be​ tween EPGs within an ACI fab​ ric. the ap​ pli​ ances do not need to be placed at any spe​ cific lo​ ca​ tion within the net​ work fab​ ric. This con​ tract es​ sen​ tially con​ trols com​ mu​ ni​ ca​ tions flow be​ tween EPGs. and thus. traf​ fic flow to/from ser​ vice ap​ pli​ ances is man​ aged very ef​ fi​ ciently. cur​ rently only VMware Hy​ per​ vi​ sors and VLAN trans​ port modes are sup​ ported. There could also be an "allow all" con​ tract that al​ lows com​ pletely open com​ mu​ ni​ ca​ tions. REST API or Python scripts. a con​ tract must be put in to place. al​ low​ ing multi-ten​ ant map​ ping of fab​ ric for​ ward​ ing and ten​ ant-spe​ cific ser​ vice con​ fig​ ur​ a​ tions. The Ser​ vice Graph then ties the con​ tract to the re​ solved ser​ vice de​ vice with the pol​ icy-based con​ fig​ ur​ a​ tion in place.

com​ mu​ ni​ ca​ tions with the ser​ vice de​ vices is sup​ ported by im​ port​ ing de​ vice pack​ ages. ports. cre​ ation of the VLAN on these de​ vices to cre​ ate the path for the func​ tions. and per​ for​ mance of all the work nec​ es​ sary to make sure that the path be​ tween EPGs is the path de​ fined in the ser​ vice graph. This pol​ icy can be copied. how it con​ nects to the fab​ ric. these ob​ jects ex​ press the in​ tent of use for that ob​ ject in re​ la​ tion to the ap​ pli​ ca​ tion. The ren​ der​ ing in​ volves al​ lo​ ca​ tion of the nec​ es​ sary bridge do​ mains. As it re​ lates to the Layer 4 to Layer 7 ser​ vices. then link two groups and sub​ ject their re​ la​ tion​ ship to that pol​ icy. and other val​ ues. This ab​ stracted process of con​ fig​ ur​ a​ tion man​ age​ ment works like a pol​ icy tem​ plate where you can de​ fine the ex​ pected be​ hav​ ior. Though the ini​ tial de​ f​ in ​i​ t​ ion of these tem​ plates can be po​ ten​ tially cum​ ber​ some. When an ap​ pli​ ca​ tion pro​ file is de​ ployed and end​ points are at​ tached to the leaf switches. such as Python or a RESTful API post through POSTman These pol​ icy ob​ jects can be cre​ ated.Layer 4 to Layer 7 Services 255 Rendering Service Graphs The ACI al​ lows users to de​ fine a pol​ icy using the fol​ low​ ing ways: • APIC GUI • API with a programmatic tool. and reused. As is the case with many cus​ tomers. When the de​ vice pack​ age is im​ ported. These de​ vice pack​ ages carry the de​ vice de​ scrip​ tion. Integration Support Function In the ACI model. the ser​ vice graph ob​ jects are then trans​ lated into spe​ cific de​ vice con​ fig​ ur​ a​ tions that gets pushed the ser​ vice nodes through a process called ren​ der​ ing. reused and repack​ aged as nec​ es​ sary. the ACI fab​ ric has full un​ der​ stand​ ing of what a de​ vice can do. The APIC also sets up the net​ work for​ ward​ ing path to make sure the cor​ rect for​ ward​ ing ac​ tion is taken to get traf​ fic flow to the ser​ vice nodes for treat​ ment. ob​ ject-groups. ma​ nip​ ul​ ated. and have con​ fig​ ur​ a​ tion script con​ tent. con​ fig​ ur​ a​ tion of IP ad​ dresses on the fire​ wall and load bal​ ancer in​ ter​ faces. ex​ posed func​ tions. . ACME has a few cookie cut​ ter tem​ plates for fire​ wall and load-bal​ anc​ ing ser​ vices. sub​ se​ quently reusing the tem​ plates is very straight​ for​ ward sim​ ply by re​ plac​ ing IP ad​ dresses.

​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ unified-fabric/​ aci_​ ecosystem.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centricinfrastructure/​ solution-overview-c22-732445.​ html De​ vice pack​ ages from mul​ ti​ ple ven​ dors that lever​ age the rich open API that ACI pro​ vides are avail​ able from ven​ dors such as Cit​ rix and F5 at the time of writ​ ing of this book. This de​ vice pack​ age is a ven​ dor-de​ vel​ oped pack​ age that is read​ ily avail​ able from the orig​ in ​al ven​ dor or from Cisco on the soft​ ware down​ load page. An up to date list​ ing of part​ ners that lever​ age the API are avail​ able at: http://​ www.​ html .​ cisco.256 Layer 4 to Layer 7 Services how to build path for​ ward​ ing to bring traf​ fic into and get traf​ fic back from the de​ vice. and how to trans​ late pol​ icy in​ tent to a de​ vice-spe​ cific con​ fig​ ur​ a​ tion. Some of the De​ vice Pack​ ages can be down​ loaded at: http://​ www.

see the Cisco APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide: http://​ www. The con​ fig​ u​ ra​ tion steps dif​ fer slightly for phys​ i​ cal de​ vices and vir​ tual de​ vices. . Create a Device Once the de​ vice pack​ age is im​ ported. but are very sim​ i​ lar. This can be done with a phys​ i​ cal or VM de​ vice. ACME will im​ port the ven​ dor/model-spe​ cific de​ vice pack​ ages.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy. the de​ vices will be added through a process of cre​ at​ ing a log​ i​ cal de​ vice clus​ ter and cre​ at​ ing a re​ la​ tion​ ship be​ tween this log​ i​ cal de​ vice and the phys​ i​ cal ap​ pli​ ance. L4-L7 Workflow For in​ for​ ma​ tion about de​ ploy​ ing Layer 4 to Layer 7 ser​ vices. as shown in this sec​ tion of the De​ ploy​ ment Guide. The key sec​ tions of the De​ ploy​ ment Guide are listed below: Import Device Package To begin the process of Ser​ vice Node in​ te​ gra​ tion.Layer 4 to Layer 7 Services 257 Services Deployment Guide Reference The di​ a​ gram below shows a high level overview of the Layer 4 to Layer 7 ser​ vices work​ flow when at​ tempt​ ing to in​ te​ grate a de​ vice.​ cisco.​ html.

Create Layer 4 to Layer 7 Service Graph Template This sec​ tion of the De​ ploy​ ment Guide ex​ plains how to cre​ ate a ser​ vice graph. . Apply a Service Graph Template to EPGs Once the ap​ pli​ ca​ tion End​ point Groups (EPGs) have been cre​ ated. the process to apply a ser​ vice graph tem​ plate to EPGs can be found in this sec​ tion of the De​ ploy​ ment Guide.258 Layer 4 to Layer 7 Services Modify a Device You can mod​ ify a de​ vice's con​ fig​ ur​ a​ tion through the GUI as de​ scribed in this sec​ tion of the De​ ploy​ ment Guide.

has de​ ployed Ser​ vice Graphs for ap​ pli​ ca​ tion ser​ vice in​ ser​ tion. For fur​ ther de​ tails of the pos​ si​ ble states and other rel​ e​ vant states.​ cisco. re​ sources al​ lo​ cated to a func​ tion and pa​ ra​ me​ ters spec​ if​ ied for a func​ tion. To mon​ it​ or a ser​ vice graph in​ stance: 1 On the menu bar. see the Cisco APIC vices De​ ploy​ ment Guide at: http://​ www. 2 In the Work pane. To sup​ port these ef​ forts. the associated contracts. in​ clud​ ing: • Monitoring a Service Graph Instance • Monitoring and Resolving Service Graph Faults • Monitoring a Virtual Device Monitoring a Service Graph Instance Once a ser​ vice graph is con​ fig​ ured and as​ so​ ci​ ated with a con​ tract that is at​ tached to an EPG. there are some pri​ mary mon​ it​ or​ ing as​ pects that should be con​ sid​ ered: State of the Ser​ vice Graph. there are a few tech​ niques that can be em​ ployed. choose Tenant_Name > L4-L7 Services > Deployed Graph Instances. the re​ quire​ ment of ob​ serv​ ing the Ser​ vice Graph be​ comes an op​ er​ at​ ional im​ per​ at​ ive.Layer 4 to Layer 7 Services 259 Service Graph Monitoring Once ACME Inc.​ html#​ task_​ F2BFF7545D9142EFB208C10F5DFBB1B4 . 3 In the Navigation pane. choose the tenant. The Work pane displays information about the deployed graph instances.​ com/​ c/​ en/​ us/​ td/​ Layer 4 to Layer 7 Ser​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy/​ b_​ L4L7_​ Deploy_​ chapter_​ 01010. and the current state of the graph policy. Func​ tions of a Graph in​ stance. A state of "Applied" means the graph has been applied and is active in the fabric and the service device. choose Tenants > ALL TENANTS. including a list of the deployed service graphs.

The Work pane displays information about the virtual devices. To un​ der​ stand the faults listed and pos​ si​ ble res​ o​ lu​ tions. 3 In the Navigation pane.​ html#​ concept_​ 307C0CA3EB57469EAF7EF87AAE5A240F Monitoring a Virtual Device After you con​ fig​ ure a ser​ vice graph and at​ tach the graph to an end​ point group (EPG) and a con​ tract. choose Tenants > ALL TENANTS. 4 In the Work pane.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy/​ b_​ L4L7_​ Deploy_​ chapter_​ 01010. you can mon​ it​ or the vir​ tual de​ vices as​ so​ ci​ ated with the ser​ vice graphs of a ten​ ant. Mon​ it​ or​ ing the vir​ tual de​ vices tells you what de​ vice clus​ ters are as​ so​ ci​ ated. choose Tenant_Name > L4-L7 Services > Virtual Devices. the sta​ tis​ tics from the de​ vices. see Ta​ bles 1 and 2 in the Cisco APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide at: http://​ www.​ cisco. 2 In the Work pane. 3 In the Navigation pane. The Work pane displays the faults that are related to the active service graph. which VLANs are con​ fig​ ured for a de​ vice clus​ ter. the func​ tions in use and the func​ tion pa​ ra​ me​ ters passed to the de​ vices. choose Tenant_Name > L4-L7 Services > Deployed Graph Instances. choose Tenants > ALL TENANTS. 2 In the Work pane.260 Layer 4 to Layer 7 Services Monitoring and Resolving Service Graph Faults To mon​ it​ or a ser​ vice graph's faults: 1 On the menu bar. and the health of the de​ vices in a de​ vice clus​ ter. choose the tenant. To mon​ it​ or a vir​ tual de​ vice: 1 On the menu bar. such as faults and health scores. . choose the tenant. choose the Faults tab.

routed FW mode. 3 Issue the following command: # show ip int brief 4 Verify that the ASAv has a correct IPv4 address on the management 0/0 interface. This ex​ am​ ple de​ tails the process they fol​ lowed to con​ fig​ ure the ASAv fire​ wall vir​ tual ser​ vice ap​ pli​ ance as a sin​ gle node in routed mode. As a routed node.Layer 4 to Layer 7 Services 261 ASAv Sample Configuration One of the ser​ vice nodes that ACME Inc. Issue the following command: # ping {ASAv management 0/0 interface} . 5 Verify connectivity from the APIC to the management 0/0 interface of the ASAv. the ASAv will be​ come the de​ fault gate​ way for the hosts in the 2 EPGs that are con​ nected by the con​ tract that this ser​ vice graph gets as​ so​ ci​ ated to. SSH to the APIC cluster IP address b. The high level steps are: • Create Logical and Concrete device clusters • Define a firewall ruleset/graph • Deploy the graph between 2 EPGs Verify ASAv VM configuration The first set of steps per​ formed is to ver​ ify ASAv VM con​ fig​ ur​ a​ tion and up​ load the ASA de​ vice pack​ age (or ver​ ify if it has al​ ready been done) 1 Log in to the ASAv VM. a. will in​ te​ grate is an ASAv de​ ployed in sin​ glenode. 2 Enter enable mode.

10. 8 If an ASA device package is not listed. Click Submit. c. Create a Layer 4 to Layer 7 Device . To re​ move the rout​ ing de​ fault gate​ way func​ tion on the EPG1 and EPG2 bridge do​ mains: 1 On the menu bar. 6 In the APIC GUI. b. d.16. choose the tenant. Ad​ dress the con​ nec​ tiv​ ity prob​ lems be​ fore mov​ ing on.262 Layer 4 to Layer 7 Services c. choose Tenant > Tenant_Name. Follow the prompts to upload the device package. perform the following actions: a.050 ms If the re​ sponse is dif​ fer​ ent. click Flood. then there is likely some sort of con​ nec​ tiv​ ity issue. then perform the following actions: a. Re​ peat this process for the Bridge Do​ mains of the af​ fected EPGs. b. Choose Import Device Package. choose Tenants > ALL TENANTS. 4 In the Work pane. Right click on the Device Types. For the L3 Unknown Multicast Flooding radio buttons. 2 In the Work pane. the next step is to re​ move SVI/GW IP ad​ dresses from the fab​ ric Bridge Do​ mains so the Layer 3 routed fire​ wall can be​ come the de​ fault gate​ way. The return response should be similar to the following example: 64 bytes from 172. click Flood. c. on the menu bar.10: icmp_seq=1 ttl=64 time=0. Uncheck the Unicast Routing check box. For the L2 Unknown Unicast radio buttons. 3 In the Navigation pane. 7 In the Navigation Pane expand Tenant_Name > L4-L7 Services > Packages > L4-L7 Service Device Types. Remove GW IP from Relevant Fabric Bridge Domains Once the ASAv pack​ age and VM are ver​ if​ ied. choose Tenant_Name > Networking > Bridge Domains > BD-EPG1.

Click SUBMIT. i. You will configure IP addressing under Interface Related Configuration for both external and internal interfaces (externalIf and interalIf). perform the following actions: a. choose Create Function Profile Group. choose Tenants > ALL TENANTS. choose the tenant. Enter: ipv4 address in a. click Create a L4-L7 Function Profile. Click Update. 2 In the Work pane. b. choose the All Parameters tab. 5 In the Create a L4-L7 Function Profile dialog box. 3 In the Navigation Pane choose Tenant_Name > L4-L7 Services. ii. Enter a name that is relevant to the tenant and fuction.p format j. such as "TXX-ASAvFP". 2 In the Work pane.c. 6 Choose L4-7 Services > Function Profiles > ALL PARAMETERS. 7 Verify the external and internal interfaces IPv4 addresses. 4 In the Work pane. such as "TXX-FP-Group". In the Profile Group drop-down list. In the Create L4-L7 Services Function Profile Group dialog box.n. choose WebServiceProfileGroup or WebPolicyForRoutedMode. . Expand Interface Related Configuration for externalIf. d. In the Feature and Parameters section. In the Profile drop-down list.Layer 4 to Layer 7 Services 263 Create a Layer 4 to Layer 7 Device Per​ form the fol​ low​ ing steps to add log​ ic ​al and con​ crete de​ vice clus​ ters to a ten​ ant: 1 On the menu bar. Expand Interface Specific Configuration.4j as needed. The fol​ low​ ing steps will cre​ ate a Layer 4 to Layer 7 de​ vice: 1 On the menu bar.o. Repeat steps 4f . h.b. perform the following actions: i. k. Double-click IPv4 Address. c. f. e.d/m. Enter a meaningful name. choose Tenants > ALL TENANTS. g. choose the tenant.

4 In the Work pane. specify the following values: i. VM: Tenant ASAv Controller (in the dropdown box) iv. 5 In the Create L4-L7 Devices dialog box. enter interface values accordingly: 1. Name: GigabitEthernet0/0 vNIC: Network Adapter 2 Direction: provider 2. choose Tenants > ALL TENANTS. vii. choose the tenant. Click UPDATE after each entry. Enter a meaningful name: Txx-ASAv-Cluster b. choose Tenant_Name > L4-L7 Services. Device Package: CISCO-ASA-1. perform the following actions: a. Management Port: https iii. Function Type: Goto e. Click NEXT. APIC to Device: Out of Band g. Click FINISH. vi. Name: GigabitEthernet0/1 vNIC: Network Adapter 3 Direction: consumer v. click Create a L4-L7 virtual device. . 5 Select TXX-ASAv-Cluster_Device_1 to see a graphic view of the concrete device. Connectivity VMM Domain: Txx-vCenter f. Ver​ ify the Log​ ic ​al and Con​ crete De​ vice Clus​ ters have been con​ fig​ ured: 1 On the menu bar. click + twice.1 Model: ASAv c. Credentials: {uid/pwd} h. Mode: Single Node d. 4 Expand both TXX-ASAv-Cluster and TXX-ASAv-Cluster_Device_1 to view the logical and physical interfaces. Virtual Interfaces: Create two entries.264 Layer 4 to Layer 7 Services 3 In the Navigation pane. 2 In the Work pane. Under Device 1. Management IP address: ASAv IP address ii. 3 In the Navigation Pane choose Tenant_Name > L4-L7 Services > L4-L7 Devices.

In the Provider EPG / External Network drop-down list.Firewall. 3 In the Navigation pane. choose Single Node . Apply the Service Graph Template 1 On the menu bar. In the Type drop-down list. . choose TXX-ASAv-FP. which is the functional profile you created previously. c. 6 You can verify that the template was created successfully by expanding Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates > Txx-ASAvL3-Routed-Template > Function Node . In the Consumer EPG / External Network drop-down list. Create a Layer 4 to Layer 7 Service Graph Template 1 On the menu bar. choose the tenant. choose Actions > Create a L4-L7 Service Graph Template. choose EPG2. Click Submit. 2 In the Work pane. 5 In the Create L4-L7 Devices dialog box. choose CISCO-ASA-1. choose Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates. choose the tenant. choose Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates. choose EPG1. 2 In the Work pane. choose Tenants > ALL TENANTS.Firewall in Routed Mode. 3 In the Navigation pane. e. choose Tenants > ALL TENANTS. In the Profile drop-down list. b. perform the following actions: a.Layer 4 to Layer 7 Services 265 Note This com​ pletes the Log​ ic ​al and Con​ crete de​ vice cre​ ation. 5 In the Apply L4-L7 Service Graph Template to EPGs dialog box. 4 Right click Txx-ASAv-L3-Routed-Template and choose Apply L4-L7 Service Graph Template. In the Device Function drop-down list. perform the following actions: a. d. b. In the Name field. enter “TXX-ASAv-L3-Routed-Template”. 4 In the Work pane.1/Firewall.

g. For the Contract radio buttons. In the Service Graph Template drop-down list. d. choose Txx-ASAv-L3Routed-Template. the configuration will not push to the ASA. Click Finish. This inbound access list is not desirable for the lab traffic-flow. Expand Device Config > Interface Related Configuration internalIf > Interface Specific Configuration to verify the IP address assignment with the mask. n. 4 Assign the Bridge Domain to BD-EPG1. e. Click Next. Expand Device Config > Interface Related Configuration externalIf > Interface Related Configuration to verify the IP address assignment. h. f. You can see that the Consumer and Provider EPGs are associated with the EPG1 and EPG2 server EPGs. 2 Expand TXX-EPG1-to-EPG2-TXX-ASAv-L3-Router-Template-Firewall. This unassigns the inbound access list from the internal interface. In the Features and Parameters section. Double click Device Config > Interface Related Configuration internalIf > Access Group > Inbound Access List. Double click Device Config > Interface Related Configuration externalIf > Access Group > Inbound Access List. 3 Select External. q. . i. The Devices Selection Polices folder and Deployed Graph Instances folder are now populated. Click Update. l. m. choose the All Parameters tab. In the Contract Name field. choose access-list-inbound. j. o. Check the No Filter (Allow All Traffic) check box. enter “TXX-EPG1-to-EPG2”. In the Value drop-down list. 6 Choose TXX-EPG1-to-EPG2-TXX-ASAv-L3-Router-Template-TXXProduction. In the L4-L7 Devices drop-down list. p. If the mask is missing. Click Reset.266 Layer 4 to Layer 7 Services c. Associate the Web and App Bridge domains to the Interfaces 1 Expand Devices Selection Policies. k. choose Txx-ASAv-Cluster. click Create A New One.

10 Filters: common/default. 16 Click SUBMIT. 14 Filters: common/default. 11 CTX Terms: TXX-ASAv-L3Routed Template/T1/Out. . 6 Click SUBMIT.Layer 4 to Layer 7 Services 267 5 Repeat the process to assign the Bridge Domain to BD-EPG2. Issue the fol​ low​ ing com​ mand: # show run | grep access This will show the ac​ cess list con​ fig​ ur​ a​ tion and you can re​ late that to the con​ fig​ ur​ a​ tion that was done in the APIC. This can be ver​ if​ ied by going into the ASAv VM con​ sole and issue the fol​ low​ ing com​ mand: # show ip int brief Verifying service node configuration Once the De​ vice has been in​ te​ grated and the Ser​ vice Graph has been con​ fig​ ured and as​ so​ ci​ ated. 12 Click SUBMIT. 9 Expand Function Node – Firewall > choose external. 8 Expand TXX-ASAv-L3-Router-Template. 13 Repeat the process for internal. The ASAv will now have IP ad​ dresses asigned in the Ser​ vice Graph Pro​ file. 15 CTX Terms: TXX-ASAv-L3Routed Template/T1/In. Display the Access List configuration pushed-down from APIC SSH to the ASAv ser​ vice node for ac​ cess list val​ id ​a​ tion. 7 Expand Layer 4 to Layer 7 Service Graph Templates. the re​ sult​ ing con​ fig​ ur​ a​ tion pushed to the ser​ vice node can be ver​ if​ ied by sim​ ple show com​ mands on the real ser​ vice node de​ vice.

. Issue the fol​ low​ ing com​ mand: # show nameif This will show the in​ ter​ face names pushed from the APIC and will show the re​ lated in​ ter​ face names to the log​ ic ​al in​ ter​ face names that were con​ fig​ ured in the APIC above. Issue the fol​ low​ ing com​ mand: # show run interface This will show the in​ ter​ face con​ fig​ ur​ a​ tion where the IP ad​ dress con​ fig​ ur​ a​ tion was pushed from the APIC. Display the current Interface names SSH to the ASAv ser​ vice node for in​ ter​ face name con​ fig​ ur​ a​ tion val​ id ​a​ tion.268 Layer 4 to Layer 7 Services Display the Interface configuration pushed-down from APIC SSH to the ASAv ser​ vice node for in​ ter​ face con​ fig​ ur​ a​ tion val​ id ​a​ tion.

269 Health Scores .

.

Health Scores 271 Section Content • Understanding Health Scores • Understanding Faults • How Are Health Scores Calculated • Health Score Use Cases Using Health Scores for Proactive Monitoring Using Health Scores for Reactive Monitoring .

.

but with health scores. To man​ ua ​lly col​ lect and cor​ re​ late in​ for​ ma​ tion would have pre​ vi​ ously been a long and te​ dious task. To ad​ dress these chal​ lenges they can now uti​ lize the Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI). This vis​ i-​ bil​ ity has a num​ ber of prac​ ti​ cal use cases. .Health Scores 273 Understanding Health Scores ACME's Op​ er​ at​ ions team has been chal​ lenged on a reg​ ul​ ar basis to an​ swer basic ques​ tions re​ gard​ ing the cur​ rent sta​ tus. and how var​ io ​us faults im​ pact the cal​ cu​ la​ tion of the health score. and there​ fore the cur​ rent sta​ tus of all of the ob​ jects in​ clud​ ing links. ACI also pro​ vides the flex​ ib ​il​ ity to mon​ it​ or some as​ pects of how the health scores are cal​ cu​ lated. The ob​ ject model at the heart of ACI is in​ her​ ent to the in​ fra​ struc​ ture. de​ vices. com​ puted. and in this chap​ ter we will clas​ sify these use cases as re​ ac​ tive and proac​ tive. data through​ out the fab​ ric is col​ lected. The vis​ ib ​il​ ity pro​ vided by health scores give the op​ er​ at​ or a quick at-a-glance as​ sess​ ment of the cur​ rent sta​ tus of the en​ tire sys​ tem. per​ for​ mance. as well as the real time sta​ tus of their uti​ liza​ tion. can be rep​ re​ sented in a sin​ gle con​ sol​ id ​ated health score. and de​ scribe the re​ la​ tion​ ship be​ tween the var​ io ​us de​ vices and links in at​ tempt to pro​ vide a cor​ re​ la​ tion. Most ob​ jects in the model will have an as​ so​ ci​ ated health score. and cor​ re​ lated by the APIC in real time and then pre​ sented in an un​ der​ stand​ able for​ mat. their re​ la​ tion​ ships. which can be found from the Dash​ board or Pol​ icy tabs of the ob​ ject from the GUI. and avail​ abil​ ity of the sys​ tem they are re​ spon​ si​ ble for op​ er​ at​ ing. Ad​ di​ tion​ ally. which pro​ vides Health Scores that make in​ for​ ma​ tion on sta​ tus. Tra​ di​ tional net​ work mon​ it​ or​ ing and man​ age​ ment sys​ tems at​ tempt to pro​ vide a model of in​ fra​ struc​ ture that has been pro​ vi​ sioned. all health scores are in​ stan​ ti​ ated from the health​ Inst class and can be ex​ tracted through the API. this in​ for​ ma​ tion by it​ self is of lit​ tle to no value with​ out ad​ di​ tional data on its ef​ fect on the over​ all health of the net​ work. or any sub​ set of the sys​ tem. and avail​ abil​ ity read​ ily avail​ able. per​ for​ mance. It is worth not​ ing that while pro​ vid​ ing such an​ swers may be easy as it re​ lates to an in​ de​ pen​ dent de​ vice or link.

im​ me​ di​ ately pro​ vid​ ing feed​ back as to which ten​ ants. If so.274 Health Scores Tenant Health Score In a re​ ac​ tive ca​ pac​ ity. . ACI health scores pro​ vide a quick check as to whether an issue being re​ ported is con​ firmed in a degra​ da​ tion of the health score. if you nav​ i​ gate to the ap​ pli​ ca​ tion pro​ file it has a Health tab. Health scores also pro​ vide a real-time cor​ re​ la​ tion in the event of a fail​ ure sce​ nario. the root cause of the issue can be found by ex​ plor​ ing the faults and how these get rolled up in the larger model. In this tab is a tree that will show the var​ i​ ous ob​ jects in a tree form to re​ veal faults. ap​ pli​ ca​ tions. As a ex​ am​ ple. and EPGs are im​ pacted by that fail​ ure.

how​ ever. ACI health scores can help iden​ tify po​ ten​ tial bot​ tle​ necks in terms of hard​ ware re​ sources. this is not al​ ways re​ al​ is​ tic given the dy​ namic na​ ture of data cen​ ter en​ vi​ ron​ ments.Health Scores 275 Object with a fault Proac​ tively. the health of all ap​ pli​ ca​ tion and in​ fra​ struc​ ture com​ po​ nents would al​ ways be at 100%. and other ca​ pac​ ity plan​ ning ex​ er​ cises. band​ width uti​ liza​ tion. Op​ er​ a​ tions teams also stand a bet​ ter chance of iden​ ti​ fy​ ing is​ sues be​ fore they im​ pact cus​ tomers or users. In​ stead the health score should be seen as a met​ ric that will change over time. equip​ ment. with the goal of in​ creas​ ing the av​ er​ age health score of a given set of com​ po​ nents over time. Ide​ ally. and end​ points have fail​ ures. . Links.

.

state​ ful.All devices (Fabric Switches. Fault Lifecycle . In this model. and sta​ tis​ tics in the ACI fab​ ric are rep​ re​ sented as a col​ lec​ tion of Man​ aged Ob​ jects (MOs) within the over​ all ACI Ob​ ject Model/Man​ age​ ment In​ for​ ma​ tion Tree (MIT). It maintains a comprehensive. a fault is rep​ re​ sented as a mu​ ta​ ble. 2 Telemetry device . Virtual Switches. integrated Layer 4 to Layer 7 devices) in an ACI fabric report faults. events and statistics to the APIC. up-to-date run-time representation of the administrative or configured state. events. Faults.Where all fabric configuration is created.Health Scores 277 Understanding Faults From a man​ age​ ment point of view we look at the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) from two per​ spec​ tives: 1 Policy Controller . managed and applied. and per​ sis​ tent MO. All ob​ jects within ACI can be queried. in​ clud​ ing faults.

There can be at most one fault with a given code under an MO. the fault con​ di​ tions are de​ fined by the fault rules of the par​ ent ob​ ject class. To re​ move a fault. es​ ca​ lated. Events. If the same con​ di​ tion is de​ tected mul​ ti​ ple times while the cor​ re​ spond​ ing fault MO is ac​ tive. such as a com​ po​ nent fail​ ure or an alarm. RN.278 Health Scores When a spe​ cific con​ di​ tion oc​ curs. a DN. In most cases. whether by con​ fig​ ur​ a​ - . For a fault ob​ ject class. and so on. de-es​ ca​ lated. the con​ di​ tion rais​ ing the fault must be cleared. The Fault “code” is an al​ phanu​ mer​ ic ​al string in the form FXXX. pag​ i​ na​ tion.xml?query-target=self&rsp-subtreeinclude=health The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the sta​ tis​ tics for a ten​ ant named "3tier​ app": https://hostname/api/node/mo/uni/tn-3tierapp. A fault MO re​ mains in the sys​ tem until the fault con​ di​ tion is cleared.xml?query-target=self&rsp-subtreeinclude=stats The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the faults for a leaf node: https://hostname/api/node/mo/topology/pod-1/node-103. Man​ The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the health score for a ten​ ant named "3tier​ app": https://hostname/api/node/mo/uni/tn-3tierapp.xml?query-target=self&rspsubtree-include=faults As you can see. Fault MOs ap​ pear as reg​ ul​ ar MOs in MIT. no ad​ di​ tional in​ stances of the fault MO are cre​ ated. and Sys​ tem Mes​ sages age​ ment Guide. the sys​ tem cre​ ates a fault MO as a child ob​ ject to the MO that is pri​ mar​ ily as​ so​ ci​ ated with the fault. a fault MO is au​ to​ mat​ ic ​ally cre​ ated. only one fault is raised while a counter for the re​ cur​ rence of that fault will be in​ cre​ mented. For more in​ for​ ma​ tion about fault codes. In other words. if the same con​ di​ tion is de​ tected mul​ ti​ ple times for the same af​ fected ob​ ject. and so on. they have a par​ ent. with prop​ erty fil​ ters. MOs can be queried by class and DN. see the Cisco APIC Faults. and deleted by the sys​ tem as spe​ cific con​ di​ tions are de​ tected.

or a change in the run time state of the fab​ ric. you can set fault thresh​ olds on sta​ tis​ ti​ cal mea​ sure​ ments such as health scores. data traf​ fic.Health Scores 279 tion. Sever​ ity pro​ vides an in​ di​ ca​ tion of the es​ ti​ mated im​ pact of the con​ di​ tion on the ca​ pa​ bil​ ity of the sys​ tem or com​ po​ nent to pro​ vide ser​ vice. An ex​ cep​ tion to this is if the fault is in the cleared or re​ tained state. Pos​ si​ ble val​ ues are: • Warning (possibly no impact) • Minor • Major • Critical (system or component completely unusable) The cre​ ation of a fault MO can be trig​ gered by in​ ter​ nal processes such as: • Finite state machine (FSM) transitions or detected component failures • Conditions specified by various fault policies. in which case the fault can be deleted by the user by ac​ knowl​ edg​ ing it. or tem​ per​ at​ ures. . some of which are user configurable For ex​ am​ ple.

.

All health scores are cal​ cu​ lated using the num​ ber and im​ por​ tance of faults that apply to it.Health Scores 281 How Health Scores Are Calculated Health scores exist for sys​ tems and pods. man​ aged ob​ jects (such as switches and ports). Each fault is weighted de​ pend​ ing on the level of im​ por​ tance. In other words: Health Score calculation Ten​ ant health scores are sim​ i​ lar. which is di​ rectly by the faults they have as​ so​ ci​ ated with them. Crit​ i​ cal faults might have a . Sys​ tem and pod health scores are a weighted av​ er​ age of the leaf health scores. For ex​ am​ ple. mul​ ti​ plied by the spine co​ ef​ fi​ cient which is de​ rived from the num​ ber of spines and their health scores. but con​ tain health scores of log​ i​ cal com​ po​ nents within that ten​ ant. it will only be weighted by the end points that are in​ cluded in that ten​ ant. di​ vided by the total num​ ber of learned end points. You can see how all of these scores are ag​ gre​ gated by look​ ing at how man​ aged ob​ ject scores are cal​ cu​ lated. ten​ ants. as well as an over​ all health score for the over​ all sys​ tem.

a fab​ ric admin will be able to see all health scores. the ten​ ant admin should be able to drill into the health scores that are vis​ ib ​le to them. but there should be a basic un​ der​ stand​ ing of whether faults should have high. such as fire​ walls.282 Health Scores high fault level at 100%. Keep in mind. Luck​ ily there is re​ ally no need to un​ der​ stand the cal​ cu​ la​ tions of the health scores to use them ef​ fec​ tively. and over​ all sys​ tem health scores. but a ten​ ant admin would only be able to see the health scores that per​ tain to the ten​ ants to which they have ac​ cess. be​ cause of the role-based ac​ cess con​ trol. not all ad​ min​ is​ tra​ tors will be able to see all of the health scores. medium. see the Trou​ bleshoot​ ing Cisco Ap​ pli​ ca​ tion tric In​ fra​ struc​ ture book: http://​ datacenter. or “none” fault lev​ els. low. load bal​ ancers. In most cases. along with faults within our VMM do​ mains will all roll up into our ten​ ant.​ github.​ io/​ aci-troubleshooting-book/. These. it is pos​ si​ ble to change these val​ ues to bet​ ter match your en​ vi​ ron​ ment. For more in​ for​ ma​ tion on how to use faults. Faults that have been iden​ ti​ fied as not im​ pact​ ing might even be re​ as​ signed a per​ cent​ age value of 0% so that it does not af​ fect the health score com​ pu​ ta​ tion.​ Cen​ . In this case the fab​ ric ad​ min​ is​ tra​ tor may have to start trou​ bleshoot​ ing. pod. Though faults in ACI come with de​ fault val​ ues. and in​ tru​ sion pre​ ven​ tion/de​ tec​ tion sys​ tems. but it is pos​ si​ ble a fault may be oc​ cur​ ring that is af​ fect​ ing more than that one ten​ ant. while warn​ ings might have a low fault level at only 20%. The ten​ ant and fab​ ric ad​ mins may also see health scores of any layer four through seven de​ vices. For ex​ am​ ple.

and the next week the leaf is show​ ing a warn​ ing. it is pos​ si​ ble the links are over​ sub​ scribed and so it can be time to ei​ ther move some of of the work​ load to an​ other leaf or maybe to add more band​ width by con​ nect​ ing more ca​ bles. it is pos​ si​ ble to send these no​ ti​ fi​ ca​ tions to ap​ pli​ ca​ tion own​ ers. If you see that one of the leaf switches is at 100% (green for good) one week. Since it is still only a warn​ ing. and their as​ so​ ci​ ated faults. Data​ base Ad​ min​ is​ tra​ tor. or fire​ wall. Using Health Scores for Reactive Monitoring Re​ ac​ tively. As shown in the above ex​ am​ ples. there is time to re​ solve the issue be​ fore any bot​ tle​ necks on the net​ work are no​ tice​ able. In these cases adding an​ other load bal​ ancer. and so on. This would pro​ vide mon​ i-​ tor​ ing of the en​ vi​ ron​ ment across the net​ work that has not pre​ vi​ ously been avail​ able and which is not able to be re​ trieved by any other means. Most ob​ jects will have a Health tab which can be used to ex​ plore the re​ la​ tion​ ship be​ tween ob​ jects. you can drill down to see what changed. . Once the root cause faults have been iden​ ti​ fied. this baselin​ ing method can be used as a ca​ pac​ ity plan​ ning tool. Health scores not only act as in​ di​ ca​ tors of faults. This pro​ vides the abil​ ity to “dou​ ble-click to root cause”.Health Scores 283 Health Score Use Cases Using Health Scores for Proactive Monitoring While ACME ad​ min​ is​ tra​ tors have tra​ di​ tion​ ally spent a lot of time re​ act​ ing to is​ sues on the net​ work. or maybe even op​ ti​ miz​ ing the rules may be needed to make traf​ fic flow more ef​ fi​ cient. In this sce​ nario. VMware ad​ min​ is​ tra​ tors. an op​ er​ at​ or can use the GUI to eas​ ily nav​ ig ​ate the re​ la​ tion​ ships and faults that are con​ tribut​ ing to that health score. Other ways health scores can be used to proac​ tively mon​ it​ or your ACI en​ vi​ ron​ ment are by giv​ ing vis​ ib ​il​ ity of cer​ tain com​ po​ nents to other groups. the fault it​ self will con​ tain in​ for​ ma​ tion about pos​ si​ ble re​ me​ di​ at​ ion steps. health scores can be used to di​ ag​ nose prob​ lems with the ACI fab​ ric. ACI health scores will allow them to start pre​ vent​ ing is​ sues. Since you can ex​ port the scores and faults. they are es​ sen​ tially base​ lines to which you can make com​ par​ isons later. The same sce​ nario can ob​ served with a load bal​ ancer or fire​ wall that is get​ ting over​ loaded. Upon no​ ti​ fi​ ca​ tion that a health score has been de​ graded.

.

285 Monitoring .

.

Tenant and Fabric Policies Stats Collection Policies Stats Export Policies Diagnostics Policies Call Home/SNMP/syslog Event Severity and Fault Severity Assignments Fault Lifecycle Policies TCAM Policy Usage Create TCAM Policy Monitor TCAM Prefix Usage Health Score Evaluation Policy Communication Policy • Proactive Monitoring .Monitoring 287 Section Content • Proactive Monitoring .Infrastructure Monitoring APICs CPU utilization and Memory Disk Utilization Physical and Bond Interface Statistics APIC Fan Status Temperature Status Power Supply Status ​Monitoring Leaf Switches Monitoring Switch CPU Utilization Monitoring Switch Memory Utilization Monitoring File System Health Monitoring CoPP (Control Plane Policing) Statistics Physical Interface Statistics and Link State Module Status Switch Fan Status .

288 Monitoring Power Supply Status LLDP Neighbor Status GOLD Diagnostic Results • Proactive Monitoring Use Cases Monitoring Workload Bandwidth EPG Level Statistics • Reactive Monitoring Reactive Monitoring Tools Switch Port Analyzer (SPAN) Traceroute Atomic Counters Traffic Map Enhanced Troubleshooting Wizard IPing Audit Logs • Reactive Monitoring Use Cases ​Loss of Connectivity to Endpoint Users Report that an Application Running in the Fabric is Slow .

or Ac​ cess. but is often ne​ glected be​ cause putting out fires in the net​ work usu​ ally takes pri​ or​ ity. Add test VMs to port groups on ei​ ther a DVS or AVS as​ so​ ci​ ated with the APIC. A Fab​ ric Wide pol​ icy would be cre​ ated as a de​ fault pol​ icy to be ap​ plied to all ten​ ants. CLI. this will save net​ work ad​ min​ is​ tra​ tors both time and frus​ tra​ tion. it is sen​ si​ ble to start by test​ ing be​ fore going straight to pro​ duc​ tion. the human error and ef​ fort is min​ im ​al. How​ ever. since the APIC makes it in​ cred​ ib ​ly easy to gather sta​ tis​ tics and per​ form analy​ ses. Sta​ tis​ tics gath​ er​ ing has been a some​ what man​ ual and even re​ source in​ ten​ sive process for ACME in the past. but more im​ por​ tantly it en​ ables the setup of poli​ cies to keep per​ sis​ tent data to an​ al​ yze trends in the en​ vi​ ron​ ment. For ex​ am​ ple. to over​ ride that pol​ icy for a par​ tic​ ul​ ar ten​ ant. There are four dif​ fer​ ent scopes for sta​ tis​ tics gath​ er​ ing: Com​ mon or Fab​ ric Wide. When plan​ ning to move an ap​ pli​ ca​ tion from a legacy net​ work to the ACI in​ fra​ struc​ ture. At this point the APIC is al​ ready gath​ er​ ing sta​ tis​ tics for the VMM do​ main and the phys​ ic ​al de​ vices. can be used to drill into any of the com​ po​ nents and pro​ vides the abil​ ity to click on a Stats tab to dis​ play on-de​ mand sta​ tis​ tics. How​ ever. ACI pro​ vides an APIC which will do all of the sta​ tis​ tics gath​ er​ ing. Ten​ ant. and pro​ vides the abil​ ity to proac​ tively mon​ it​ or your en​ tire en​ vi​ ron​ ment with​ out all of the has​ sle of main​ tain​ ing a third party mon​ it​ or​ ing tool. Since sta​ tis​ tics are gath​ ered au​ to​ mat​ ic ​ally and poli​ cies are used and can be re-used in other places. it has still been nec​ es​ sary to man​ ua ​lly spec​ ify which de​ vices are to be mon​ it​ ored and how they should be mon​ it​ ored. Fab​ ric. the ten​ - . The APIC. and add phys​ ic ​al test servers to VPCs on the leaf switches. or API. but they strug​ gled with en​ ter​ ing cor​ rect SNMP in​ for​ ma​ tion on each de​ vice. SNMP and a third party tool may have been used to mon​ it​ or the CPU of switches or band​ width uti​ liza​ tion on ports.Monitoring 289 Proactive Monitoring Tenant and Fabric Policies Proac​ tive mon​ it​ or​ ing is a very im​ por​ tant piece of the net​ work ad​ min​ is​ tra​ tor's job. This could also be in a test​ ing ten​ ant which is com​ pletely sep​ ar​ ate from the pro​ duc​ tion en​ vi​ ron​ ment. as well as to trou​ bleshoot or pre​ dict any is​ sues that may be aris​ ing. whether ac​ cessed through the GUI. or often for​ got to add a new de​ vice to their Net​ work Mon​ it​ or​ ing Sys​ tem (NMS). Even when they have used tools to gather data on layer one through seven de​ vices. The next step is to con​ fig​ ure a pol​ icy for trend analy​ sis.

just click on the pol​ icy that spec​ if​ ies a 5 minute gran​ ul​ ar​ ity and change the re​ ten​ tion pe​ riod to 2 hours. For ex​ am​ ple. perform the following actions: a. to have it poll a com​ po​ nent every 5 min​ utes. In the fol​ low​ ing test​ ing ex​ am​ ple. Even if this ten​ ant is shared with other ap​ pli​ ca​ tions.290 Monitoring ant pol​ icy will over​ ride the Fab​ ric pol​ icy. choose Actions > Create Monitoring Policies. but a dou​ ble click on them will change the admin state or re​ ten​ tion pe​ ri​ ods. 5 In the Create Monitoring Policies dialog box. 6 In the Navigation pane. choose Tenant_Name > Monitoring Policies > Policy_Name to display the following information: • Stats Collection Policies • Stats Export Policies • Diagnostics Policies • Callhome. test cases. Most likely the de​ faults will be kept. choose the Tenant_Name. SNMP. choose Tenant_Name > Monitoring Policies. Click Submit. Create Tenant Monitor policy To cre​ ate a ten​ ant mon​ it​ or​ ing pol​ icy: 1 On the menu bar. it will pro​ vide a real world ex​ am​ ple of how the ap​ pli​ ca​ tion will be​ have in a pro​ duc​ tion en​ vi​ ron​ ment. In the Name field enter a name for the Monitoring Policy. It is sim​ il​ arly pos​ si​ ble to change the poli​ cies for spe​ cific Mon​ it​ or​ ing Ob​ jects. choose Tenants > ALL TENANTS. b. 4 In the Work pane. 3 In the Navigation pane. but be re​ tained for 2 hours. cus​ tomers. a Ten​ ant pol​ icy is cre​ ated to gather sta​ tis​ tics. and syslog • Event Severity Assignment Policies • Fault Lifecycle Policies Stats Collection Policies Click​ ing on Stats Col​ lec​ tion Poli​ cies will dis​ play the de​ fault re​ ten​ tion pe​ ri​ ods and admin states (En​ abled/Dis​ abled) for ALL Mon​ it​ ored Ob​ jects. A mon​ it​ or​ ing ob​ ject tells the APIC which com​ po​ nents to gather . 2 In the Work pane.

choose the Tenant_Name. c. 3 In the Navigation pane choose Monitoring Policies > Monitoring Policy_Name > Stats Collection Policies. or explicitly select enabled or disabled. choose Tenants > ALL TENANTS. 2 In the Work pane. Stats Export Policies . f. in the Stats Collection Policy dialog box. For ex​ am​ ple. changes might be made to Mon​ it​ or​ ing Ob​ ject poli​ cies for Ten​ ant. Leave the state as inherited to stick with the defaults as set for ALL. b. c. 2 In the Work pane. to change the in​ for​ ma​ tion gath​ ered for Bridge Do​ mains. There are sev​ eral op​ tions and this will all de​ pend on what is im​ por​ tant to mon​ it​ or in the en​ vi​ ron​ ment. To add a pol​ icy to a Mon​ it​ or​ ing Ob​ ject: 1 On the menu bar.​ RSOInfraBD) Mon​ it​ or​ ing Ob​ ject. Click on the pull down menu to se​ lect a mon​ it​ or​ ing ob​ ject and add a re​ ten​ tion pol​ icy to it. VXLAN Pool. Click Update. To add mon​ it​ or​ ing ob​ jects: 1 On the menu bar.Monitoring 291 sta​ tis​ tics about. Click + to add the policy. perform the following actions: a. and remove any checkmarks next to Monitoring Objects to be left out. For this ex​ am​ ple. Select the Monitoring Object. use the Bridge Do​ main (infra. The retention policy may either be inherited or explicitly specified as enabled or disabled as well. and/or Taboo Con​ tract. 3 In the Navigation pane choose Tenant_Name > Monitoring Policies > Monitoring Policy_Name > Stats Collection Policies a. Click on the Pencil icon to edit the Monitoring Objects. choose Tenants > ALL TENANTS. Leaf Port. b. Click Submit. Select the granularity with which it is to poll. choose the Tenant_Name. Put a checkmark next to the Monitoring Objects to be included. e. 4 In the Work pane. d.

2 In the Work pane. Another wizard will pop up to enable specification of the protocols and credentials used to connect to this server. Choose either JSON or XML as the format. choose Tenants > ALL TENANTS. Click + to add the policy.292 Monitoring Stats Export Policies It is de​ sir​ able to col​ lect these on​ go​ ing sta​ tis​ tics as well as to see how this data be​ haves over time. b.Boot-Up di​ ag​ nos​ tics or On​ go​ ing di​ ag​ nos​ tics. d. There's really no difference other than personal preference. There are two dif​ fer​ ent kind of poli​ cies for con​ fig​ ur​ a​ tion . This is a re​ ally slick fea​ ture that al​ lows the setup of di​ ag​ nos​ tics test for the Mon​ it​ or​ ing Ob​ jects that were spec​ if​ ied in the Stats Col​ lec​ tion Poli​ cies. in the Stats Export Policy dialog box. Much like the Stats Col​ lec​ tion Poli​ cies. 5 Click Submit. Select ALL or a specific monitoring object from the drop-down list. or it may be dictated by the tool used to read it. perform the following actions: a. Click Ok. Use the Stats Ex​ port Poli​ cies op​ tion in the left nav​ ig ​a​ tion pane. it is pos​ si​ ble to cre​ ate a pol​ icy for ALL mon​ it​ or​ ing ob​ jects. Now define the Stats Export Policy in the wizard. or leave it uncompressed. To cre​ ate a Stats Ex​ port Pol​ icy: 1 On the menu bar. Choose to compress it using GZIP. Click + under Export Destinations to specify a server where this information is to be collected. lo​ cated under the mon​ it​ or​ ing pol​ icy. or se​ lect spe​ cific mon​ it​ or​ ing ob​ jects and spec​ ify where this in​ for​ ma​ tion will be saved. Diagnostics Policies Next are the di​ ag​ nos​ tics poli​ cies in the nav​ ig ​a​ tion pane on the left. e. c. 3 In the Navigation pane choose Tenant_Name > Monitoring Policies > Monitoring Policy_Name > Stats Export Policies. . Next to the Mon​ it​ or​ ing Ob​ ject is the Pen​ cil but​ ton which en​ ables se​ lec​ tion of the mon​ it​ or​ ing ob​ jects to be con​ fig​ ured with di​ ag​ nos​ tics poli​ cies. choose the Tenant_Name. g. 4 In the Work pane. f.

This al​ lows ad​ min​ is​ tra​ tors to re​ solve is​ sues be​ fore they turn into out​ ages. perform the following actions: Note: Click on the Pencil Icon and put checks next to the Monitoring Objects which diagnostics tests are to be added to. b. The di​ ag​ nos​ tics found here can be use​ ful in find​ ing failed com​ po​ nents be​ fore they cause major is​ sues within your en​ vi​ ron​ ment. select either Boot-Up or Ongoing. Cisco Call Home is a fea​ ture in many Cisco prod​ ucts that will pro​ vide email or webbased no​ ti​ fi​ ca​ tion alerts in sev​ eral dif​ fer​ ent for​ mats for crit​ ic ​al events. Peripherals. full tests. The Call Home/SNMP/sys​ log pol​ icy will allow alert​ ing to be con​ fig​ ured in a flex​ ib ​le man​ ner. choose Fabric > Fabric Policies. 3 In the Work pane. Boot-Up runs the tests while the devices are booting.Monitoring 293 To con​ fig​ ure di​ ag​ nos​ tic poli​ cies: 1 On the menu bar. 2 In the Navigation pane choose Tenant_Name > Monitoring Policies > default > Diagnostics Policies. Dif​ fer​ ent log​ ging lev​ els may be se​ lected for no​ ti​ fi​ ca​ tions and alert lev​ els spec​ if​ ied for Mon​ it​ or​ ing Ob​ jects from which alerts are to be re​ ceived. and Ongoing will run the tests as often as specified within the wizard. Double-click on each to obtain the option of specifying no tests. iii. i. or recommended tests. Select one of the Monitoring Objects. Click Submit. In the wizard give it a name and select the admin state. v. a. in the Diagnostic Policies dialog box. and System Memory. ii. . Call Home/SNMP/syslog There are a few dif​ fer​ ent ways to setup no​ ti​ fi​ ca​ tion or alert poli​ cies. Internal Connectivity. SNMP or sys​ log poli​ cies can also be used with cur​ rent no​ ti​ fi​ ca​ tion sys​ tems. Click + to add an Object. iv. CPU. There are five different diagnostics tests available: ASIC.

2 In the Work pane. For ex​ am​ ple." It is only in this state briefly and moves on to the "clear​ ing time" state. After a cer​ tain amount of time. re​ ferred to as the "soak​ ing in​ ter​ val" it will move on to the "raised" state. the de​ fault sever​ ity as​ sign​ ments for Events and Faults will be kept. but there are ex​ am​ ples where an ACI ad​ min​ is​ tra​ tor may de​ cide the event or fault is more or less se​ vere than the de​ fault value." . Select a Monitoring Object. Warning. Select the particular fault code for which severity is to be changed. "Raised" means the fault is still pre​ sent after the soak​ ing in​ ter​ val. you can change the sever​ ity for that par​ tic​ ul​ ar fault code. It re​ mains in the "clear​ ing time" state for the amount of time spec​ if​ ied in the "clear​ ing in​ ter​ val. 3 In the Navigation pane.294 Monitoring Event Severity and Fault Severity Assignments Event and fault sever​ it​ ies can be changed for events raised by Mon​ it​ or​ ing Ob​ jects. but there is a major fault you'd also like to be no​ ti​ fied about im​ me​ di​ ately. Squelched. Note: Squelched gives it a weight of 0%. in the Fault Severity Assignment Policies dialog box." Lastly it moves on to the "re​ tain​ ing" state and does not get re​ moved until the end of the "re​ tain​ ing in​ ter​ val. Info. choose Tenants > ALL TENANTS. perform the following actions: a. Once a fault is de​ tected it is in the "soak​ ing" state. The Event Sever​ ity As​ sign​ ment Poli​ cies are con​ fig​ ured in the same way. b. Select the severity: Cleared. Major. Most likely. Click + to add an Object. After the fault clears it's in a state called "raised clear​ ing. Fault Lifecycle Policies Fault Life​ cy​ cle is the term Cisco uses to de​ scribe the life of a fault. 1 On the menu bar. if only crit​ ic ​al faults are being no​ ti​ fied. d. Critical. meaning it does not affect health scores. c. 5 Click Update. Inherit. which will dictate the fault codes for which you are changing the fault severity. Minor. 4 In the Work pane. choose Tenant_Name > Monitoring Policies > Monitoring_Policy > Fault Lifecycle Policies. choose the Tenant_Name.

Click +. Retention Interval. choose Tenants > ALL TENANTS. Within the Cisco ACI fab​ ric. Select a Monitoring Object. choose Tenant_Name > Monitoring Policies > Monitoring_Policy > Fault Lifecycle Policies. 4 In the Work pane. choose the Tenant_Name. 3 In the Navigation pane. and Soaking Interval (all in seconds). There is a a sys​ tem wide view of avail​ able TCAM re​ sources. TCAM Policy Usage The phys​ ic ​al ternary con​ tent-ad​ dress​ able mem​ ory (TCAM) in which pol​ icy is stored for en​ force​ ment is an ex​ pen​ sive com​ po​ nent of switch hard​ ware and there​ fore tends to lower pol​ icy scale or raise hard​ ware costs. m is the num​ ber of des​ ti​ na​ tions. and you will see a table sum​ ma​ riz​ ing ca​ pac​ i-​ tiy for all nodes. . perform the following actions: a. Specify times for the Clearing Interval. At this point there will be a fully work​ ing ten​ ant mon​ it​ or​ ing pol​ icy. sources and des​ ti​ na​ tions be​ come one entry for a given EPG. Within the Cisco ACI fab​ ric. 2 In the Work pane.Monitoring 295 To change Fault Life​ cy​ cle In​ ter​ vals: 1 On the menu bar. in the Fault Lifecycle Policies dialog box. ACME will have other poli​ cies to con​ fig​ ure in the fab​ ric as out​ lined in the fol​ low​ ing sec​ tions. the Retention Interval is 3600 seconds. This pol​ icy size can be ex​ pressed as n*m*f. Note: The default for the Clearing Interval is 120 seconds. To see this click on Fab​ ric > In​ ven​ tory > Pod1 and then se​ lect the Op​ er​ a​ tional tab in the work pane. where n is the num​ ber of sources. and f is the num​ ber of pol​ icy fil​ ters. pol​ icy is ap​ plied based on the EPG rather than the end​ point it​ self. which will dictate the fault codes for which you are changing the default intervals. TCAM is a fab​ ric re​ source that should be mon​ it​ ored. b. which re​ duces the num​ ber of total en​ tries re​ quired. and the Soaking Interval is 120 seconds. c.

There is a Fab​ ric Re​ source Cal​ cu​ la​ tion tool on Github that will help with es​ ti​ ma​ tion of nor​ mal op​ er​ at​ ing pa​ ra​ me​ ters: https://​ github.296 Monitoring Switch TCAM capacity dashboard TCAM is a crit​ i​ cal sys​ tem re​ source in an ACI fab​ ric and should be mon​ i​ tored for uti​ liza​ tion. The ar​ chi​ tec​ ture/de​ sign team should ar​ tic​ u​ late what the as​ sump​ tions were for TCAM uti​ liza​ tion.​ com/​ datacenter/​ FabricResourceCalculation As a gen​ eral rule. 2 In the Navigation pane. . If your en​ vi​ ron​ ment has a high rate of change. the de​ fault mon​ i​ tor​ ing poli​ cies will alert you to a re​ source short​ age and lower over​ all fab​ ric health score. choose Fabric > Fabric Policies. or you an​ tic​ i​ pate the pos​ si​ bil​ ity of con​ sis​ tently being over​ sub​ scribed. you may want to set dif​ fer​ ent thresh​ olds. Create TCAM Policy Monitor 1 On the menu bar. choose Monitor Policies > default > Stats Collection Policies.

in the Properties dialog box. select the desired %. select the blue pencil icon next to policy CAM entries usage current value. d. d. b. 3 In the Work pane. Select the Stats TypeLayer3 Entry. c. Click + under Config Thresholds. b. d. 1 On the menu bar. select the desired %. c. in the Stats Collection Policies dialog box. Health Score Evaluation Policy 1 On the menu bar. 2 In the Navigation pane. In the Penalty of fault severity major dropdown menu.Entity). select the desired %. in the Stats Collection Policies dialog box. 3 In the Work pane.Monitoring 297 3 In the Work pane. Select the Stats Type Policy Entry. In the Penalty of fault severity critical dropdown menu. perform the following actions: a. Select the Monitoring Object Equipment Capacity Entity (eqptcapacity. Select the Monitoring Object Equipment Capacity Entity (eqptcapacity. select the desired %. In the Penalty of fault severity warning dropdown menu. choose Monitor Policies > default > Stats Collection Policies. b. In the Thresholds For Collection 5 Minute window. choose Fabric > Fabric Policies. In the Thresholds For Collection 5 Minute window. choose Fabric > Fabric Policies. . c. choose Monitor Policies > Common Policies > Health Score Evaluation Policy > Health Score Evaluation Policy. Click + under Config Thresholds. In the Penalty of fault severity minor dropdown menu. TCAM Prefix Usage This pro​ ce​ dure man​ ages a TCAM Pre​ fix Usage. select the blue pencil icon next to policy CAM entries usage current value. 2 In the Navigation pane.Entity). perform the following actions: a. perform the following actions: a.

j. f. From the HTTPS Port dropdown menu select the desired port. c.298 Monitoring 4 Click Submit. d. b. perform the following actions: a. 4 In the Create Communication Policy dialog box. e. 3 In the Work pane. From the HTTP Admin State dropdown menu select the desired state. g. expand Pod Policies > Policies > Communication. Select the desired HTTP redirect state. . From the Telnet Port dropdown menu select the desired port. Enter Communication Policy Name. Select the desired HTTPS redirect state. 2 In the Navigation pane. choose Fabric > Fabric Policies. i. From the Telnet Admin State dropdown menu select the desired state. h. From the SSH Admin State dropdown menu select the desired state. From the HTTPS Admin State dropdown menu select the desired state. 5 Click Submit. choose Actions > Create Communication Policy. From the HTTP Port dropdown menu select the desired port. Communication Policy 1 On the menu bar.

​ cisco. GUI. The screen​ shot shows where this in​ for​ ma​ tion is vis​ ib ​le. the health of the APICs and the health of the clus​ ter it​ self are dis​ played right at the dash​ board. This sec​ tion of the book at​ tempts to cover some key per​ for​ mance in​ di​ ca​ tors that should be mon​ it​ ored. In this ex​ am​ ple.Monitoring 299 Proactive Monitoring . and fab​ ric and ten​ ant poli​ cies show us a nar​ rower view of fab​ ric and ten​ ant health. SNMP. and when pos​ si​ ble. . and ref​ er​ ences are pro​ vided to ways of ob​ tain​ ing the data when pos​ si​ ble. When log​ ging into the sys​ tem dash​ board.​ html It is im​ por​ tant to note that this list changes as newer soft​ ware ver​ sions are made avail​ able. A full list of MIBs sup​ ported on the 1. Monitoring APICs CPU utilization and Memory GUI The eas​ ie ​st way to quickly ver​ ify the health of the con​ trollers is the APIC. ACME will also need to set up poli​ cies to mon​ it​ or spe​ cific re​ sources. pro​ ce​ dures for mon​ it​ or​ ing them. sug​ gest rec​ om​ mended thresh​ olds for trig​ ger​ ing alerts or alarms from Net​ work Mon​ it​ or​ ing Sys​ tems (NMS). and more vari​ ables may be ex​ posed via SNMP MIBs in the fu​ ture.Infrastructure While health scores pro​ vide a com​ pre​ hen​ sive view of the health sta​ tus of var​ io ​us ob​ jects. REST API calls and CLI.x is avail​ able at: http://​ www. There are sev​ eral meth​ ods to ob​ tain this data. Op​ er​ at​ ors can use a wide array of tools in​ clud​ ing Sys​ log.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. two out of the three APICs are in a sub-op​ ti​ mal state and APIC 1 is also ex​ pe​ ri​ enc​ ing is​ sues.

xml? . A more de​ tailed drill​ down is avail​ able by click​ ing on Sys​ tem > Con​ trollers.300 Monitoring System health dashboard The nor​ mal state for these is to have them all green in a "fully fit" state im​ ply​ ing the APICs are syn​ chro​ nized with each other. REST API Con​ trollers pro​ vide in​ for​ ma​ tion re​ gard​ ing the cur​ rent sta​ tus of CPU and mem​ ory uti​ liza​ tion by cre​ at​ ing in​ stances of the pro​ cEn​ tity class.The max​ i​ mum amount of avail​ able mem​ ory for the sys​ tem Sam​ ple Usage: This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: http[s]://apic_ip/api/node/class/procEntity.CPU uti​ liza​ tion maxMemAl​ loc . This ob​ ject holds de​ tailed in​ for​ ma​ tion about var​ i​ ous processes run​ ning on the APIC. pro​ cEn​ tity is a con​ tainer of processes in the sys​ tem.The max​ i​ mum mem​ ory al​ lo​ cated for the sys​ tem mem​ Free . The pro​ cEn​ tity ob​ jects con​ tain the fol​ low​ ing use​ ful prop​ er​ ties: cpuPct .

0 stopped.04 svc_ifc_applian 32120 ifc 20 0 660m 343m 86m S 2.bin 1291 root VIRT 0k free.34 svc_ifc_reader. 0. .2 38:11. RES SHR S %CPU %MEM 0.0%wa.11:41:51 up 16:50. Cpu(s): Mem: 0.0%st 1952656k cached PID USER PR NI 32102 root 20 0 556m 205m 85m S 3.24 nginx.3 27:58. 409540k buffers TIME+ COMMAND Disk Utilization GUI There are sev​ eral disks and file sys​ tems pre​ sent on the APICs. The disk uti​ liza​ tion can be viewed by click​ ing on Sys​ tem > Con​ trollers > Apic-X > Stor​ age The work pane dis​ plays the uti​ liza​ tion of all par​ ti​ tions in the sys​ tem. user@apic1:~> top top .2 17:13.4%sy. 353 sleeping.0 0.2 17:08. 32132 ifc 20 0 657m 252m 71m S 1. The GUI pro​ vides ready ac​ cess to disk space uti​ liza​ tion of all par​ ti​ tions on the sys​ tem and can be used for mon​ i​ tor​ ing this in​ for​ ma​ tion.0%si.0%ni.0 0.6 20:03. 0k total. 7473180k used. Tasks: 354 total.74 svc_ifc_scripth 20 0 834m 419m 94m S 1.7 0. 131954932k total. 124481752k free.3 0.2 16:28. load average: 4.7 0.Monitoring 301 CLI The Linux Top util​ ity also comes built into the APIC con​ trollers and can be used for trou​ bleshoot​ ing and/or ver​ if​ i​ ca​ tion. Swap: 4 users.3%id.37 svc_ifc_observe 32128 ifc 20 0 639m 315m 69m S 1. 0.19.27.29 1 running. 99.2 17:41.73 svc_ifc_dbgr.3 0.92 svc_ifc_topomgr 32105 root 20 0 659m 258m 85m S 1.bi 32121 ifc 20 0 631m 286m 86m S 2.0%hi. 0 zombie 0.3%us. 0. 0k used. 4. 0.35 svc_ifc_bootmgr 32113 ifc 20 0 1083m 721m 69m S 1.3 20:35.7 0.7 0. 4.

The GUI should be used as a sin​ gle source of truth for file sys​ tem uti​ liza​ tion. Bond0 is the bond in​ ter​ face used to con​ nect to the fab​ ric it​ self (to con​ nect to leaves that con​ nect into the fab​ ric). .xml? Physical and Bond Interface Statistics APICs use a bonded in​ ter​ face that is typ​ ic ​ally dual-homed to two leaves for con​ nec​ tiv​ ity to the ACI fab​ ric and have the abil​ ity to use a bonded in​ ter​ face that can be dual homed to the out-of-band man​ age​ ment net​ work.302 Monitoring CLI user@apic1:~> df Filesystem 1K-blocks /dev/dm-1 41282880 10518960 Used Available Use% Mounted on 28666872 tmpfs 4194304 56456 4137848 tmpfs 65977464 964 65976500 1% /tmp 10518960 28666872 27% /data 13860672 25327104 36% /firmware 27% / 2% /dev/shm /dev/mapper/vg_ifc0-data 41282880 /dev/mapper/vg_ifc0-firmware 41284928 /dev/mapper/vg_ifc0-data2 583149656 1281104 552246280 1% /data2 * Note that not all file sys​ tems are vis​ ib ​le from the CLI as some re​ quire root ac​ cess to reach the mount points. REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: http[s]://apic-ip/api/node/class/eqptStorage. Bond1 is the bond in​ ter​ face used to con​ nect to the out-of-band seg​ ment (to con​ nect to an OOB seg​ ment that al​ lows setup of the APIC it​ self).

GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > Equip​ ment-Fans REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-1. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. The CLI also pro​ vides in​ for​ ma​ tion on de​ tailed in​ ter​ face sta​ tis​ tics such as RX and TX coun​ ters.json?querytarget=subtree&target-subtree-class=eqptFan CLI . nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > In​ ter​ faces CLI Both "if​ con​ fig" and the "ip link" CLI com​ mands can be used to ver​ ify link state.Monitoring 303 The bond in​ ter​ faces rely on un​ der​ ly​ ing phys​ ic ​al in​ ter​ faces and it is im​ por​ tant to note that the GUI pro​ vides link in​ for​ ma​ tion for both the phys​ ic ​al and log​ ic ​al bond in​ ter​ faces.json?querytarget=subtree&target-subtree-class=l3EncRtdIf APIC Fan Status The fol​ low​ ing sec​ tion de​ scribes method​ olo​ gies to re​ trieve the sta​ tus of the fan trays on the APICs. REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-1/sys.

The CIMC port is the in​ te​ grated lights-out man​ age​ ment port that can be used to re​ cover an APIC in the event of a cat​ as​ trophic fail​ ure. To ob​ tain this.179' (RSA) to the list of known hosts.179 Warning: Permanently added '172.16. the de​ fault user​ name is admin and the de​ fault pass​ word is pass​ word.------.176.-------- Min. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. admin@172.--------- FAN1_TACH1 Normal 7490 RPM 1712 N/A 1284 N/A FAN1_TACH2 Normal 7490 RPM 1712 N/A 1284 N/A FAN2_TACH1 Normal 7490 RPM 1712 N/A 1284 N/A FAN2_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN3_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A FAN3_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN4_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A FAN4_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN5_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A Temperature Status To mon​ it​ or the tem​ per​ at​ ure state of the var​ io ​us sen​ sors avail​ able on the APICs use the fol​ low​ ing steps. user@apic1:~> ssh -l admin 172. If this has not been setup pre​ vi​ ously. login to the CIMC using the cre​ den​ tials used for set​ ting up the CIMC (may not be the same as the cre​ den​ tials used for APIC).176.16. Max.-------.304 Monitoring CLI The Fan sta​ tus for the APICs can be mon​ it​ ored using the CLI on the CIMC port of the APIC. nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > Equip​ ment-Sen​ sors REST API . Max.179's password: C220-FCH1807V02V# scope sensor C220-FCH1807V02V /sensor # show fan Name Sensor Reading Units Status ----------.176.16. Warning Warning Failure Failure ----.--------.--------. Min.

Monitoring 305 REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}//api/node/mo/topology/pod-1/node-1. Min.0 DDR3_P2_E1_TEMP Normal 43.0 N/A 90.0 DDR3_P1_A1_TEMP Normal 42.0 C N/A 60.0 C N/A 85.--------- P1_TEMP_SENS Normal 49.0 C N/A 60.0 C N/A 65. Max. Warning Warning Failure Failure ------.0 RISER2_OUTLETTMP Normal 41.0 DDR3_P1_C1_TEMP Normal 44.0 DDR3_P2_H1_TEMP Normal 41.0 RISER1_INLET_TMP Normal 45.0 DDR3_P1_D1_TEMP Normal 44.0 N/A 85.0 C N/A 60.0 N/A 70.0 C N/A 65.--------.5 C N/A 81.0 RISER1_OUTLETTMP Normal 50.0 DDR3_P1_B1_TEMP Normal 43.0 C N/A 65.0 N/A 65.0 PSU1_TEMP Normal 37.0 N/A 86.0 VICP81E_0_TMP3 Normal 56.0 N/A 85.0 N/A 85.0 C N/A 60. .0 FP_TEMP_SENSOR Normal 37.0 N/A 85.0 C N/A 60. Max.0 C N/A 65.0 N/A 85.0 C N/A 65.0 RISER2_INLET_TMP Normal 41.0 N/A 70.json?querytarget=subtree&target-subtree-class=eqptSensor CLI C220-FCH1807V02V /sensor # show temperature C220-FCH1807V02V /sensor # show temperature Name Sensor Reading Units Status -----------------.------.0 N/A 70.0 C N/A 60.0 N/A 86.-------- Min.0 N/A 85.0 C N/A 80.-------.0 N/A 85.0 DDR3_P2_F1_TEMP Normal 43.0 N/A 85.0 P2_TEMP_SENS Normal 50.0 C N/A 65.0 N/A 70.0 C N/A 65.0 DDR3_P2_G1_TEMP Normal 42.0 N/A 70.5 C N/A 81.0 PCH_TEMP_SENS Normal 51.0 C N/A 65.-------.0 N/A 85.0 Power Supply Status To mon​ it​ or the tem​ per​ at​ ure state of the var​ io ​us sen​ sors avail​ able on the APICs use the fol​ low​ ing steps.

-------- -------- Min.-------. all ex​ ter​ nal con​ nec​ tiv​ ity is pro​ vided through a set of leaf switches that pro​ vide high den​ sity 10gig or 40gig con​ nec​ tiv​ ity to el​ e​ ments out​ side the fab​ ric.0 N/A 86. Min. Max.--------- P1_TEMP_SENS Normal 49. in the ACI fab​ ric. Warning Warning Failure Failure ------. To ac​ cess the dash​ board to mon​ it​ or switches nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-* .--------. Max. The leaf switches pro​ vide first hop con​ nec​ tiv​ ity to any​ thing that at​ taches to the fab​ ric.-------.5 C N/A 81.306 Monitoring CLI C220-FCH1807V02V /sensor # show psu Name Sensor Reading Units Status -----------------.0 POWER_USAGE Normal 160 Watts N/A N/A N/A 800 PSU1_POUT Normal 136 Watts N/A 624 N/A 648 PSU1_PIN Normal 160 Watts N/A 720 N/A 744 PSU1_STATUS Normal present PSU2_STATUS Normal absent PSU1_PWRGD Normal good PSU1_AC_OK Normal good Monitoring Leaf Switches Leaf switches in the ACI fab​ ric typ​ ic ​ally equate to the Nexus 9300 fam​ ily of switches (with the ex​ cep​ tion of the Nexus 9336 switch). Note that un​ like tra​ di​ tional two or three tier de​ signs where the "WAN" layer at​ taches to ei​ ther the dis​ tri​ bu ​t​ ion/ag​ gre​ ga​ tion layer.

The fol​ low​ ing sec​ tions de​ scribe a few of the meth​ ods avail​ able. proc:SysCPU5min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 5 minute sam​ pling in​ ter​ val. This class up​ dates every 10 sec​ onds. Monitoring Switch CPU Utilization There are sev​ eral meth​ ods to poll CPU uti​ liza​ tion and trend it over dif​ fer​ ent pe​ ri​ ods of time. and the sub tabs on the right hand side (topol​ ogy/gen​ eral/stats/health/faults/trou​ bleshoot​ ing/his​ tory) can be used to quickly drill down into the var​ i​ ous prop​ er​ ties of the switch to un​ der​ stand how the switch is de​ ployed from a hard​ ware con​ fig​ u​ ra​ tion stand​ point. and trou​ bleshoot the switch from a hard​ ware per​ spec​ tive. based on the de​ sired timescale and gran​ u​ lar​ ity. re​ me​ di​ ate faults on the switch.Monitoring 307 Leaf switch monitoring dashboard No​ tice that the dash​ board for this switch de​ faults to pre​ sent​ ing the health score of the switch at a node level on the dash​ board. REST API Spine and Leaf switches CPU uti​ liza​ tion can be mon​ i​ tored using the fol​ low​ ing classes. .

http[s]://apic_ip//api/node/class/procSysCPU1d. proc:SysCPU1qtr A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 quar​ ter sam​ pling in​ ter​ val.4% svc_ifc_policye 4302 1904 1862 1022 0.3% python 4292 3841 134758 28 0. This class up​ dates every 5 min​ utes. proc:SysCPU1w A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 week sam​ pling in​ ter​ val. This class up​ dates every 15 min​ utes. ACME would like to see the av​ er​ age CPU uti​ liza​ tion of all of the fab​ ric switches over the last day.xml? CLI Leaf-1# show proc cpu sort PID Runtime(ms) Invoked uSecs 1Sec Process ----- ----------- -------- ----- ------ ----------- 4012 69510 493837 140 1. proc:SysCPU1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 hour sam​ pling in​ ter​ val. This class up​ dates every day. re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 proc:SysCPU1d A class that rep​ day sam​ pling in​ ter​ val.4% nginx 4067 1911 206 9278 0. This class up​ dates every day.8% svc_ifc_opflexe 4391 2355 4423 532 0. proc:SysCPU1mo A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 month sam​ pling in​ ter​ val. This class up​ dates every day.3% t2usd_tor 4065 7239 27609 262 1.3% svc_ifc_observe .308 Monitoring proc:SysCPU15min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 15 minute sam​ pling in​ ter​ val. proc:SysCPU1year A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 year sam​ pling in​ ter​ val. This class up​ dates every day. This class up​ dates every hour.

5..0% mcecm In order to ob​ tain a his​ tor​ ic ​al view of CPU uti​ liza​ tion from the CLI it may be nec​ es​ sary to jump into an al​ ter​ na​ tive shell from the switch bash prompt.1% ospf 13606 435 211 2065 0...4..0% policy_mgr 4301 1860 19152 97 0. Leaf-1# show processes cpu history 1 1 33 1 746554661885833055376572545534667663554785033943645665335644 100 90 80 70 60 50 40 # 30 ## 20 ## 10 # ### ####### ######## # ## ##### ## #### # # #### ## 0.5..3% svc_ifc_confele 4123 1407 251 5606 0.......1..3% svc_ifc_eventmg 4310 1802 689 2616 0....2% stats_manager 3923 15406 2645 5824 0. 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) # = average CPU% 5 0 5 .1% pfmclnt 4864 2361 2812 839 0..0% dc3_sensor 4299 8559 8225 1040 0.0% snmpd 4297 6667 4542 1467 0.3.....5..0% plog 4866 2792 3269 854 0...2.0% bgp 4296 6263 7413 844 0....4.3....0% isis 5025 1611 1743 924 0...1% ospfv3 4865 2402 2717 884 0.. This shell is called vsh (or v-shell)..1..3% svc_ifc_dbgrele 4846 119693 36527 3276 0..Monitoring 309 4311 1811 1018 1779 0.2..

...3..4..1...2. 0 5 0 5 0 5 0 5 CPU% per hour (last 72 hours) * = maximum CPU% # = average CPU% 0 5 0 5 0 .6............6...5...1............7...2..4.........5..3..5.4.310 Monitoring 32 13311134111111111311111111131 11111113 1111 11231 1111111 749513800432206328353370732175609342000769025791144192680117 100 90 80 70 60 50 40 * * 30 * ** 20 ** **** * ** ** * * * ** * * * * * *** ** * ** ** ** * 10 ############################################################ 0......3....5.1....2........5..2.....3.......1...4. 0 5 0 5 0 5 0 5 0 5 CPU% per minute (last 60 minutes) * = maximum CPU% # = average CPU% 1 440 030 100 * 90 * 80 * 70 * 60 * 50 * 40 *** 30 *** 20 *** 10 ### 0.5..

and would use the fol​ low​ ing REST call: http[s]://apic_ip/api/node/class/procSysMem1d. proc:Sys​ Mem1w A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 week sam​ pling in​ ter​ val. proc:Sys​ Mem1year A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 year sam​ pling in​ ter​ val. This class up​ dates every day.xml? . This class up​ dates every 5 min​ utes. This class up​ dates every day. This class up​ dates every 15 min​ utes. proc:Sys​ Mem1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 hour sam​ pling in​ ter​ val. ACME would like to mon​ it​ or mem​ ory over the last day. This class up​ dates every 10 sec​ onds. proc:Sys​ Mem1qtr A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 quar​ ter sam​ pling in​ ter​ val. re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ proc:Sys​ Mem1mo A class that rep​ ory in a 1 month sam​ pling in​ ter​ val. proc:Sys​ Mem5min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 5 minute sam​ pling in​ ter​ val. proc:Sys​ Mem1d A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 day sam​ pling in​ ter​ val.Monitoring 311 Monitoring Switch Memory Utilization There are sev​ eral meth​ ods to poll mem​ ory uti​ liza​ tion and trend it over dif​ fer​ ent pe​ ri​ ods of time. REST API Spine and Leaf switches mem​ ory uti​ liza​ tion can be mon​ it​ ored using the fol​ low​ ing classes. proc:Sys​ Mem15min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 15 minute sam​ pling in​ ter​ val. This class up​ dates every day. The fol​ low​ ing sec​ tions de​ scribe a few of the meth​ ods avail​ able. This class up​ dates every hour. This class up​ dates every day. based on the de​ sired timescale and gran​ ul​ ar​ ity.

boot​ flash and logflash Leaf-1# df df: `/nxos/tmp': No such file or directory df: `/var/home': No such file or directory df: `/var/tmp': No such file or directory df: `/nginx': No such file or directory df: `/debugfs': No such file or directory df: `/recovery': No such file or directory df: `/cfg0': No such file or directory df: `/cfg1': No such file or directory df: `/logflash/INXOS_SYSMGR/startup-cfg': No such file or directory df: `/mnt/plog': No such file or directory Filesystem 1K-blocks Used Available Use% Mounted on . Memory usage: 5 minutes: 1.4% idle 9054020K used.14 2. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive. the CLI is only way to mon​ it​ or the uti​ liza​ tion of the file sys​ tem on the leaves. 2 running CPU states : 4.80 93.312 Monitoring CLI Leaf-1# show system resources Load average: 1 minute: 1. 15 minutes: 0. See http://​ www. The crit​ ic ​al vol​ umes to keep track of in terms of uti​ liza​ tion are /volatile.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mibsupport. 7347052K free Current memory status: OK SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.​ html Monitoring File System Health CLI Cur​ rently. 16401072K total.​ cisco.5% kernel.21 Processes : 513 total.x.1% user. It is nor​ mal to see a higher % uti​ liza​ tion on some of the mount points in the file sys​ tem hi​ er​ ar​ chy.

old.Monitoring 313 rootfs 512000 1064 510936 1% / rootfs 512000 1064 510936 1% / none 512000 1064 510936 1% /isan none 512000 1064 510936 1% /var none 51200 2288 48912 5% /etc none 51200 108 51092 1% /var/log none 3145728 336664 2809064 11% /dev/shm 512000 0 512000 0% /volatile 7782036 1080636 6306088 15% /bootflash none /dev/sda4 /dev/sda5 60485 5356 52006 10% /mnt/cfg/0 /dev/sda6 60485 5356 52006 10% /mnt/cfg/1 /dev/sda3 120979 13349 101384 /dev/sda7 512000 1064 510936 /dev/sda9 15748508 591216 14357292 4% /mnt/ifc/cfg /dev/sda8 15748508 991204 13957304 7% /mnt/ifc/log /dev/sda8 15748508 991204 13957304 7% /var/log/dme/oldlog /dev/sda9 15748508 591216 14357292 716800 665728 51072 93% /mnt/ifc/cfg/mgmt/opt/controller/sbin 93% /controller/sbin rootfs rootfs /dev/sda8 rootfs /dev/sda4 rootfs /dev/sda9 rootfs rootfs 716800 665728 51072 15748508 991204 13957304 716800 665728 51072 7782036 1080636 6306088 12% /mnt/pss 1% /logflash 4% /controller 7% /data/techsupport 93% /bin 15% /bootflash 716800 665728 51072 15748508 591216 14357292 93% /data/challenge.plugin 716800 665728 51072 93% /controller/sbin 93% /dev 4% /controller 716800 665728 51072 none 3145728 336664 2809064 none 51200 2288 48912 none 2097152 682360 1414792 33% /isan/plugin/0/isan/utils none 2097152 682360 1414792 33% /isan/plugin/0/lc/isan/utils none 2097152 682360 1414792 33% /isan/plugin/0/lc/isan/lib none 2097152 682360 1414792 33% /isan/plugin/0/isan/lib none 2097152 682360 1414792 33% /isan/lib none 2097152 682360 1414792 33% /isan/plugin/0/lib none 2097152 682360 1414792 33% /isan/utils rootfs 716800 665728 51072 93% /lc/isan/utils rootfs 716800 665728 51072 93% /lib rootfs 716800 665728 51072 93% /mnt/cfg 60485 5356 52006 10% /mnt/cfg/0 /dev/sda5 11% /dev/shm 5% /etc .

To show the CoPP pol​ icy that is pro​ gramed by the sys​ tem use the fol​ low​ ing CLI com​ mand: Leaf-1# show copp policy COPP Class COPP proto COPP Rate COPP Burst . CoPP sta​ tis​ tics are avail​ able through the CLI.314 Monitoring /dev/sda6 60485 5356 52006 10% /mnt/cfg/1 716800 665728 51072 93% /mnt/ifc 15748508 591216 14357292 716800 665728 51072 /dev/sda8 15748508 991204 13957304 /dev/sda3 120979 13349 101384 rootfs /dev/sda9 rootfs rootfs /dev/sda8 none rootfs none 4% /mnt/ifc/cfg 93% /mnt/ifc/cfg/mgmt/opt/controller/sbin 7% /mnt/ifc/log 12% /mnt/pss 716800 665728 51072 15748508 991204 13957304 93% /sbin 1572864 39444 1533420 3% /tmp 716800 665728 51072 93% /usr 7% /data/techsupport 51200 108 51092 15748508 991204 13957304 51200 108 51092 rootfs 716800 665728 51072 93% /var/log/dme rootfs 716800 665728 51072 93% /var/log/dme/nginx rootfs 716800 665728 51072 93% /usr/share/vim /dev/sda7 11811760 375608 10836140 4% /var/log/dme/log /dev/sda8 15748508 991204 13957304 7% /var/log/dme/oldlog none 512000 1064 510936 1% /var/run/mgmt/log none 512000 1064 510936 1% /var/run/utmp 11811760 375608 10836140 40960 8 40952 11811760 375608 10836140 rootfs 716800 665728 51072 none 512000 0 512000 /dev/sda8 none /dev/sda7 none /dev/sda7 1% /var/log 7% /var/log/dme/oldlog 1% /var/log/messages 4% /var/sysmgr 1% /var/sysmgr/startup-cfg 4% /logflash/core 93% /usb 0% /volatile Monitoring CoPP (Control Plane Policing) Statistics CLI CoPP is en​ abled by de​ fault and the pa​ ra​ me​ ters can​ not be changed at this time.

use the fol​ low​ ing CLI com​ mand: Leaf-1# show copp policy stats COPP Class COPP proto COPP Rate COPP Burst AllowPkts AllowBytes DropPkts DropBytes mcp mcp 500 500 0 0 0 0 ifc ifc 5000 5000 195072 161613961 0 0 igmp igmp 1500 1500 3 192 0 0 nd nd 1000 1000 6 564 0 0 cdp cdp 1000 1000 494 140543 0 0 pim pim 500 500 0 0 0 0 dhcp dhcp 1360 340 4 1400 0 0 lacp lacp 1000 1000 0 0 0 0 ospf ospf 2000 2000 0 0 0 0 arp arp 1360 340 1284 90068 0 0 lldp lldp 1000 1000 5029 1717208 0 0 acllog acllog 500 500 0 0 0 0 stp stp 1000 1000 0 0 0 0 eigrp eigrp 2000 2000 0 0 0 0 coop coop 5000 5000 4722 470546 0 0 .Monitoring 315 mcp mcp 500 500 ifc ifc 5000 5000 igmp igmp 1500 1500 nd nd 1000 1000 cdp cdp 1000 1000 pim pim 500 500 dhcp dhcp 1360 340 lacp lacp 1000 1000 ospf ospf 2000 2000 arp arp 1360 340 lldp lldp 1000 1000 acllog acllog 500 500 stp stp 1000 1000 eigrp eigrp 2000 2000 coop coop 5000 5000 traceroute traceroute 500 500 isis isis 1500 5000 icmp icmp 500 500 bgp bgp 5000 5000 To show drops against spe​ cific CoPP classes.

316 Monitoring traceroute traceroute 500 500 0 0 0 0 isis isis 1500 5000 17141 2167565 0 0 icmp icmp 500 500 0 0 0 0 bgp bgp 5000 5000 864 73410 0 0 Physical Interface Statistics and Link State GUI To ac​ cess in​ ter​ face link state in​ for​ ma​ tion. nav​ i​ gate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > In​ ter​ faces > Phys​ i​ cal In​ ter​ faces. loop​ backs. Physical interface throughput statistics Note that click​ ing on the check icon en​ ables you to se​ lect ad​ di​ tional sta​ tis​ tics that can be graphed sim​ i​ lar to the fol​ low​ ing screen​ shot. the oper state col​ umn dis​ plays the op​ er​ a​ tional state of the link. vir​ tual port-chan​ nels. nav​ i​ gate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > In​ ter​ faces > Phys​ i​ cal in​ ter​ faces > Eth X/Y. Note that there are other tabs avail​ able in the work pane that ref​ er​ ence other types of in​ ter​ faces like port-chan​ nels. in the APIC GUI. in the APIC GUI. routed in​ ter​ faces. In the work pane. etc. To ac​ cess in​ ter​ face sta​ tis​ tics. and then click on the Stats tab in the work pane on the right-hand side. .

json . the par​ ent man​ aged ob​ ject is pro​ vided below. etc. 5 minute rate. input/out​ put / du​ plex. mul​ ti​ cast pack​ ets. 30 sec​ ond rates.g. As a pointer. in​ for​ ma​ tion pro​ vided in pre​ ced​ ing sec​ tions and the tools de​ scribed therein An ex​ am​ ple of the base API call for phys​ i​ cal in​ ter​ face sta​ tis​ tics is: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/phys-[eth1/1]. sev​ eral ob​ jects are avail​ able. There are sev​ eral such coun​ ters that are avail​ able (e. It is ex​ pected that the reader has a good un​ der​ stand​ ing of the ob​ ject model and is able to nav​ i​ gate through the model to ob​ tain the in​ for​ ma​ tion de​ siered using the ex​ am​ ple below.Monitoring 317 Granular physical interface statistics REST API For cus​ tomers that pre​ fer the REST API in​ ter​ face to poll for in​ ter​ face sta​ tis​ tics. RX/TX. uni​ cast pack​ ets.). as the chil​ dren can be de​ rived from it.

the ACME op​ er​ at​ or could issue the fol​ low​ ing API call: /topology/pod-1/node-101/sys/phys-[eth1/1].318 Monitoring For ex​ am​ ple. txload 1/255. 1000 Mb/s. address: 7c69. From the prior com​ mand.f60f. Dedicated Interface Hardware: 1000/10000/auto Ethernet. output flow-control is off Auto-mdix is turned off Rate mode is dedicated Switchport monitor is off EtherType is 0x8100 EEE (efficient-ethernet) : n/a Last link flapped 04:19:13 Last clearing of "show interface" counters never 1 interface resets 30 seconds input rate 169328 bits/sec.8771 (bia 7c69.8771) MTU 9000 bytes. medium is broadcast Port mode is trunk full-duplex. such as ingress and egress bytes.json Vi​ sore al​ lows the op​ er​ at​ or to dig deeper into the hi​ er​ ar​ chi​ cal tree. BW 1000000 Kbit. media type is 1G Beacon is turned off Auto-Negotiation is turned on Input flow-control is off. 97 packets/sec 30 seconds output rate 424528 bits/sec. One of the child ob​ jects in​ cludes the fol​ low​ ing: topology/pod-1/node-101/sys/phys-[eth1/1]/dbgEtherStats&#10 CLI The show in​ ter​ face eth x/y com​ mand can be used to mon​ it​ or in​ ter​ faces from the CLI. DLY 1 usec reliability 255/255. to de​ ter​ mine the total ingress bytes on Leaf 101 port Eth1/1. rxload 1/255 Encapsulation ARPA. the op​ er​ at​ or could see chil​ dren of the in​ ter​ face ob​ ject. Other sup​ ported com​ mands in​ clude "show in​ ter​ face port-chan​ nel x/y" Leaf-1# show int e1/1 Ethernet1/1 is up admin state is up.f60f. 115 packets/sec .

and the ALE or ALE2 (Ap​ pli​ ca​ tion Leaf En​ gine ASIC) de​ pend​ ing on the gen​ er​ at​ ion of switch hard​ ware. From a for​ ward​ ing per​ spec​ tive.Monitoring 319 Load-Interval #2: 5 minute (300 seconds) input rate 644416 bps.x. 134 pps. The fol​ low​ ing meth​ ods can be used to de​ ter​ mine the sta​ tus of the mod​ ules in the switch. the NFE (Net​ work For​ ward​ ing En​ gine ASIC) which pro​ vide the front panel ports. there are two data plane com​ po​ nents. 114 pps RX 2474537 unicast packets 8434 multicast packets 2482973 input packets 0 jumbo packets 0 runts 0 storm suppression bytes 0 giants 0 input error 0 watchdog 2 broadcast packets 1686129815 bytes 0 CRC 0 no buffer 0 short frame 0 bad etype drop 0 input with dribble 0 overrun 0 underrun 0 bad proto drop 0 ignored 0 if down drop 712 input discard 0 Rx pause TX 1673907 unicast packets 1674489 output packets 575 multicast packets 7 broadcast packets 455539518 bytes 0 jumbo packets 0 output error 0 collision 0 deferred 0 lost carrier 0 no carrier 0 babble 0 late collision 0 output discard 0 Tx pause SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. which pro​ vides up​ link con​ nec​ tiv​ ity to the spines. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for in​ ter​ faces See: http://​ www. .​ html Module Status Even though the leaves are con​ sid​ ered fixed switches.​ cisco. output rate 365544 bps. they have a su​ per​ vi​ sor com​ po​ nent which refers to the CPU com​ plex.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. viz.

-----------------. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Mod​ ule > Line mod​ ules and the sta​ tus of the mod​ ule is dis​ played in the work pane.152) 0. To ac​ cess mod​ ule sta​ tus for the ALE/ALE2. Leaf-1# show module Mod Ports Module-Type --- ----- ----------------------------------.1(0. REST API The fol​ low​ ing REST API call(s) can be used to mon​ it​ or the state of the su​ per​ vi​ sor and the mod​ ule.---------- Model Status 1 48 1/10G Ethernet Module N9K-C9396PX active GEM 12 40G Ethernet Expansion Module N9K-M12PQ ok Mod Sw Hw --- -------------- ------ 1 11. in the APIC GUI. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/supslot-1/sup https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/lcslot-1/lc CLI The show mod​ ule com​ mand can be used to ob​ tain the sta​ tus of the base mod​ ule and the up​ link mod​ ule.320 Monitoring GUI To ac​ cess mod​ ule sta​ tus for the NFE and the CPU com​ plex. in the APIC GUI. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Mod​ ule > Su​ per​ vi​ sor mod​ ules and the sta​ tus of the mod​ ule is dis​ played in the work pane.2050 Mod MAC-Address(es) Serial-Num --- -------------------------------------- ---------- 1 7c-69-f6-0f-87-71 to 7c-69-f6-0f-87-ad SAL17267Z9U .

​ cisco. GUI To ac​ cess fan sta​ tus for the leaf switch. Leaf-1# show environment fan Fan: . the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for mod​ ules. in the APIC GUI.Monitoring 321 Mod Online Diag Status --- ------------------ 1 pass SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.x. See: http://​ www. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Fan Tray and the sta​ tus of the mod​ ules is dis​ played in the work pane. REST API The fol​ low​ ing REST API call(s) and their child ob​ jects can be used to mon​ it​ or the state of the fans on a leaf switch (note that there are 3 slots on this par​ tic​ ul​ ar switch).​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-1 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-2 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-3 CLI The fol​ low​ ing CLI's can be used to mon​ it​ or the state of the fans on a leaf switch.​ html Switch Fan Status The fol​ low​ ing sec​ tion de​ scribes method​ olo​ gies to re​ trieve the sta​ tus of the fan trays on the leaf switches.

​ cisco. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for fan trays (http://​ www. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/psuslot-1 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/psuslot-2 . in the APIC GUI. REST API The fol​ low​ ing REST API call(s) and their child ob​ jects can be used to mon​ it​ or the state of the fans on a leaf switch (note that there are 3 slots on this par​ tic​ ul​ ar switch). Power Supply Status The fol​ low​ ing sec​ tions de​ scribe method​ olo​ gies to re​ trieve the sta​ tus of the power sup​ plies on the leaf switches GUI To ac​ cess power sup​ ply sta​ tus for the leaf switch. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Power Sup​ ply Units and the sta​ tus of the mod​ ules is dis​ played in the work pane.322 Monitoring -----------------------------------------------------Fan Model Hw Status -----------------------------------------------------Fan1(sys_fan1) N9K-C9300-FAN1-B -- ok Fan2(sys_fan2) N9K-C9300-FAN1-B -- ok Fan3(sys_fan3) N9K-C9300-FAN1-B -- ok Fan_in_PS1 -- -- unknown Fan_in_PS2 -- -- ok Fan Speed: Zone 1: 0x5f Fan Air Filter : Absent SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.​ html).x.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.

.x. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for power sup​ plies.Monitoring 323 CLI The fol​ low​ ing CLI com​ mands can be used to mon​ it​ or the state of the fans on a leaf switch: Leaf-1# show environment power Power Supply: Voltage: 12.Per module power not available Power Usage Summary: -------------------Power Supply redundancy mode (configured) Non-Redundant(combined) Power Supply redundancy mode (operational) Non-Redundant(combined) Total Power Capacity (based on configured mode) 648 W Total Power of all Inputs (cumulative) 648 W Total Power Output (actual draw) 168 W Total Power Allocated (budget) N/A Total Power Available for additional modules N/A SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.0 Volts Power Supply Model Actual Total Output Capacity (Watts ) (Watts ) Status ------- ------------------- ----------- ----------- 1 UCSC-PSU-650W 0 W 648 W shut 2 UCSC-PSU-650W 168 W 648 W ok Actual Power Draw Allocated Module Model -------------- Status (Watts ) (Watts ) ----------- ----------- 168 W 456 W Powered-Up -------- ------------------- 1 N9K-C9396PX -------------- fan1 N9K-C9300-FAN1-B N/A N/A Powered-Up fan2 N9K-C9300-FAN1-B N/A N/A Powered-Up fan3 N9K-C9300-FAN1-B N/A N/A Powered-Up N/A .

To ob​ tain a list of LLDP neigh​ bors on an in​ ter​ face.​ html LLDP Neighbor Status The APIC pro​ vides a sin​ gle pane of glass to query and de​ ter​ mine all LLDP neigh​ bors in a fab​ ric.​ cisco. (O) Other Device ID Local Intf Hold-time Capability Port ID apic2 Eth1/1 120 apic3 Eth1/4 120 4c:4e:35:09:77:2f 5548-2 Eth1/7 120 B Eth1/3 Spine-1 Eth1/49 120 BR Eth4/1 Spine-2 Eth1/50 120 BR Eth4/1 Spine-3 Eth1/51 120 BR Eth1/3 Spine-4 Eth1/52 120 BR Eth1/3 Spine-5 Eth1/53 120 BR Eth1/5 90:e2:ba:4b:fa:d4 . (B) Bridge. (P) Repeater. REST API The fol​ low​ ing rest API call can be used to ob​ tain the same in​ for​ ma​ tion: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/lldp/inst/if-[eth1/1] CLI Leaf-1# show lldp neighbors Capability codes: (R) Router. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Pro​ to​ cols > LLDP > Neigh​ bors > eth x/y A full list​ ing of all LLDP neigh​ bors on the in​ ter​ face can be ob​ tained in the work pane.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. (T) Telephone. (C) DOCSIS Cable Device (W) WLAN Access Point.324 Monitoring See: http://​ www. In the above work​ flow click​ ing on Neigh​ bors (in​ stead of eth x/y) gives you a list of all LLDP neigh​ bors on the switch. (S) Station.

IGMP. V . See: http://​ www. In the above work​ flow click​ ing on Neigh​ bors (in​ stead of eth x/y) gives you a list of all LLDP neigh​ bors on the switch.Repeater. I . CDP Neigh​ bor Sta​ tus: GUI To ob​ tain a list of CDP neigh​ bors on an in​ ter​ face.VoIP-Phone.x.Supports-STP-Dispute Device-ID Local Intrfce Hldtme Capability Platform Port ID . B .​ cisco.Remotely-Managed-Device. T . r . s . the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for LLDP. H .Trans-Bridge. D .Switch.Host.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.Source-Route-Bridge S .Monitoring 325 Spine-6 Eth1/54 120 BR Eth1/5 Total entries displayed: 9 SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. REST API The fol​ low​ ing rest API call can be used to ob​ tain the same in​ for​ ma​ tion: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/cdp/inst/if-[eth1/1] CLI Leaf-1# show cdp neighbors Capability Codes: R . nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod1 > Leaf-X > Pro​ to​ cols > CDP > Neigh​ bors > eth x/y A full list​ ing of all CDP neigh​ bors on the in​ ter​ face can be ob​ tained in the work pane.​ html APIC pro​ vides a sin​ gle pane of glass to query and de​ ter​ mine all CDP neigh​ bors in a fab​ ric.Router.

To view the same for mod​ ules. click on Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-1 > Chas​ sis > Line Mod​ ules > Slot-x. as well as the abil​ ity to run on de​ mand di​ ag​ nos​ tics to iso​ late po​ ten​ tial hard​ ware at fault. .​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. GOLD di​ ag​ nos​ tics pro​ vide an easy and quick way for op​ er​ at​ ions teams to con​ firm that bootup and non-dis​ rup​ tive tests that run dur​ ing nor​ mal op​ er​ at​ ions have ex​ e​ cuted prop​ erly.326 Monitoring Services-UCS-A(SSI15450J63) Eth1/5 129 S I s UCS-FI-6248UP Eth1/17 Eth1/7 123 S I s N5K-C5548UP Eth1/3 5548-2(SSI154300VL) SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.​ cisco. Then click trou​ bleshoot​ ing in the work pane.x. See: http://​ www. Then click Trou​ bleshoot​ ing in the work pane. click on Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-1 > Chas​ sis > Su​ per​ vi​ sor Mod​ ules > Slot-1. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for CDP.​ html GOLD Diagnostic Results GOLD is cov​ ered in greater de​ tail in the Hard​ ware Re​ place​ ment sec​ tion. GUI To view GOLD Di​ ag​ nos​ tic test re​ sults in the GUI for the Su​ per​ vi​ sors.

. . 26) ge-eeprom----------------------> . 4) nsa-mem------------------------> . 40) rtc-test-----------------------> . 6) fabp-prbs: Port 1 2 3 4 5 6 7 8 9 10 11 12 ----------------------------------------. 31) obfl-acc-----------------------> . 25) act2-acc-----------------------> . 32) nvram-cksum--------------------> . I = Incomplete. 34) asic-scratch-------------------> . . 2) mgmtp-lb-----------------------> . . . 23) mem-health---------------------> . . . = Pass. 24) ssd-acc------------------------> . E = Error disabled) 1) bios-mem-----------------------> . . F = Fail. 41) pcie-bus-----------------------> . 33) fpga-reg-chk-------------------> . . A = Abort. . 29) usb-bus------------------------> .Monitoring 327 CLI Leaf-1# show diagnostic result module all Current bootup diagnostic level: bypass Module 1: 1/10G Ethernet Module (Active) Test results: (. U = Untested. . 22) cpu-cache----------------------> . . 30) cons-dev-----------------------> .

.

Cre​ ate an in​ ter​ face mon​ it​ or​ ing pol​ icy. choose Monitoring Policies. Click the edit button next to Stats Type and select Ingress and Egress. e. b. ACME's server team has de​ ter​ mined that they would like to mon​ it​ or link uti​ liza​ tion over a 5 minute pe​ riod. choose Ingress. and when the av​ er​ age uti​ liza​ tion is above 80%. and select L1 Physical Interface Configuration. 4 In the Create Monitoring Policy dialog box. they would like to raise a fault for the af​ fected in​ ter​ face.Monitoring 329 Proactive Monitoring Use Cases Monitoring Workload Bandwidth ACME would like to proac​ tively mon​ it​ or con​ nec​ tions to servers to de​ ter​ mine whether ad​ e​ quate band​ width is avail​ able to a given work​ load. Next they will con​ fig​ ure a thresh​ old pol​ icy to gen​ er​ ate a fault when the link uti​ liza​ tion ex​ ceeds var​ io ​us thresh​ olds ac​ cord​ ing to the fol​ low​ ing table. Select 5 minutes from the granularity column. choose L1 Physical Interface Configuration. f. Expand the newly created monitoring policy and select Stats Collection Policy. Click on the edit button next to Monitoring Object. and click on update. 2 In the Navigation pane. To ac​ com​ plish these tasks re​ quires con​ fig​ ur​ ing a mon​ it​ or​ ing pol​ icy with two dis​ tinct types of poli​ cies. 1 On the menu bar. The fol​ low​ ing will pro​ vide an ex​ am​ ple of how sta​ tis​ tics poli​ cies and thresh​ olds can be used to alert the op​ er​ at​ ors when ad​ di​ tional band​ width is re​ quired to a server. . From the Monitoring Object drop-down list. ACME will con​ fig​ ure a stats col​ lec​ tion pol​ icy to col​ lect the av​ er​ age uti​ liza​ tion of a link over the de​ sired 5 minute in​ ter​ val. perform the following actions: a. From the stats type drop-down list. choose Fabric > Access Policies. d. choose Actions > Create Monitoring Policy. First. Click +. 3 In the Work pane. g. c. or mul​ ti​ ple in​ ter​ faces need to be bonded to pro​ vide more band​ width to the server. This en​ ables them to an​ swer ques​ tions about whether a server needs to be up​ graded from 10G to 40G.

From the Monitoring Policy drop down menu. and clear the warn​ ing when the uti​ liza​ tion falls below 75. Select UCS-10G-PG. and look for de​ vi​ at​ ions above or below that nor​ mal value. 4 Enter 0 for the normal value and click the Rising radio button. these poli​ cies can be fur​ ther cus​ tomized to pro​ vide a nor​ mal value. Next we will con​ fig​ ure a thresh​ old which will alert us if the uti​ liza​ tion ex​ ceeds 80% and as​ sign an de​ fault fault sever​ ity to it. a. 1 Click + in the Config Thresholds column. For ex​ am​ ple. 80% uti​ liza​ tion does not nec​ es​ sar​ ily mean that the ap​ pli​ ca​ tion per​ for​ mance is de​ graded. Re​ peat these steps for the egress sta​ tis​ tics as well. and the reset col​ umn will spec​ ify the the level at which the fault will be cleared. we will apply the pol​ icy to the UCS-10G-PG 1 2 On the menu bar. In . so we will flag this as a warn​ ing. Ad​ di​ tional lev​ els/sever​ it​ ies can also be spec​ if​ ied if de​ sired. we will be rais​ ing a warn​ ing when the uti​ liza​ tion goes above 80. we will as​ so​ ci​ ate the newly cre​ ated pol​ icy with an in​ ter​ face pol​ icy group that rep​ re​ sents the in​ ter​ faces we to mon​ it​ or with this pol​ icy. For our ex​ am​ ple. 2 In the Thresholds for Collection 5 minute window. select the newly created monitoring policy. choose Interface Policies > Policy Groups. click +. choose Fabric > Access Policies. we are in​ ter​ ested in an ab​ solute per​ cent​ age. As an ex​ am​ ple. In this sce​ nario. we will mon​ it​ or the amount of traf​ fic to the web tier of a given ap​ pli​ ca​ tion.330 Monitoring We have now cre​ ated a pol​ icy which will mon​ it​ or as​ so​ ci​ ated in​ ter​ faces and ingress traf​ fic rate at 5 minute in​ ter​ vals. 3 Click on Ingress Link utilization average value. such as the ag​ gre​ gate amount of traf​ fic to a spe​ cific tier. The set col​ umn spec​ if​ ies the level at which the fault will be raised. Note: In this ex​ am​ ple. In the Work pane. Fi​ nally. EPG Level Statistics The ap​ pli​ ca​ tion owner would like to be able to mon​ it​ or net​ work-re​ lated in​ for​ ma​ tion for their ap​ pli​ ca​ tion. b.

Monitoring 331

this ex​
am​
ple, the de​
fault mon​
i​
tor​
ing poli​
cies are ap​
pro​
pri​
ate, and they are sim​
ply ex​
tract​
ing them from the sys​
tem to be con​
sumed ex​
ter​
nally. This in​
for​
ma​
tion is use​
ful in
sce​
nar​
ios such as a new re​
lease being pushed, and to make sure that no traf​
fic anom​
alies are cre​
ated after the push.
This can be ac​
com​
plished by nav​
i​
gat​
ing to the EPG, and se​
lect​
ing the Stats tab:

EPG-level throughput statistics
Ad​
di​
tion​
ally, this in​
for​
ma​
tion can be gath​
ered from the API:
http[s]://apic_ip/api/node/mo/uni/tn-mynewproject/ap-app1/epg-web-epg.xml?querytarget=self&rsp-subtree-include=stats

Monitoring 333

Reactive Monitoring
It is cru​
cial that the ACME op​
er​
a​
tional staff are able to react to any in​
di​
ca​
tion of some​
thing going wrong. If there is a no​
ti​
fi​
ca​
tion that some​
thing has gone wrong, such as a
fault no​
ti​
fi​
ca​
tion, a low health score, or a ticket/re​
port that end-user func​
tion​
al​
ity has
been im​
pacted, knowl​
edge of the avail​
able mon​
i​
tor​
ing tools is im​
por​
tant for the iden​
ti​
fi​
ca​
tion and col​
lec​
tion of ev​
i​
dence. This ev​
i​
dence can then be used to iden​
tify and an​
a​
lyze the root cause of the prob​
lem be​
fore tak​
ing cor​
rec​
tive ac​
tion. For more in​
for​
ma​
tion
re​
gard​
ing faults and health scores please refer to those spe​
cific sec​
tions within this book.
A deep dive into the processes of trou​
bleshoot​
ing is out of the scope of this book.
bleshoot​
ing Cisco Ap​
pli​
ca​
tion Cen​
tric In​
fra​
struc​
ture: An​
a​
lyt​
i​
cal
Please refer to "Trou​
lem solv​
ing ap​
plied to the pol​
icy dri​
ven data cen​
ter" avail​
able at: http://​
datacenter.​
prob​
github.​
io/​
aci-troubleshooting-book/​

Tenant—Troubleshoot Policies
Within the APIC GUI, under each Ten​
ant you can find a Trou​
bleshoot Pol​
icy sec​
tion.
This sec​
tion will allow con​
fig​
ur​
a​
tion of poli​
cies that are spe​
cific to one ten​
ant, and the
mon​
it​
or​
ing of traf​
fic and test con​
nec​
tiv​
ity be​
tween end​
points.
As seen in the image above, the fol​
low​
ing trou​
bleshoot​
ing poli​
cies can be con​
fig​
ured:

SPAN (Switched Port ANalyzer)—Configuration of SPAN and ERSPAN sources
and destinations to be used in external monitoring of Tenant traffic flows

Endpoint-To-Endpoint Traceroute—Configuration of a path validation tool for
verifying validity of communications between Tenant endpoints in an ACI fabric

Atomic Counters—Configuration of a set of customizable counters to collect
and report on information between a definable set of objects. As shown in the
image below, a policy can be configured to collect statistics between EPs,
between EPGs, between EPGs and specific IPs and other special objects, such
as Any or External ​
traffic flows

Fabric—Troubleshoot Policies
For trou​
bleshoot​
ing within the en​
tire fab​
ric, there are the fol​
low​
ing tools and poli​
cies:

334 Monitoring

SPAN (Switched Port Analyzer)—Configuration of SPAN and ERSPAN sources

On-demand Diagnostics—Configuration of a policy for collection of diagnostic

and destinations to be used in external monitoring of fabric traffic flows
information that can be executed at a point in time and which will return a set
of valuable output for investigation

Leaf Nodes Traceroute—Configuration of a path validation tool for verifying

Traffic Map—At-a-glance hotspot map of node-to-node traffic flow in an ACI

validity of communications between ACI fabric nodes
fabric

Enhanced Troubleshooting Wizard
From ver​
sion 1.1, APIC pro​
vides a trou​
bleshoot​
ing graphic tool to find rel​
e​
vant faults
and sta​
tis​
tics, re​
cent changes, and run con​
nec​
tiv​
ity tests in a sim​
ple man​
ner.
It also gives the op​
tion to gen​
er​
ate a re​
port to save the re​
sults so it can be used as a
ref​
er​
ence.

Other Tools

iPing—A troubleshooting tool in the ACI fabric that can be used to verify
reachability of a device connected to the fabric utilizing the fabric as the
pervasive source

Audit Logs—Audit logs are continually collected on all actions taken in an ACI
fabric and can give a quick indication of which user took which actions at what
time

Reactive Monitoring Use Cases
At the end of this chap​
ter, we are going to de​
scribe two sit​
ua
​t​
ions where ACME ran
into is​
sues and how they made use of the tools pre​
vi​
ously de​
scribed.
1

No access to application: An end-user calls and reports that they can no longer
access a web application running within the fabric.

2

Users report that an application running in the fabric is slow or there is a report
of slowness to the web application running within the fabric.

Monitoring 335

Reactive Monitoring Tools
Switch Port Analyzer (SPAN)
SPAN is gen​
er​
ally used in two ways:

Proactively as part of a third party or offline analysis requirement.
Security tools (IDS/IPS, Data Loss Prevention)
Call Recording

Troubleshooting application and networking issues within the fabric.

It may be help​
ful to per​
form a cap​
ture of some par​
tic​
ul​
ar traf​
fic to see what is going on
within the stream of data. Look​
ing through traf​
fic flows will allow in​
ves​
ti​
ga​
tion of
packet and pro​
to​
col level de​
tails, such as traf​
fic re​
sets, mis​
be​
hav​
ing pro​
to​
cols, im​
proper host re​
quests or re​
sponses, or node level com​
mu​
ni​
ca​
tions. This will pro​
vide deeper in​
sight into how de​
vices are using the net​
work than sim​
ple traf​
fic flow and
fab​
ric con​
fig​
ur​
a​
tion re​
view.
Switched Port AN​
al​
yzer, or SPAN, is a stan​
dard fea​
ture that al​
lows copy and repli​
ca​
tion
of traf​
fic to a net​
work an​
al​
yzer for fur​
ther de​
cod​
ing and in​
ves​
ti​
ga​
tion. It can be used to
copy traf​
fic from one or more ports, VLANs, or end​
point groups (EPGs).
The SPAN fea​
ture process is non-dis​
rup​
tive to any con​
nected de​
vices and is fa​
cil​
it​
ated
in hard​
ware, which pre​
vents any un​
nec​
es​
sary CPU load.
SPAN ses​
sions can be con​
fig​
ured to mon​
it​
or traf​
fic re​
ceived by the source (ingress
traf​
fic), traf​
fic trans​
mit​
ted from the source (egress traf​
fic), or both. By de​
fault, SPAN
mon​
it​
ors all traf​
fic, but fil​
ters can be con​
fig​
ured to mon​
it​
or only se​
lected traf​
fic.

Multinode SPAN
APIC traf​
fic mon​
it​
or​
ing poli​
cies can en​
force SPAN at the ap​
pro​
pri​
ate places to copy
traf​
fic from mem​
bers of each End Point Group wher​
ever they are con​
nected. If a mem​
ber moves, APIC au​
to​
mat​
ic
​ally pushes the pol​
icy to the new leaf switch. For ex​
am​
ple,
when a VMo​
tion event re​
lo​
cates an End​
point to a new leaf switch, the SPAN fea​
ture
con​
fig​
ur​
a​
tion au​
to​
mat​
ic
​ally ad​
justs.

336 Monitoring

SPAN Guidelines and Restrictions

Use SPAN for troubleshooting. SPAN traffic competes with user traffic for
switch resources. To minimize the load, configure SPAN to copy only the
specific traffic that you want to analyze.

An l3extLIfP Layer 3 subinterface cannot be configured as a SPAN source. The
entire port must be used for monitoring traffic from external sources.

Tenant and access SPAN use the encapsulated remote extension of SPAN
(ERSPAN) type I, while fabric SPAN uses ERSPAN type II. For information
regarding ERSPAN headers, refer to the IETF Internet Draft at this URL: https:/​
/​
tools.ietf.org/​
html/​
draft-foschiano-erspan-00.

See the Verified Scalability Guide for Cisco ACI document for SPAN-related
limits, such as the maximum number of active SPAN sessions.

Configuring a SPAN Session
This pro​
ce​
dure shows how to con​
fig​
ure a SPAN pol​
icy to for​
ward repli​
cated source
pack​
ets to a re​
mote traf​
fic an​
al​
yzer.
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the tenant.

3

In the Navigation pane, choose Tenant_Name > Troubleshooting Policies >
SPAN > SPAN Destination Groups.

4

In the Work pane, choose Actions > Create SPAN Destination Group.

5

In the Create SPAN Destination Group dialog box, perform the following
actions:
a. In the Name field, enter a name for the SPAN destination group.
b. In the Destination EPG dropdowns, select the destination Tenant,
Application Profile, and EPG.
c. Enter the Destination IP.
d. Enter the Source IP Prefix.
e. Optionally, modify the other fields as needed.
f. Click OK.
g. If needed, add additional destinations.
h. Click Submit.

Monitoring 337

6

Under SPAN, right-click SPAN Source Groups and choose Create SPAN Source
Group.

7

In the Create SPAN Source Group dialog box, perform the following actions:
a. In the Name field, enter a name for the SPAN source group.
b. From the Destination Group drop-down list, choose the SPAN destination
group that you configured previously.
c. In the Create Sources table, click the + icon to open the Create Sources
dialog box.
d. In the Name field, enter a name for the source.
e. In the Direction field, choose the radio button based on whether you want to
replicate and forward packets that are incoming to the source, outgoing from
the source, or both incoming and outgoing.
f. From the Source EPG drop-down list, choose the EPG (identified by
Tenant/ApplicationProfile/EPG) whose packets will be replicated and
forwarded to the SPAN destination. Click OK to save the SPAN source.
g. Click Submit to save the SPAN source group.

Traceroute
Tracer​
oute is a use​
ful fea​
ture in tra​
di​
tional net​
work​
ing. In ACI this fea​
ture is im​
ple​
mented tak​
ing into ac​
count the way the fab​
ric works.
Tracer​
oute sup​
ports a va​
ri​
ety of modes, in​
clud​
ing end​
point-to-end​
point, and leaf-toleaf (tun​
nel end​
point, or TEP to TEP). It dis​
cov​
ers all paths across the fab​
ric, dis​
cov​
ers
point of exits for ex​
ter​
nal end​
points, and helps to de​
tect if any path is blocked.
A tracer​
oute that is ini​
ti​
ated from the ten​
ant end​
points shows the de​
fault gate​
way as
an in​
ter​
me​
di​
ate hop that ap​
pears at the ingress leaf switch.
Note: If tracer​
oute is done from the OS of a con​
nected server or VM, it will show the
hops for the leaves and spines as un​
known, and will keep record​
ing the in​
for​
ma​
tion
after the packet gets out of the fab​
ric. For more pre​
cise in​
for​
ma​
tion, please use tracer​
oute from the APIC (GUI or CLI)

338 Monitoring

Traceroute Guidelines and Restrictions

When the traceroute source or destination is an endpoint, the endpoint must
be dynamic and not static. Unlike a dynamic endpoint (fv:CEp), a static endpoint
(fv:StCEp) does not have a child object (fv:RsCEpToPathEp) that is required for
traceroute.

See the Verified Scalability Guide for Cisco ACI document for traceroute-related
limits.

Traceroute results will display the IP address of the remote node and interface
which it came it came in on. (Browse on the APIC GUI to Fabric | Inventory |
Fabric Management to view the IP address information for correlation.)

Traceroute cannot be used for endpoints that reside in an external EPG.

Performing a Traceroute Between Endpoints
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the tenant.

3

In the Navigation pane, choose Tenant_Name > Troubleshooting Policies >
Endpoint-to-Endpoint Traceroute Policies.

4

In the Work pane, choose Actions > Create Endpoint-to-Endpoint Traceroute
Policy.

5

In the Create Endpoint-to-Endpoint Traceroute Policy dialog box, perform the
following actions:
a. In the Name field, enter a name for the traceroute policy.
b. In the Source End Points table, click the + icon to edit the traceroute source.
c. From the Source MAC drop-down list, choose or enter the MAC address of
the source endpoint and click Update.
d. In the Destination End Points table, click the + icon to edit the traceroute
destination.
e. From the Destination MAC drop-down list, choose or enter the MAC address
of the destination endpoint and click Update.
f. In the State field, click the Start radio button.
g. Click Submit to launch the traceroute.

6

In the Navigation pane or the Traceroute Policies table, click the traceroute
policy. The traceroute policy is displayed in the Work pane.

Monitoring 339

7

In the Work pane, click the Operational tab, click the Source End Points tab,
and click the Results tab.

8

In the Traceroute Results table, verify the path or paths that were used in the trace.
a. More than one path might have been traversed from the source node to the
destination node.
b. For readability, increase the width of one or more columns, such as the Name
column.

Atomic Counters
Atomic Coun​
ters are use​
ful for trou​
bleshoot​
ing con​
nec​
tiv​
ity be​
tween end​
points, EPGs,
or an ap​
pli​
ca​
tion within the fab​
ric. A user re​
port​
ing ap​
pli​
ca​
tion may be ex​
pe​
ri​
enc​
ing
slow​
ness, or atomic coun​
ters may be needed for mon​
it​
or​
ing any traf​
fic loss be​
tween
two end​
points. One ca​
pa​
bil​
ity pro​
vided by atomic coun​
ters is the abil​
ity to place a
trou​
ble ticket into a proac​
tive mon​
it​
or​
ing mode, for ex​
am​
ple when the prob​
lem is in​
ter​
mit​
tent, and not nec​
es​
sar​
ily hap​
pen​
ing at the time the op​
er​
at​
or is ac​
tively work​
ing
the ticket.
Atomic coun​
ters can help de​
tect packet loss in the fab​
ric and allow the quick iso​
la​
tion of the
source of con​
nec​
tiv​
ity is​
sues. Atomic coun​
ters re​
quire NTP to be en​
abled on the fab​
ric.
Leaf-to-leaf (TEP to TEP) atomic coun​
ters can pro​
vide the fol​
low​
ing:

Counts of drops, ad​
mits, and ex​
cess pack​
ets

Short-term data col​
lec​
tion such as the last 30 sec​
onds, and long-term data col​
lec​
tion such as 5 min​
utes, 15 min​
utes, or more

A break​
down of per-spine traf​
fic (avail​
able when the num​
ber of TEPs, leaf or

On​
go​
ing mon​
it​
or​
ing

VPC, is less than 64)

Leaf-to-leaf (TEP to TEP) atomic coun​
ters are cu​
mu​
la​
tive and can​
not be cleared. How​
ever, be​
cause 30 sec​
ond atomic coun​
ters reset at 30 sec​
ond in​
ter​
vals, they can be used
to iso​
late in​
ter​
mit​
tent or re​
cur​
ring prob​
lems.
Ten​
ant atomic coun​
ters can pro​
vide the fol​
low​
ing:

Application-specific counters for traffic across the fabric, including drops,
admits, and excess packets

5 6 In the Work pane. Choose or enter the identifying information for the traffic destination. choose Actions > Add Policy_Name Policy. Choose or enter the identifying information for the traffic source. external interfaces. 4 Choose a policy (traffic topology). choose Tenant_Name > Troubleshooting Policies > Atomic Counter Policy. endpoint group. or endpoint to endpoint IP address. b. • EPG to EPG with optional drill down • EPG to endpoint • EPG to * (any) • Endpoint to external IP address ***Atomic coun​ ters track the amount pack​ ets of be​ tween the two end​ points and use this as a mea​ sure​ ment. endpoint groups. Traffic can be measured between a combination of endpoints. perform the following actions: a. choose Tenants > ALL TENANTS. In the Add Policy dialog box. c. The required identifying information differs depending on the type of source (endpoint. 2 In the Work pane. Configuring Atomic Counters 1 On the menu bar. or IP address). . choose the tenant. They do not take into ac​ count drops or error coun​ ters in a hard​ ware level.*** Dropped pack​ ets are cal​ cu​ lated when there are less pack​ ets re​ ceived by the des​ ti​ na​ tion than trans​ mit​ ted by the source. enter a name for the policy. and IP addresses.340 Monitoring • Modes include the following: • Endpoint to endpoint MAC address. external interface. Ex​ cess pack​ ets are cal​ cu​ lated when there are more pack​ ets re​ ceived by the des​ ti​ na​ tion than trans​ mit​ ted by the source. Note that a single target endpoint could have multiple IP addresses associated with it. 3 In the Navigation pane. In the Name field.

choose the new atomic counter policy. choose Troubleshooting Policies > Traffic Map. In the resulting Create Atomic Counter Filter dialog box. Traffic Map Low per​ for​ mance and con​ ges​ tion can be iden​ ti​ fied by use of Traf​ fic maps. Configuring Traffic Map 1 On the menu bar. 2 In the Navigation pane. Traf​ fic maps make use of atomic coun​ ters to con​ tin​ u​ ously mon​ i​ tor and dis​ play traf​ fic be​ tween leaf switches to help with quick de​ bug​ ging and iso​ la​ tion of ap​ pli​ ca​ tion con​ nec​ tiv​ ity is​ sues. choose Fabric > Fabric Policies. Optional: (Optional) In the Filters table.Monitoring 341 d. The policy configuration is displayed in the Work pane. under the selected topology. . Click Submit to save the atomic counter policy. for example) and by source and destination IP port numbers. 7 In the Navigation pane. specify filtering by the IP protocol number (TCP=6. e. click + to specify filtering of the traffic to be counted. 8 In the Work pane. choose the Operational tab and choose the Traffic subtab to view the atomic counter statistics.

The En​ hanced Trou​ bleshoot​ ing Wiz​ ard pro​ vides a sin​ gle lo​ ca​ tion that in​ cludes sev​ eral com​ monly used tools and out​ puts re​ quired for trou​ bleshoot​ ing end point con​ nec​ tiv​ ity. Show relevant statistics for those EPs. Show any related recent changes in the path between EPs. received. dropped and excess • All spines and a specific spine switch The per​ cent​ age is shown in rel​ at​ ive terms to all traf​ fic by Source or re​ ceived by Des​ ti​ na​ tion. 3 In the next screen. and click search. Show the faults in the path between the selected EPs. b. c. 1 In the menu bar. enter a name for the session. c. In the Destination field. b. Run traceroute between EPs. Click Start. e. Select the Time Window duration for the debug. In the Source field. enter the MAC or IP address for the destination endpoint. enter the MAC or IP address for the source endpoint. you can click the following items for more information: a. 4 Clicking on a cell opens a table with all data for all trails and links. click the check box. d. and click search. f. . In the Name field. highlighting the affected component. the En​ hanced Trou​ bleshoot​ ing Wiz​ ard may be used to quickly iden​ tify con​ nec​ tiv​ ity is​ sues. Optional: Generate a Report to download the results. perform the following actions: a. Optional: If it is an external IP address. d. click the wrench icon to launch the enhanced troubleshooting wizard.342 Monitoring 3 Set the drop-down menu options to view the source Leaf to destination Leaf traffic paths. • Last interval and cumulative • Sent. Enhanced Troubleshooting Wizard When trou​ bleshoot​ ing con​ nec​ tiv​ ity be​ tween end​ points within the fab​ ric. 2 In the Create SPAN Destination Group dialog box.

256 ms 64 bytes from 10. IPing is a trou​ bleshoot​ ing tool for net​ work users sim​ i​ lar to the PING com​ mand. IPing IPing is used to test and val​ i​ date con​ nec​ tiv​ ity within the from leaf node to end​ points within the fab​ ric.0.154: icmp_seq=2 ttl=55 time=0. Show the configured contracts between EPs. tak​ ing into ac​ count the pri​ vate net​ work.254 ms 64 bytes from 10. Using IPing iping [ -V vrf ] [ -c count ] [ -i wait ] [ -p pattern ] [ -s packetsize ] [ -t timeout ] host Syntax Description Examples pod1-leaf1# iping -V overlay-1 10.0.59.154: icmp_seq=1 ttl=55 time=0.0.0.59.59.154 (10.154: icmp_seq=0 ttl=55 time=0.59.0.154): 56 data bytes 64 bytes from 10. Configure Atomic Counters between EPs. Configure SPAN between EPs g.59.245 ms . f.Monitoring 343 e.59.154 PING 10.0.

4 In the Work pane. choose Common.59. 2 In the Work pane.154 ping statistics --5 packets transmitted. This pro​ ce​ dure shows how to view ten​ ant events as an ex​ am​ ple. 5 packets received. .154: icmp_seq=4 ttl=55 time=0. choose the Events subtab to view the event log. Wher​ ever a His​ tory tab ap​ pears in the GUI Work pane. 3 In the Navigation pane. and a de​ scrip​ tion of the ac​ tion.00% packet loss round-trip min/avg/max = 0.154: icmp_seq=3 ttl=55 time=0. 5 Under the History tab. 7 Double-click a log entry to view additional details about the event.0. Audit logs can be found in sev​ eral places within the GUI.245/0. Audit logs also record when users logon and lo​ goff.10. choose the History tab.241 ms 64 bytes from 10. choose Common.59. An out​ age re​ ported on a host or ap​ pli​ ca​ tion in the fab​ ric may need to be tracked. choose Tenants > ALL TENANTS.0. Viewing Audit Logs Pro​ ce​ dure (ex​ am​ ple for view​ ing audit log on a ten​ ant) 1 On the menu bar. the audit log can be viewed.23 ms --. fil​ tered to show only those events rel​ e​ vant to the cur​ rent GUI con​ text.256 ms Audit Logs At times it may be re​ quired to view changes which have taken place in the fab​ ric.23/0.0.59. or data pulled for an audit re​ quire​ ment. Audit logs are records of who made a change.344 Monitoring 64 bytes from 10. choose the Audit Log subtab to view the audit log. 6 Under the History tab. when the change was made. 0.

These two End Points (EPs) be​ long to two dif​ fer​ ent End Point Groups (EPGs). Note: these ex​ am​ ples as​ sume that basic low-level in​ ves​ ti​ ga​ tion has been done and the issue has been iso​ lated to an issue with traf​ fic flows across the fab​ ric. there​ fore the de​ fault gate​ way for both EPs exist in the leaf switches. VMs are run​ ning. 2 Verify that the required contracts are in place between the EPs. Ca​ bles and con​ nec​ tiv​ ity have been ver​ if​ ied. The bridge do​ main has uni​ cast rout​ ing en​ abled. 3 In the Navigation pane. The fol​ low​ ing steps trou​ bleshoot this sit​ ua ​t​ ion: 1 Check that the EPs have been learned by the leaf switches. Loss of Connectivity to Endpoint The ACME ap​ pli​ ca​ tion owner has re​ ported that two servers have lost con​ nec​ tiv​ ity. and each EP is con​ nected to dif​ fer​ ent leaf switches. choose the Operational tab and verify that the endpoint is present. etc. bridge do​ main and Ten​ ant. Verify the required contracts are in place between the EPs . 4 In the Work pane. choose the tenant. 2 In the Work pane.Monitoring 345 Reactive Monitoring Use Cases This chap​ ter will show some ex​ am​ ples of how ACME's op​ er​ at​ ions teams can use their ACI mon​ it​ or​ ing tools to react to a few com​ mon pos​ si​ ble sce​ nar​ ios. Check that the EPs have been learned by the leaf switches 1 On the menu bar. within the same sub​ net. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name. mem​ ory and CPU uti​ liza​ tion has been checked. processes are run​ ning. choose Tenants > ALL TENANTS. hosts are up. 5 Repeat this procedure for the destination EPG.

One method could be to use endpoint-to-endpoint traceroute to show if there are paths available between those endpoints. If this test is successful. examine the troubleshooting policies. 10 Alternate techniques are available to validate communications between endpoints within the fabric. A good starting place is the Enhanced Troubleshooting Wizard. . 5 Choose the Contracts subtab. Another option inside the fabric could be to utilize the iPing tool to verify connectivity between the default gateway and the endpoints. choose Tenants > ALL TENANTS. This displays the filters that are present within that contract. 8 Inspect the contents of each filter by examining the contract under the Security Policies folder and verifying that each filter contains the appropriate filter entries. it then might be necessary to use SPAN to verify where the traffic is entering and leaving the fabric. In the event that these things seem valid. 6 Check for a relationship between the source EPG and destination EPG. a. To check the connectivity to the remote end.346 Monitoring Verify the required contracts are in place between the EPs 1 On the menu bar. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name. noting the direction of the arrows. use iPing from each leaf to the remote endpoint using the default gateway as source. choose the tenant. Each leaf has an SVI used as default gateway. the connectivity between endpoints and leaf switches is not the problem. and make sure frames have the right format. 4 In the Work pane. 9 If the endpoints are discovered within each EPG and the contract relationships look correct. 7 Click on the contract to verify the contents of the contract. choose the Operational tab. 3 In the Navigation pane. 2 In the Work pane. b.

. they can use EP-to-EP Tracer​ oute to show the path va​ lid​ ity. the end-user is try​ ing to ac​ cess the web por​ tal from a test VM. packet drops. Look​ ing through the Traf​ fic Map can also show an at-a-glance view in the fab​ ric and high​ light any pos​ si​ ble hotspots or con​ ges​ tion in cer​ tain el​ e​ ments of the fab​ ric. This per​ for​ mance issue could be caused due to la​ tency. SPAN may be used to ver​ ify where the traf​ fic is en​ ter​ ing and leav​ ing the fab​ ric and make sure frames have the right for​ mat.Monitoring 347 Users Report that an Application Running in the Fabric is Slow ACME test users re​ port slow re​ sponse of the web servers run​ ning in the fab​ ric. or in​ ter​ mit​ tent con​ nec​ tiv​ ity loss. Atomic coun​ ters can also be de​ ployed to check if there are any drops or ir​ reg​ ul​ ar​ it​ ies be​ tween the de​ fined de​ vices. Check spe​ cific coun​ ters for CRC er​ rors on the in​ ter​ faces used to trans​ mit EP spe​ cific traf​ fic. from the EPs on a OS level to the switches. If every​ thing in the fab​ ric looks fine. In this case. Cor​ rupted frames could cause drops in dif​ fer​ ent points. First the op​ er​ at​ ions staff should make sure end points are learned by the leaf switches in a con​ sis​ tent way so that the EPs are vis​ ib ​le in the op​ er​ at​ ional tab under the Ten​ ant-> Ap​ pli​ ca​ tion Pro​ file-> EPGs-> Op​ er​ at​ ional Tab Once they ver​ ify end​ points have been learned. The VM and the web por​ tal be​ long to a dif​ fer​ ent EPGs in the same bridge do​ main and ten​ ant.

.

349 Scripting .

.

Scripting 351 Section Content • Leveraging Network Programmability Reference to Object Model Programmatic Interfaces REST Read Operations Write Operations Authentication Filters • API Inspector • Development Techniques • POSTman Installation Collections Build Login request Make Query to APIC Make Configuration Change in APIC Use API Inspector for Query Guidance • Cobra SDK and Arya Establish Session Work with Objects Cisco APIC REST to Python Adapter .

352 Scripting • ACI Toolkit ACI Toolkit Applications Endpoint Tracker ACI Lint • GitHub Source Control GitHub "It's on github" .

yield​ ing a higher qual​ ity prod​ uct. This move​ ment from man​ ual to au​ to​ mated op​ er​ at​ ions changed human pro​ duc​ tiv​ ity. they re​ al​ ize that these ben​ e​ fits will allow them to keep up with the pace of busi​ ness. in​ crease in speed and in​ creased qual​ ity al​ lowed for more work to be done for less money in less time. .Scripting 353 Leveraging Network Programmability The in​ dus​ trial rev​ ol​ u​ tion mod​ ern​ ized the tech​ niques used to man​ uf​ ac​ ture goods. using com​ mon tools and lan​ guages to pro​ vide net​ work en​ gi​ neers. and yielded more con​ sis​ tency in the con​ fig​ ured re​ sults. The as​ so​ ci​ ated de​ crease in costs. Though ACME is just get​ ting started with true De​ vOps in their IT or​ ga​ ni​ za​ tion. Pro​ gram​ ma​ bil​ ity promises to offer the same out​ come for net​ works as the in​ dus​ trial rev​ ol​ u​ tion did for goods. ACME's net​ work en​ gi​ neer​ ing and de​ sign teams can ben​ e​ fit from the quick time to pro​ vi​ sion large con​ fig​ ur​ a​ tions. and the con​ sis​ tency pro​ vided by the abil​ ity to au​ to​ mate all of the mov​ ing parts. yield​ ing faster time to res​ ol​ u​ tion and higher cus​ tomer sat​ is​ fac​ tion. de​ vel​ op​ ers and even novices an ap​ proach​ able path to​ ward au​ toma​ tion. Their op​ er​ at​ ions teams can uti​ lize the plethora of in​ for​ ma​ tion con​ tained within the APIC to stream​ line their processes. going from hand pro​ duc​ tion meth​ ods to mech​ an ​ized man​ uf​ ac​ tur​ ing. ACI is able to take ad​ van​ tage of all of these ben​ e​ fits by com​ pletely ex​ pos​ ing all of the na​ tive func​ tion​ al​ ity in pro​ gram​ ma​ ble ways. Given the com​ pre​ hen​ sive​ ness of the pro​ gram​ ma​ bil​ ity fea​ tures avail​ able on ACI. a more cost-ef​ fec​ tive way to rapidly pro​ vi​ sion in​ fra​ struc​ ture in a timely fash​ ion ac​ cord​ ing to de​ mand. every​ one can ben​ e​ fit. The in​ evitable move to​ ward au​ toma​ tion in the IT in​ dus​ try has pro​ vided peo​ ple and busi​ nesses a faster way to achieve their de​ sired goals. al​ low​ ing peo​ ple to free them​ selves from repet​ it​ ive tasks that could be more eas​ ily ac​ com​ plished by a ma​ chine. gather bet​ ter met​ rics and cor​ re​ late events more ac​ cu​ rately.

.

from a pro​ gram​ ma​ bil​ ity per​ spec​ tive it is im​ por​ tant to note that every as​ pect of ACI func​ tion​ al​ - . and con​ fig​ u​ ra​ tions are dri​ ven using text input that is sim​ pler for a per​ son to type. and also can eas​ ily rep​ re​ sent the full de​ tail that a com​ pre​ hen​ sive ob​ ject-ori​ ented con​ fig​ u​ ra​ tion model may rep​ re​ sent. at a va​ ri​ ety of lev​ els that will cater to the level of com​ fort the user has with pro​ gram​ ming. how​ ever the meth​ ods by which these goals may be re​ al​ ized have been more dif​ fi​ cult to grasp. ACI uses an ad​ vanced ob​ ject model that rep​ re​ sents net​ work con​ fig​ u​ ra​ tion with ap​ pli​ ca​ tion-based se​ man​ tics which can be con​ sumed and posted against using a well doc​ u​ mented REST API. how​ ever these goals stand in con​ trast to an au​ toma​ tion-dri​ ven ap​ proach.Scripting 355 ACI and Scripting The goals for net​ work pro​ gram​ ma​ bil​ ity are clear. In ad​ di​ tion to pro​ vid​ ing this in​ ter​ face into the ob​ ject model. all of which use open stan​ dards and open source. ACI also pro​ vides a num​ ber of ac​ cess meth​ ods to read and ma​ nip​ u​ late this data. Ma​ chines are able to more eas​ ily process data that is pro​ vided in some struc​ tured form. Reference to Object Model Representation of the top levels of the Object Model While a com​ pre​ hen​ sive overview of the Ob​ ject Model is out​ side of this book. Struc​ tured data that may not be vi​ su​ ally ap​ peal​ ing can be rapidly parsed. Tra​ di​ tional net​ work​ ing de​ vices pro​ vide out​ put that is meant for vi​ sual con​ sump​ tion by peo​ ple.

South​ bound in​ ter​ faces on APIC allow for the de​ clar​ at​ ive model of in​ tent to be ex​ tended be​ yond the fab​ ric. This in​ ter​ face is a cru​ cial com​ po​ nent for the GUI and CLI. This means that all of the con​ fig​ ur​ a​ tion that can be made on the fab​ ric.356 Scripting ity is en​ com​ passed within the ob​ ject model. This south​ bound ex​ ten​ sion is re​ al​ ized through two meth​ ods: L4-7 De​ vice Pack​ ages and OpFlex. The REST API is a sin​ gu​ lar entry point to the fab​ ric for mak​ ing con​ fig​ ur​ a​ tion changes. This data is stored within the Man​ age​ ment In​ for​ ma​ tion Tree. Programmatic Interfaces APIC is very flex​ ib ​le in terms of how it can ac​ cept con​ fig​ ur​ a​ tion and pro​ vide ad​ min​ is​ tra​ tive and op​ er​ ab ​le states. There are two pri​ mary cat​ e​ gories of in​ ter​ faces that fa​ cil​ it​ ate these func​ tions: the north​ bound REST API and the south​ bound pro​ gram​ matic in​ ter​ faces. ma​ nip​ u-​ lated. pro​ vi​ sion​ ing scripts and third party mon​ it​ or​ ing and man​ age​ ment tools. so long as the de​ vice has some form of in​ ter​ face which is ac​ ces​ si​ ble via IP. As a . as well as pro​ vid​ ing ac​ cess to man​ age​ ment func​ tions for the con​ troller. The north​ bound REST API is re​ spon​ si​ ble for ac​ cept​ ing con​ fig​ ur​ a​ tion. and also pro​ vides a touch point for au​ toma​ tion tools. with​ out the need to in​ di​ vid​ u​ ally con​ fig​ ure those de​ vices. These de​ vices can be from any ven​ dor. This en​ sures that the con​ fig​ ured state of the model will never get out of hand with stale nodes or en​ tries. as well as ex​ tend​ ing that con​ fig​ ur​ a​ tion into sub​ or​ di​ nate com​ po​ nents. and all other facets of the prod​ uct. vir​ tu​ al​ iza​ tion in​ te​ gra​ tion. L4-7 de​ vices and po​ ten​ tially more in the fu​ ture. can be made pro​ gram​ mat​ ic ​ally using the REST API. This in​ cludes in​ ter​ nal fab​ ric net​ work​ ing. com​ pute in​ te​ gra​ tion. The L4-7 de​ vice pack​ age in​ ter​ face al​ lows for ACI to apply pol​ icy to ex​ ist​ ing L4-7 de​ vices that do not have an im​ plicit knowl​ edge of ACI pol​ icy. into sub​ or​ di​ nate de​ vices. SOAP or oth​ ers. be that REST. in that pol​ icy can be pro​ grammed once via APIC and then pushed out to hy​ per​ vi​ sors. with every piece of the model rep​ re​ sented as a pro​ gram​ matic ob​ ject with prop​ er​ ties. This is a key as​ pect to the open​ ness of the ACI fab​ ric. iden​ tity. which can reach the de​ vice through their na​ tive con​ fig​ ur​ a​ tion in​ ter​ faces. and every as​ pect can be in​ spected. CLI. and con​ sis​ tency rules that are en​ forced. and made to cater for the user's needs. ex​ ter​ nal net​ work​ ing. and as such is a crit​ ic ​al as​ pect of the ar​ chi​ tec​ ture for being able to pro​ vide a con​ sis​ tent pro​ gram​ matic ex​ pe​ ri​ ence. The ac​ tual im​ ple​ men​ ta​ tion of de​ vice pack​ ages is done via Python scripts which run on the APIC within a con​ tained ex​ e​ cu​ tion en​ vi​ ron​ ment.

The POST and DELETE meth​ ods are idem​ po​ tent. OpFlex is de​ signed to allow a data ex​ change of a set of man​ aged ob​ jects that is de​ fined as part of an in​ for​ ma​ tional model. The REST API is the in​ ter​ face into the MIT and al​ lows ma​ nip​ ul​ a​ tion of the ob​ ject model state.Scripting 357 user makes changes to ser​ vice graphs or EPG pol​ icy. . In ACI. the de​ vice pack​ age will trans​ late the APIC pol​ icy into API calls on the L4-7 de​ vice. in​ clud​ ing sta​ tis​ tics. Pay​ loads to and from the REST in​ ter​ face can be en​ cap​ su​ lated through ei​ ther XML or JSON en​ cod​ ing. and any prop​ er​ ties of that ob​ ject are spec​ if​ ied as at​ trib​ utes of that el​ e​ ment. In the case of XML. The API ac​ cepts and re​ turns HTTP (not en​ abled by de​ fault) or HTTPS mes​ sages that con​ tain JavaScript Ob​ ject No​ ta​ tion (JSON) or Ex​ ten​ si​ ble Markup Lan​ guage (XML) doc​ um ​ents. it is read through the REST API. mean​ ing that it can be called zero or more times with​ out mak​ ing any changes (or that it is a read-only op​ er​ at​ ion). The REST API also pro​ vides an in​ ter​ face through which other in​ for​ ma​ tion can be re​ trieved. an event can be sent through a web socket. so that when a change oc​ curs in the MIT. faults. The pro​ to​ col is de​ signed to sup​ port XML and JSON (as well as the bi​ nary en​ cod​ ing used in some sce​ nar​ ios) and to use stan​ dard re​ mote pro​ ce​ dure call (RPC) mech​ an ​isms such as JSON-RPC over TCP. they are writ​ ten through the REST API. and DELETE op​ er​ at​ ions through HTTP. the en​ cod​ ing op​ er​ at​ ion is sim​ ple: the el​ e​ ment tag is the name of the pack​ age and class. mean​ ing that there is no ad​ di​ tional ef​ fect if they are called more than once with the same input pa​ ra​ me​ ters. The same REST in​ ter​ face is used by the Cisco APIC com​ mand-line in​ ter​ face (CLI). and SDK. which in​ cludes POST. You can use any pro​ gram​ ming lan​ guage to gen​ er​ ate the mes​ sages and the JSON or XML doc​ um ​ents that con​ tain the API meth​ ods or MO de​ scrip​ tions. so that when​ ever in​ for​ ma​ tion is dis​ played. and audit events. Con​ tain​ ment is de​ fined by cre​ at​ ing child el​ e​ ments. and when con​ fig​ ur​ a​ tion changes are made. OpFlex it​ self does not dic​ tate the in​ for​ ma​ tion model. GUI. REST The Cisco APIC REST API is a pro​ gram​ matic in​ ter​ face for Cisco APIC that uses REST ar​ chi​ tec​ ture. and can be used with any tree-based ab​ stract model in which each node in the tree has a uni​ ver​ sal re​ source iden​ ti​ fier (URI) as​ so​ ci​ ated with it. GET. OpFlex is cur​ rently used to ex​ tend pol​ icy to the Ap​ pli​ ca​ tion Vir​ tual Switch as well as ex​ tend Group Based Pol​ icy into Open​ Stack. Stan​ dard REST meth​ ods are sup​ ported on the API. The GET method is nul​ lipo​ tent. and it even pro​ vides a means of sub​ scrib​ ing to push-based event no​ ti​ fi​ ca​ tion.

• The children key contains a list that defines all the child objects. Next in the re​ quest URI is the lit​ eral string /api. in​ di​ cat​ ing that the API will be in​ voked. in which the key is the name of the package and class. read. read op​ er​ a​ tions are for an ob​ ject or class. Read Operations After the ob​ ject pay​ loads are prop​ erly en​ coded as XML or JSON. so the next part of the URI spec​ i​ fies . • All objects are described as JSON dictionaries. so it is fairly sim​ ple to im​ ple​ ment after it is ini​ tially un​ der​ stood. or delete op​ er​ a​ tions on the REST API. the de​ f​ i​ n​ i​ tion is re​ peated at all lev​ els of the tree. up​ date.358 Scripting For JSON. REST syntax Be​ cause the REST API is HTTP based. The children in this list are dictionaries containing any nested objects. they can be used in cre​ ate. The first two sec​ tions of the re​ quest URI sim​ ply de​ fine the pro​ to​ col and ac​ cess de​ tails of Cisco APIC. how​ ever. as dis​ cussed ear​ lier. and the value is another nested dictionary with two keys: attribute and children. en​ cod​ ing re​ quires de​ f​ i​ n​ i​ tion of cer​ tain en​ ti​ ties to re​ flect the tree-based hi​ er​ ar​ chy. Gen​ er​ ally. The fol​ low​ ing di​ a​ gram shows the syn​ tax for a read op​ er​ a​ tion from the REST API. which are defined as described here. defin​ ing the uni​ ver​ sal re​ source iden​ ti​ fier (URI) to ac​ cess a cer​ tain re​ source type is im​ por​ tant. • The attribute key contains a further nested dictionary describing key-value pairs that define attributes on the object.

REST Payload . Both cre​ ate and up​ date op​ er​ a​ tions can con​ tain com​ plex ob​ ject hi​ er​ ar​ chies. Write Operations Cre​ ate and up​ date op​ er​ a​ tions in the REST API are both im​ ple​ mented using the POST method. The final manda​ tory part of the re​ quest URI is the en​ cod​ ing for​ mat: ei​ ther . so that a com​ plete tree can be de​ fined in a sin​ gle com​ mand so long as all ob​ jects are within the same con​ text root and are under the 1MB limit for data pay​ loads for the REST API.json. so that if an ob​ ject does not al​ ready exist. This limit is in place to guar​ an​ tee per​ for​ mance and pro​ tect the sys​ tem under high load. The next com​ po​ nent de​ fines ei​ ther the fully qual​ i​ fied Dn being queried for ob​ ject-based queries. For the most part. it will be up​ dated to re​ flect any changes be​ tween its ex​ ist​ ing state and de​ sired state. the con​ fig​ u​ ra​ tion should be trans​ par​ ent to the user. it will be cre​ ated. or the pack​ age and class name for class-based queries. This is the only method by which the pay​ load for​ mat is de​ fined (Cisco APIC ig​ nores Con​ tent-Type and other head​ ers).Scripting 359 whether the op​ er​ a​ tion will be for an MO or class. The con​ text root helps de​ fine a method by which Cisco APIC dis​ trib​ utes in​ for​ ma​ tion to mul​ ti​ ple con​ trollers and helps en​ sure con​ sis​ tency. though very large con​ fig​ u​ ra​ tions may need to be bro​ ken into smaller pieces if they re​ sult in a dis​ trib​ uted trans​ ac​ tion.xml or . and if it does al​ ready exist.

if you want to re​ trieve the re​ sults of your POST op​ er​ a​ tion in the re​ sponse. The fil​ ters them​ selves are ap​ pended as query URI op​ tions. you can pass the rsp-sub​ tree=mod​ i​ fied query string to in​ di​ cate that you want the re​ sponse to in​ clude any ob​ jects that have been mod​ i​ fied by your POST op​ er​ a​ tion. use​ ful for nar​ row​ ing the scope of your search to allow in​ for​ ma​ tion to be lo​ cated more quickly. for which the XPath is /im​ data/aaaLo​ gin/@to​ ken if the en​ cod​ ing is XML. <aaaUser name='admin' pwd='in​ sieme'/>. so the lit​ eral string /mo in​ di​ cates that the Dn of the man​ aged ob​ ject will be pro​ vided. ex​ cept that they al​ ways are tar​ geted at an ob​ ject level. Their pay​ loads con​ tain a sim​ ple XML or JSON pay​ load con​ tain​ ing the MO rep​ re​ sen​ ta​ tion of an aaaUser ob​ ject with the at​ tribute name and pwd defin​ ing the user​ name and pass​ word: for ex​ am​ ple. Filters The REST API sup​ ports a wide range of flex​ i​ ble fil​ ters. .and pass​ word-based au​ then​ ti​ ca​ tion uses a spe​ cial sub​ set of re​ quest URIs. The cre​ ate or up​ date op​ er​ a​ tion should tar​ get a spe​ cific man​ aged ob​ ject. be​ cause you can​ not make changes to every ob​ ject of a spe​ cific class (nor would you want to). fol​ lowed next by the ac​ tual Dn. Fil​ ter strings can be ap​ plied to POST op​ er​ a​ tions. Sub​ se​ quent op​ er​ a​ tions on the REST API can use this token value as a cookie named APIC-cookie to au​ then​ ti​ cate fu​ ture re​ quests. for ex​ am​ ple. Authentication REST API user​ name. in​ clud​ ing aaaLo​ gin. The pay​ load of the POST op​ er​ a​ tion will con​ tain the XML or JSON en​ coded data rep​ re​ sent​ ing the man​ aged ob​ ject the de​ fines the Cisco API com​ mand body. and aaaRe​ fresh as the Dn tar​ gets of a POST op​ er​ a​ tion. Mul​ ti​ ple con​ di​ tions can be joined to​ gether to form com​ plex fil​ ters. aaaL​ o​ gout. The re​ sponse to the POST op​ er​ a​ tion will con​ tain an au​ then​ ti​ ca​ tion token as both a SetCookie header and an at​ tribute to the aaaLo​ gin ob​ ject in the re​ sponse named token.360 Scripting Cre​ ate and up​ date op​ er​ a​ tions use the same syn​ tax as read op​ er​ a​ tions. start​ ing with a ques​ tion mark (?) and con​ cate​ nated with an am​ per​ sand (&).

Scripting 361 The fol​ low​ ing query fil​ ters are avail​ able: .

.

To get started with the API In​ spec​ tor. and when in​ for​ ma​ tion is dis​ played on the GUI. it can be ac​ cessed from the Ac​ count menu.Scripting 363 API Inspector All op​ er​ a​ tions that are per​ formed in the GUI in​ voke REST calls to fetch and com​ mit the in​ for​ ma​ tion being ac​ cessed. the API In​ spec​ tor dis​ plays the re​ sult​ ing POST re​ quests. Click Wel​ come. vis​ i​ ble at the top right of the Cisco APIC GUI. URIs. and pay​ loads. API Inspector . There may also be oc​ ca​ sional up​ dates in the list as the GUI re​ freshes sub​ scrip​ tions to data being shown on the screen. the GET re​ quest is dis​ played. <user​ name> and then choose the Show API In​ spec​ tor op​ tion After the API In​ spec​ tor is brought up. When a new con​ fig​ u​ ra​ tion is com​ mit​ ted. The API In​ spec​ tor fur​ ther sim​ pli​ fies the process of ex​ am​ in​ ing what is tak​ ing place on the REST in​ ter​ face as the GUI is nav​ i​ gated by dis​ play​ ing in real time the URIs and pay​ loads. time stamps will ap​ pear along with the REST method.

364 Scripting From the out​ put above it can see that the last logged item has a POST re​ quest with the JSON pay​ load con​ tain​ ing a ten​ ant named Cisco and some at​ trib​ utes de​ fined on that ob​ ject: url: http://​ 172.​ 176/​ api/​ node/​ mo/​ uni/​ tn-Cisco. "children": [ { "fvRsCtx": { "attributes": { "tnFvCtxName": "CiscoVrf". { "fvCtx": { "attributes": { "name": "CiscoVrf".​ 176. "children": [] } } ] } }. "status": "created" }. "status": "created.​ json { "fvTenant": { "attributes": { "name": "Cisco". "status": "created" }.modified" }. "children": [ { "fvBD": { "attributes": { "mac": "00:22:BD:F8:19:FF". "name": "CiscoBd".​ 16. "status": "created" .

Scripting 365 }. "children": [] } } ] } } .

.

and using com​ mon freely avail​ able tools. ACI Toolkit is a util​ ity de​ vel​ oped in open-source that ex​ poses the most com​ mon ACI build​ ing blocks. to en​ able users to rapidly cre​ ate ten​ ants. or by sav​ ing XML/JSON di​ rectly from the GUI. . With a com​ plete rep​ re​ sen​ ta​ tion of the ACI ob​ ject model avail​ able. EPGs and the as​ so​ ci​ ated con​ cepts to con​ nect those to phys​ ic ​al in​ fra​ struc​ ture.Scripting 367 Development Techniques ACI has a num​ ber of meth​ ods for de​ vel​ op​ ing code that can be used by en​ gi​ neers who have vary​ ing lev​ els of com​ fort with pro​ gram​ ming or in​ ter​ act​ ing with pro​ gram​ matic in​ ter​ faces. Cobra en​ sures that the com​ plete ACI ex​ pe​ ri​ ence is avail​ able to de​ vel​ op​ ers and users alike. com​ pre​ hen​ sive data val​ id ​a​ tion. ap​ pli​ ca​ tion pro​ files. to send this in​ for​ ma​ tion back to the REST API. such as POST​ man. The most pow​ er​ ful of the de​ vel​ op​ ment tools avail​ able is the Cobra SDK. A step up from this method en​ ables users to use com​ mon ter​ mi​ nol​ ogy and well un​ der​ stood net​ work​ ing con​ structs. The stream​ lined in​ ter​ face pro​ vided makes it very quick to adopt and al​ lows users to begin to quickly de​ velop their ap​ pli​ ca​ tions. and ex​ ten​ sive sup​ port for query​ ing and fil​ ter​ ing. cou​ pling these with the power and flex​ ib ​il​ ity of the ACI pol​ icy lan​ guage and the pop​ ul​ ar Python pro​ gram​ ming lan​ guage to con​ fig​ ure ACI in a pro​ gram​ matic fash​ ion. Vi​ sore. The most basic and straight-for​ ward tech​ nique in​ volves sim​ ply tak​ ing in​ for​ ma​ tion gleaned by the API in​ spec​ tor.

.

the user can switch be​ tween the his​ tory of REST re​ quests sent by POST​ man. pol​ icy and op​ er​ at​ ional state data. POST​ man can be used to in​ ter​ act with the APIC REST in​ ter​ face. the user should first click into the Col​ lec​ tions tab in the side​ bar.Scripting 369 POSTman POST​ man is an open source ex​ ten​ sion for the Chrome web browser. Once the plu​ gin is in​ stalled. In order to do this. ac​ tions. it is very sim​ ple to uti​ lize the API In​ spec​ tor to view what the un​ der​ ly​ ing calls being made to the GUI are for cer​ tain op​ er​ at​ ions. cap​ ture those. . at​ trib​ utes changed in the cap​ tured data and then sent back to the REST API to make the mod​ if​ i​ ca​ tions. For an in​ di​ vid​ ual un​ fa​ mil​ iar with the struc​ ture of REST. it can be ac​ cessed using the Chrome App launcher. as well as Col​ lec​ tions of re​ quests that con​ tain com​ mon tasks. Fur​ ther​ more POST​ man al​ lows for the re​ quests to be mod​ if​ ied: GUI op​ er​ at​ ions can be made once. the first step is to down​ load the plu​ gin for the Chrome web browser. Within the side​ bar. a small folder with a plus (+) sign will be​ come vis​ ib ​le. the col​ lec​ tion can be named “APIC”. which pro​ vides REST client func​ tion​ al​ ity in an easy-to-use pack​ age. at which point a popup will ap​ pear prompt​ ing the user to give a name to the col​ lec​ tion. which is avail​ able at www.​ getpostman. which should then be clicked. after which the Cre​ ate but​ ton should be clicked. to both send and re​ ceive data which may rep​ re​ sent con​ fig​ ur​ a​ tion. and then use POST​ man to re​ play those op​ er​ at​ ions. For this ex​ am​ ple. Installation To get started with POST​ man. Using the side​ bar. Ini​ tially the user will be pre​ sented with an in​ ter​ face that has two pri​ mary sec​ tions: the side​ bar on the left and the re​ quest con​ struc​ tor on the right. Collections A use​ ful post to cre​ ate in a col​ lec​ tion is a basic Login op​ er​ at​ ion.​ com.

370 Scripting

Build Login request
Now a new re​
quest can be built. In the re​
quest con​
struc​
tor, where “Enter re​
quest URL
here” is shown, the fol​
low​
ing re​
quest URI should be en​
tered, sub​
sti​
tut​
ing APIC​
IP
​AD​
DRESS with the IP of the APIC: https://<APIC-IP>/api/mo/aaaLogin.​
xml
This re​
quest URI will call the Login method in the REST API. Since a Login will re​
quire
post​
ing data, the HTTP method should be changed, which can be done by click​
ing the
drop​
down list to the right of the re​
quest URL. By de​
fault it will be a GET re​
quest, but
POST will need to be se​
lected from the drop down list.
With the POST method se​
lected, it is now pos​
si​
ble to pro​
vide the REST pay​
load. Given
that the data will be sent via REST, the “raw” Re​
quest body se​
lec​
tor should be picked.
Now the pay​
load for the re​
quest can be en​
tered, which will be the sim​
ple XML con​
tain​
ing the user​
name and pass​
word that will be used for au​
then​
ti​
ca​
tion. Note that the URL
is https, mean​
ing that it will be en​
crypted be​
tween the web browser and the APIC, so
no data is being trans​
mit​
ted in clear text. The fol​
low​
ing re​
quest body should be en​
tered, sub​
sti​
tut​
ing the cor​
rect user​
name and pass​
word in place of USER​
NAME and
PASS​
WORD: <aaaUser name='USERNAME' pwd='PASSWORD'/>
With this re​
quest built, it is now pos​
si​
ble to Send the re​
quest, but since this will be a
com​
monly used method, the re​
quest should be added to a col​
lec​
tion. This can be ac​
com​
plished by click​
ing the “Add to col​
lec​
tion” but​
ton be​
neath the re​
quest body. Se​
lect
the “APIC” col​
lec​
tion from the ex​
ist​
ing col​
lec​
tion list, and change the Re​
quest name to
“Login” and then click “Add to col​
lec​
tion”.
By adding the re​
quest to a col​
lec​
tion it can later be quickly ac​
cessed to es​
tab​
lish a login
ses​
sion with APIC as needed.
After com​
plet​
ing the above steps, the re​
quest is ready to be sent. Click the “Send” but​
ton in the re​
quest con​
struc​
tor, and the REST API will re​
turn the XML rep​
re​
sent​
ing a
login ses​
sion with the APIC. The fol​
low​
ing will be vis​
ib
​le in the POST​
man GUI:

Scripting 371

Login request in POSTman

Make Query to APIC
The next re​
quest that will be built is one that queries the APIC for a list of ten​
ants on
the sys​
tem. First click the “Reset” but​
ton in the re​
quest con​
struc​
tor, and pro​
ceed with
the same steps as above, ex​
cept that the re​
quest URL will be shown as https://<APICIP>/api/class/fvTenant.​
xml, and the re​
quest method will be changed to GET.
Click “Add to col​
lec​
tion” and place the re​
quest into the APIC col​
lec​
tion, and for the
name enter “Query APIC for ten​
ants”
Now upon click​
ing “Send”, this re​
quest will re​
turn an XML en​
coded list of ten​
ants in the
re​
sponse body sec​
tion of the con​
struc​
tor pane on the right.

Make Configuration Change in APIC
Mak​
ing a con​
fig​
u​
ra​
tion change will use a POST re​
quest sim​
i​
lar to log​
ging in, how​
ever
the re​
quest URL and body will con​
tain a dif​
fer​
ent set of in​
for​
ma​
tion.

372 Scripting

For this ex​
am​
ple, a new ten​
ant will be cre​
ated in the fab​
ric. Click the “Reset” but​
ton in
the re​
quest con​
struc​
tor to clear out all ex​
ist​
ing re​
quest fields, and use URL
https://<APIC-IP>/api/mo/uni.​
xml and change the method to POST.
In the re​
quest pay​
load, enter the fol​
low​
ing data:

<fvTenant name="Cisco"/>

The re​
quest URL spec​
if​
ies that the tar​
get for this query will be the pol​
icy uni​
verse,
which is where ten​
ants live. With this tar​
get prop​
erly scoped, the data rep​
re​
sent​
ing the
ten​
ant can be pro​
vided in the pay​
load, in this case cre​
at​
ing a ten​
ant named Cisco.

Use API Inspector for Query Guidance
As dis​
cussed in the In​
tro​
duc​
tion to Script​
ing sec​
tion, API in​
spec​
tor can be used as a
guide​
line for build​
ing cus​
tom REST re​
quests. Fur​
ther​
ing on the ex​
am​
ple in that sec​
tion,
where the re​
quest URL is https://<APIC-IP>/api/node/mo/uni/tn-Cisco.​
json and
the pay​
load is the fol​
low​
ing com​
pacted ver​
sion of JSON:
{"fvTenant": {"attributes": {"name": "Cisco", "status": "created"}, "children":
[{"fvBD": {"attributes": {"mac": "00:22:BD:F8:19:FF", "name": "CiscoBd", "status":
"created"}, "children": [{"fvRsCtx": {"attributes": {"tnFvCtxName": "CiscoVrf",
"status": "created,modified"}, "children": [] } } ] } }, {"fvCtx": {"attributes":
{"name": "CiscoVrf", "status": "created"}, "children": [] } } ] } }

It is pos​
si​
ble to mod​
ify the re​
quest URI and pay​
load and sub​
sti​
tute the ten​
ant name
“Cisco” with an​
other ten​
ant name, to cre​
ate an en​
tirely new ten​
ant, with the same con​
fig​
ur​
a​
tion. The new re​
quest URL and JSON would be: https://<APICIP>/api/node/mo/uni/tn-Acme.​
json
{"fvTenant": {"attributes": {"name": "Acme", "status": "created"}, "children":
[{"fvBD": {"attributes": {"mac": "00:22:BD:F8:19:FF", "name": "AcmeBd", "status":
"created"}, "children": [{"fvRsCtx": {"attributes": {"tnFvCtxName": "AcmeVrf",
"status": "created,modified"}, "children": [] } } ] } }, {"fvCtx": {"attributes":
{"name": "AcmeVrf", "status": "created"}, "children": [] } } ] } }

Scripting 373

These val​
ues can be placed into a POST re​
quest in POST​
man, and after es​
tab​
lish​
ing a
Login ses​
sion using the saved Login re​
quest, the new ten​
ant “Acme” can be cre​
ated,
iden​
ti​
cal to the pre​
vi​
ously cre​
ated Cisco ten​
ant, with​
out need​
ing to man​
ua
​lly click
through the GUI or use other man​
ual meth​
ods.

Scripting 375

Cobra SDK and Arya
The com​
plete Cisco ACI Python SDK is named Cobra. It is a pure Python im​
ple​
men​
ta​
tion of the API that pro​
vides na​
tive bind​
ings for all the REST func​
tions and also has a
com​
plete copy of the ob​
ject model so that data in​
tegrity can be en​
sured, as well as sup​
port​
ing the com​
plete set of fea​
tures and func​
tions avail​
able in ACI. Cobra pro​
vides
meth​
ods for per​
form​
ing lookups and queries and ob​
ject cre​
ation, mod​
if​
i​
ca​
tion, and
dele​
tion that match the REST meth​
ods used by the GUI and those that can be found
using API In​
spec​
tor. As a re​
sult, pol​
icy cre​
ated in the GUI can be used as a pro​
gram​
ming tem​
plate for rapid de​
vel​
op​
ment.
The in​
stal​
la​
tion process for Cobra is straight​
for​
ward, and you can use stan​
dard Python
dis​
tri​
bu
​t​
ion util​
it​
ies. Cobra is dis​
trib​
uted on the APIC as an .egg file and can be in​
stalled
using easy_in​
stall, and is also avail​
able on github at http://​
github.​
com/​
datacenter/​
cobra.
The com​
plete doc​
um
​en​
ta​
tion for the Cobra SDK is avail​
able at http://​
cobra.​
readthedocs.​
org/​
en/​
latest/​

Establish Session
The first step in any code that uses Cobra is es​
tab​
lish​
ing a login ses​
sion. Cobra cur​
rently sup​
ports user​
name- and pass​
word-based au​
then​
ti​
ca​
tion, as well as cer​
tifi​
catebased au​
then​
ti​
ca​
tion. The ex​
am​
ple here uses user​
name- and pass​
word-based au​
then​
ti​
ca​
tion.
import cobra.mit.access
import cobra.mit.session
apicUri = 'https://10.0.0.2'
apicUser = 'username'
apicPassword = 'password'
ls = cobra.mit.session.LoginSession(apicUri, apicUser, apicPassword)
md = cobra.mit.access.MoDirectory(ls)
md.login()

376 Scripting

This ex​
am​
ple pro​
vides an MoDi​
rec​
tory ob​
ject named md, which is logged into and au​
then​
ti​
cated for Cisco APIC. If for some rea​
son au​
then​
ti​
ca​
tion fails, Cobra will dis​
play a
cobra.​
mit.​
request.​
CommitEr​
ror ex​
cep​
tion mes​
sage. With the ses​
sion logged in, you are
ready to pro​
ceed.

Work with Objects
Use of the Cobra SDK to ma​
nip​
ul​
ate the MIT gen​
er​
ally fol​
lows this work​
flow:
1

Identify the object to be manipulated.

2

Build a request to change attributes or add or remove children.

3

Commit the changes made to the object.

For ex​
am​
ple, if you want to cre​
ate a new ten​
ant, you must first iden​
tify where the ten​
ant will be placed in the MIT, where in this case it will be a child of the pol​
icy Uni​
verse man​
aged ob​
ject (pol​
Un
​iMo):
import cobra.model.pol
polUniMo = cobra.model.pol.Uni('')

With the pol​
Un
​iMo ob​
ject de​
fined, you can cre​
ate a ten​
ant ob​
ject as a child of pol​
U-​
niMo:
import cobra.model.fv
tenantMo = cobra.model.fv.Tenant(polUniMo, 'cisco')

All these op​
er​
at​
ions have re​
sulted only in the cre​
ation of Python ob​
jects. To apply the
con​
fig​
ur​
a​
tion, you must com​
mit it. You can do this using an ob​
ject called a Con​
fi​
gRe​
quest. Con​
fi​
gRe​
quest acts as a con​
tainer for MO-based classes that fall into a sin​
gle
con​
text, and they can all be com​
mit​
ted in a sin​
gle atomic POST op​
er​
at​
ion.
import cobra.mit.request
config = cobra.mit.request.ConfigRequest()
config.addMo(tenantMo)
md.commit(config)

Scripting 377

The Con​
fi​
gRe​
quest ob​
ject is cre​
ated, then the ten​
antMo ob​
ject is added to the re​
quest,
and then you com​
mit the con​
fig​
ur​
a​
tion through the MoDi​
rec​
tory ob​
ject.
For the pre​
ced​
ing ex​
am​
ple, the first step builds a local copy of the pol​
Uni ob​
ject. Be​
cause it does not have any nam​
ing prop​
er​
ties (re​
flected by the empty dou​
ble sin​
gle
quo​
ta​
tion marks), you don’t need to look it up in the MIT to fig​
ure out what the full Dn
for the ob​
ject is; it is al​
ways known as uni.
If you wanted to post some​
thing deeper in the MIT, where the ob​
ject has nam​
ing prop​
er​
ties, you would need to per​
form a lookup for that ob​
ject. For ex​
am​
ple, if you wanted
to post a con​
fig​
ur​
a​
tion to an ex​
ist​
ing ten​
ant, you could query for that ten​
ant and cre​
ate
ob​
jects be​
neath it.
tenantMo = md.lookupByClass('fvTenant', propFilter='eq(fvTenant.name, "cisco")')
tenantMo = tenantMo[0] if tenantMo else None

The re​
sult​
ing ten​
antMo ob​
ject will be of class cobra.​
model.​
fv.​
Tenant and will con​
tain
prop​
er​
ties such as .dn, .sta​
tus, and .name, all de​
scrib​
ing the ob​
ject it​
self. The lookup​
By​
Class() entry re​
turns an array, be​
cause it can re​
turn more than one ob​
ject. In this
case, the com​
mand is spe​
cific and is fil​
ter​
ing on an fv​
Tenant ob​
ject with a par​
tic​
ul​
ar
name. For a ten​
ant, the name at​
tribute is a spe​
cial type of at​
tribute called a nam​
ing at​
tribute. The nam​
ing at​
tribute is used to build the rel​
at​
ive name, which must be unique
in its local name​
space. As a re​
sult, you can be as​
sured that lookup​
By​
Class on an fv​
Tenant ob​
ject with a fil​
ter on the name al​
ways re​
turns ei​
ther an array of length 1 or
None, mean​
ing that noth​
ing was found.
To en​
tirely avoid a lookup, you can build a Dn ob​
ject and make an ob​
ject a child of that
Dn. This method works only in cases in which the par​
ent ob​
ject al​
ready ex​
ists.
topDn = cobra.mit.naming.Dn.fromString('uni/tn-cisco')
fvAp = cobra.model.fv.Ap(topMo, name='AppProfile')

These fun​
da​
men​
tal meth​
ods for in​
ter​
act​
ing with Cobra pro​
vide the build​
ing blocks
nec​
es​
sary to cre​
ate more com​
plex work​
flows that can help au​
to​
mate net​
work con​
fig​
u-​
ra​
tion, per​
form trou​
bleshoot​
ing, and man​
age the net​
work.

378 Scripting

Cisco APIC REST to Python Adapter
The process of build​
ing a re​
quest can be time con​
sum​
ing, be​
cause you must rep​
re​
sent
the ob​
ject data pay​
load as Python code re​
flect​
ing the ob​
ject changes that you want to
make. Be​
cause the Cobra SDK is di​
rectly mod​
eled on the Cisco ACI ob​
ject model, you
should be able to gen​
er​
ate code di​
rectly from what re​
sides in the ob​
ject model. As ex​
pected, you can do this using a tool de​
vel​
oped by Cisco Ad​
vanced Ser​
vices. The tool is
the Cisco APIC REST to Python Adapter, known as Arya.

Sample REST to Python Adapter
The above fig​
ure clearly shows how the input that might come from the API In​
spec​
tor,
Vi​
sore, or even the out​
put of a REST query and can then be quickly con​
verted into
Cobra SDK code, to​
k​
enized, and reused in more ad​
vanced ways.
In​
stal​
la​
tion of Arya is rel​
a​
tively sim​
ple, and the tool has few ex​
ter​
nal de​
pen​
den​
cies. To
in​
stall Arya, you must have Python 2.7.5 and git in​
stalled. Use the fol​
low​
ing quick in​
stal​
la​
tion steps to in​
stall it and place it in your sys​
tem Python.
git clone https://github.com/datacenter/ACI.git
cd ACI/arya
sudo python setup.py install

fv.commit(c) . name='bob') # commit the generated code to APIC print toXMLStr(topMo) c = cobra. For ex​ am​ ple.py -f /home/palesiak/simpletenant.mit.pol.xmlcodec import toXMLStr # log into an APIC and create a directory object ls = cobra.MoDirectory(ls) md.LoginSession('https://1.session.session import cobra.access. enter: arya.request import cobra. 'password') md = cobra.internal.request.1'.mit.1. you can take XML or JSON rep​ re​ sent​ ing Cisco ACI mod​ eled ob​ jects and con​ vert it to Python code quickly.mit.Uni('') # build the request using cobra syntax fvTenant = cobra.xml The entry will yield the fol​ low​ ing Python code: #!/usr/bin/env python ''' Autogenerated code using arya.model.login() # the top level object on which operations will be made topMo = cobra.model. 'admin'.model.codec.Scripting 379 After Arya has been in​ stalled.addMo(topMo) md.py Original Object Document Input: <fvTenant name='bob'/> ''' raise RuntimeError('Please review the auto generated code before ' + 'executing the output.mit.access import cobra. Some placeholders will ' + 'need to be changed') # list of packages that should be imported for this code to work import cobra.mit.1.fv import cobra.mit.Tenant(topMo.ConfigRequest() c.pol from cobra.model.

In this case. it is pur​ posely put in place to help en​ sure that any other to​ ke ​nized val​ ues that must be up​ dated are cor​ rected. The same ap​ plies to the cre​ den​ tials and any other place​ hold​ ers.​ 1. You can find this Dn by query​ ing for the ob​ ject in Vi​ sore or in​ spect​ ing the re​ quest URI for the ob​ ject shown in the API In​ spec​ tor.​ 1. Note that if you pro​ vide input XML or JSON that does not have a fully qual​ if​ ied hi​ er​ ar​ chy. the Cisco APIC IP ad​ dress.380 Scripting The place​ holder rais​ ing a run​ time error must first be re​ moved be​ fore this code can be ex​ e​ cuted. which you will need to re​ place with the cor​ rect Dn.​ 1. which de​ faults to 1. a place​ holder will be pop​ ul​ ated with the text RE​ PLACEME. Arya may not be able to iden​ tify it through heuris​ tics. . For ex​ am​ ple. should be up​ dated to re​ flect the ac​ tual Cisco APIC IP ad​ dress.

In ad​ di​ tion.py—This script creates a web interface that provides a way to present the contents of the database to the operator. load bal​ ancers. a num​ ber of ap​ pli​ ca​ tions have been built on top of ACI toolkit.​ github.​ io/​ acitoolkit/​ ACI Toolkit Applications Endpoint Tracker The end​ point tracker ap​ pli​ ca​ tion cre​ ates a sub​ scrip​ tion to the end​ point class (fvCEp) and pop​ ul​ ates a MySQL data​ base with per​ ti​ nent de​ tails about each end​ point pre​ sent on the fab​ ric (for ex​ am​ ple servers. and other de​ vices). fire​ walls.py—This script creates the subscription to the endpoint class and populates the MySQL database • aci-endpoint-tracker-gui. and give users a way to quickly bring up com​ mon tasks and work​ flows. In​ stalling MySQL is out​ side the scope of this book. which may be daunt​ ing for a user being first in​ tro​ duced to net​ work pro​ gram​ ma​ bil​ ity. The end​ point tracker ap​ pli​ ca​ tion has two pri​ mary com​ po​ nents that are both python scripts. The ac​ it​ oolkit makes avail​ able a sim​ pli​ fied sub​ set of the model that can act as an in​ tro​ duc​ tion to the con​ cepts in ACI.Scripting 381 ACI Toolkit The com​ plete ACI ob​ ject model con​ tains many en​ ti​ ties. The com​ plete doc​ um ​en​ ta​ tion for ac​ it​ oolkit is avail​ able at http://​ datacenter. • aci-endpoint-tracker. so we will as​ sume you have ac​ cess to cre​ ate a new data​ base on a MySQL server. A sample is shown below: .

will actually connect to the APIC and populate the database.1 MySQL login username: root MySQL Password: * Running on http://127.0. The second script enables the content to be viewed in an understandable web UI.0. The first script. and the table is fil​ tered ac​ cord​ ingly.1 MySQL login username: root MySQL Password: user@linuxhost::~/acitoolkit/applications/endpointtracker$ python aci-endpointtracker-gui.​ 5.382 Scripting To launch Endpoint Tracker run the following python scripts. In the ex​ am​ ple below.0.0.0. Using the ACI End​ point Tracker is sim​ ply a mat​ ter of in​ putting an IP or MAC ad​ dress into the search field.py. the IP ad​ dress 192.py MySQL IP address: 127.1:5000/ * Restarting with reloader After run​ ning those python scripts you can now bring up a browser and go the Web UI.py MySQL IP address: 127.​ 168.0./aci-endpoint-tracker. aciendpoint-tracker. . and the match​ ing re​ sults are dis​ played. user@linuxhost:~/acitoolkit/applications/endpointtracker$ .​ 20 has been input into the search field.

Some sam​ ple screen​ shots are shown below. and EPGs. Pie chart view of endpoint distribution . Applications. These are rep​ re​ sen​ ta​ tions of where end points are within the ACI fab​ ric and how they re​ late to or de​ pend on other ob​ jects in the en​ vi​ ron​ ment.Scripting 383 One more interesting usage of the endpoint tracker applications is a series of visualizations that can represent how various endpoints are mapped to other fabric constructs including Tenants.

384 Scripting Tree view of endpoint relationships Force diagram of endpoint location ACI Lint .

In the sense that ACI pro​ vides in​ fra​ struc​ ture as code. Warning 007: Contract 'default' in Tenant 'common' is not consumed at all. Warning 007: Contract 'WebServers' in Tenant 'Acme' is not consumed at all. Warning 006: Contract 'WebServers' in Tenant 'Acme' is not provided at all. Warning 001: Tenant '3tierapp' has no Application Profile. A sam​ ple out​ put is pro​ vided here for ref​ er​ ence: user@linuxhost:~/acitoolkit/applications/lint$ .sup​ ports the abil​ ity to tag EPGs as ei​ ther se​ cure or in​ se​ cure. Warning 001: Tenant 'mgmt' has no Application Profile. Warning 004: Context 'oob' in Tenant 'mgmt' has no BridgeDomains. ACI Lint is an ap​ pli​ ca​ tion that checks and no​ ti​ fies an op​ er​ at​ or of mis​ con​ fig​ ur​ a​ tion er​ rors in two pri​ mary ca​ pac​ it​ ies: Se​ cu​ rity Is​ sues . and then runs a val​ id ​a​ tion that con​ tracts are not used to cross se​ cu​ rity bound​ aries.. . Critical 001: EPG 'default' in tenant 'infra' app 'access' is not assigned security clearance Critical 001: EPG 'x' in tenant 'common' app 'default' is not assigned security clearance Warning 001: Tenant 'Cisco' has no Application Profile. Warning 001: Tenant 'Books' has no Application Profile.. ACI Toolkit pro​ vides just that. Con​ fig​ u​ ra​ tion Is​ sues . Lint is a term that refers to iden​ ti​ fy​ ing dis​ crep​ an​ cies.. it is ap​ pro​ pri​ ate for ACI to also have a Lint ap​ pli​ ca​ tion. Warning 002: Tenant '3tierapp' has no Context. Warning 007: Contract 'outside-to-web' in Tenant 'roberbur' is not consumed at all.checks for com​ mon con​ fig​ ur​ a​ tion er​ rors and re​ ports them to the user. Warning 005: BridgeDomain 'inb' in Tenant 'mgmt' has no EPGs. or sim​ ple debug tool for com​ mon er​ rors.. Warning 007: Contract 'External' in Tenant 'Acme' is not consumed at all.. Warning 006: Contract 'External' in Tenant 'Acme' is not provided at all.Scripting 385 ACI Lint In com​ puter pro​ gram​ ming..py Getting configuration from APIC. Processing configuration. Warning 005: BridgeDomain 'CiscoBd' in Tenant 'Cisco' has no EPGs. Warning 006: Contract 'default' in Tenant 'common' is not provided at all./acilint. Warning 002: Tenant 'Books' has no Context.

the real value is in the abil​ ity to take these ex​ am​ ples as a start​ ing point.386 Scripting While the ACI Toolkit pro​ vides some use​ ful tools for an op​ er​ at​ or to im​ me​ di​ ately use. and mod​ ify or ex​ tend these sam​ ples to suit your par​ tic​ ul​ ar needs. Give it a try! Be sure to share your work back with the com​ mu​ nity! .

While these tools have and con​ tinue to work well. GitHub GitHub is a host​ ing plat​ form based around git. dis​ trib​ uted ar​ chi​ tec​ ture. web servers. or al​ ter​ nately move for​ ward to a newer ver​ sion. Git was cre​ ated by Linus Tor​ valds. Aside from being a wrap​ per around git. Pre​ vi​ ous tools like Con​ cur​ rent Ver​ sion Con​ trol (CVS). The com​ bi​ na​ tion of all of these fea​ tures has made GitHub a very com​ mon place for mem​ bers of the com​ mu​ nity to share code with one an​ other. so it’s not un​ com​ mon for users to store doc​ um ​en​ ta​ tion or other con​ stantly chang​ ing files in git. Git also main​ tains an audit of changes that have been made to files and even has ad​ vanced sup​ port for branch​ ing ver​ sions of files to allow mul​ ti​ ple con​ cur​ rent mod​ if​ i​ ca​ tions to a file to take place. with a cen​ tral server main​ tain​ ing a com​ mon data​ base of source code. What is stored on GitHub is usu​ ally source code. and allow for them to be merged after work ef​ forts have com​ pleted. how​ ever the git pro​ to​ col it​ self sup​ ports stor​ age and ver​ sion con​ trol of any file type. there has been a slow mi​ gra​ tion away from those server-based tools to de​ cen​ tral​ ized util​ it​ ies. Git has a num​ ber of ad​ van​ tages over most other source con​ trol tools: com​ plete local repos​ it​ ory copies. that allow for in​ di​ vid​ ua ​ls to col​ lab​ o​ rate with over eight-mil​ lion other GitHub users on pro​ jects to​ gether. in​ clud​ ing con​ sumer soft​ ware. build on each other's work. and con​ tribute their ef​ forts back into larger pro​ jects. data​ bases and even en​ tire op​ er​ at​ ing sys​ tems. with the fore​ most being Git. . and Sub​ ver​ sion (SVN) were used to allow many de​ vel​ op​ ers to work to​ gether. and more ef​ fi​ cient sup​ port for branches. not lim​ ited to any spe​ cific lan​ guage. The pri​ mary ad​ van​ tage is that the ver​ sion con​ trol pro​ vided by git al​ lows a user to re​ vert a file back to any pre​ vi​ ously stored ver​ sion. se​ cur​ ing ac​ cess to pro​ jects. GitHub also pro​ vides tech​ niques for track​ ing is​ sues.Scripting 387 GitHub Source Control Open source soft​ ware has been a pop​ ul​ ar move​ ment in IT. the au​ thor of the pop​ ul​ ar open-source op​ er​ at​ ing sys​ tem Linux. which pro​ vides both free and paid host​ ing ser​ vices. and has been the mo​ ti​ va​ tion be​ hind many suc​ cess​ ful pro​ jects. and built-in pro​ ject doc​ um ​en​ ta​ tion. One of the key as​ pects to the suc​ cess of open source is the abil​ ity for many de​ vel​ op​ ers around the globe to col​ lab​ or​ ate to​ gether on a sin​ gle pro​ ject.

​ com. which may re​ quire some read​ ing be​ fore at​ tempt​ ing to con​ tribute. “It’s on github”. the next step would be to sign up for an ac​ count on GitHub. which es​ sen​ tially means that the user is propos​ ing their ef​ forts should be pulled back into the orig​ in ​al pro​ ject. What a user does with these files will greatly de​ pend on what the con​ tents are. make changes and com​ mit those back to their pri​ vate branch.​ com and the Mac ver​ sions at http://​ mac. . or split off a pro​ ject that is avail​ able into their own pri​ vate repos​ it​ ory. how​ ever for those who have not had an in​ tro​ duc​ tion it can seem like a com​ plex topic. If those changes work. how​ ever one of the most highly en​ cour​ aged be​ hav​ iors on GitHub is to pro​ vide clear and ob​ vi​ ous doc​ um ​en​ ta​ tion for a pro​ ject. GitHub it​ self has a graph​ ic ​al client with the Win​ dows ver​ sion avail​ able at http://​ windows. avail​ able at http://​ sourcetreeapp.​ com.388 Scripting "It's on github" A com​ mon phrase in mod​ ern IT jar​ gon is. they will typ​ ic ​ally be able to find in​ struc​ tions on how to down​ load and in​ stall the pro​ ject. Other com​ mon source con​ trol tools in​ clude Source​ Tree from At​ lass​ ian. they can “Fork”. mod​ ify and con​ tribute to the pro​ ject.​ github.​ github. so if a new user ac​ cesses the front page of a pro​ ject on Git. though many more ad​ vanced pro​ jects have stan​ dards and rules for con​ tribut​ ing to those pro​ jects that put in place re​ quire​ ments around how work is com​ mit​ ted back into the pro​ jects. and down​ load a graph​ ic ​al-based client to pro​ vide a sim​ pler in​ ter​ face to the com​ mand line-based git tool. Once a user has an ac​ count and a github client. GitHub is ac​ tu​ ally a very sim​ ple tool to use. and the sim​ plest way to begin to take ad​ van​ tage of the in​ for​ ma​ tion stored on GitHub is to sim​ ply ac​ cess a pro​ jects main page and look for the “Down​ load ZIP” but​ ton at the bot​ tom right of any pro​ ject's main page. right on the first page they see. and the user wishes to con​ tribute them back into the orig​ in ​al pro​ ject. The process can be that sim​ ple. and for users fa​ mil​ iar with GitHub. it is pos​ si​ ble to sub​ mit a “Pull” re​ quest. For users look​ ing to con​ tribute back to a pro​ ject. this is an in​ vi​ ta​ tion to down​ load. The re​ sult​ ing down​ loaded file will con​ tain the lat​ est ver​ sion of the files in the pro​ ject.

389 Hardware Expansion and Replacement .

.

Hardware Expansion and Replacement 391 Section Content • Expanding and Shrinking the Fabric ​Switches Add Connected Switch Pre-Provision Switch Before Connection Decommission Existing Switch APICs Add New APIC Decommission Existing APIC • Hardware Diagnostics and Replacement ​Identify Hardware Failure Resolve Leaf Hardware Failure Resolve APIC Hardware Failure Diagnose Equipment Failures .

.

spine switches are added for more through​ put. This sec​ tion will also cover de​ com​ mis​ sion​ ing switches.Hardware Expansion and Replacement 393 Expanding and Shrinking the Fabric ACME may de​ cide to ex​ pand their ACI fab​ ric as their data cen​ ter grows. Switches There are two ways switches can be added to ACME's ex​ ist​ ing fab​ ric: by dis​ cov​ er​ ing the switches au​ to​ mat​ ic ​ally in the APIC after they have been ca​ bled to the fab​ ric. This sec​ tion will walk through the op​ er​ at​ ions of adding and re​ mov​ ing switches and APICs in your ex​ ist​ ing ACI fab​ ric. 2 On the APIC click on Fabric at the top of the screen. APICs are added as the num​ ber of poli​ cies and end​ points in​ crease. . Gen​ er​ ally. Add Connected Switch To add a switch that has al​ ready been at​ tached to the fab​ ric go through the fol​ low​ ing steps in the APIC GUI: 1 In the case of a leaf switch. Both meth​ ods have the same out​ come: an ex​ panded fab​ ric in the mat​ ter of min​ utes. cable it to all the leaf switches. All devices should connect to the leaf switches. or by prepro​ vi​ sion​ ing the switches by adding their se​ ri​ al num​ bers and later con​ nect​ ing them phys​ ic ​ally to the fab​ ric when the switches ar​ rive. leaves should never connect to other leaves. cable it to all of the spine switches. and leaf switches are added for more ac​ cess ports. some switches or APICs may need to be de​ com​ mis​ sioned. there may be times when you need to re​ place failed hard​ ware. 3 Click on Fabric Membership in the left navigation pane. Adding APICs will also be cov​ ered. Also. which means adding new leaf and spine switches. which is dis​ cussed in the Hard​ ware Re​ place​ ments chap​ ter. Ad​ di​ tion​ ally. This is done the same way for both spine and leaf switches. and spines should never connect to other spines. In the case of a spine switch. and pos​ si​ bly APICs. a best-practice ACI fabric is connected in a full mesh topology with every leaf cabled to every spine. Ideally.

As a best practice. The new entry in the Fab​ ric Mem​ ber​ ship win​ dow will show Un​ sup​ ported in the Role col​ umn until the switch is ac​ tu​ ally con​ nected to the fab​ ric. enter the serial number of the switch that will be arriving. Lower numbers are reserved for APICs. and spine nodes with 201. Fab​ ric poli​ cies are cov​ ered in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter. For more in​ for​ ma​ tion on pre-pro​ vi​ sion​ ing poli​ cies. you'll see a node with a serial number but no Node ID or Node Name configured. 7 Repeat this process for all new switches connected to the fabric. number leaf nodes starting with 101. 3 In the Work pane. and spine nodes with 201. you can also pre-pro​ vi​ sion fab​ ric poli​ cies. choose Actions > Create Add Fabric Node Member. 5 Click Submit. 4 In the Create Add Fabric Node Member dialog box. Pre-Provision Switch Before Connection Pre-pro​ vi​ sion​ ing a switch is a handy op​ er​ at​ ionally proac​ tive step to get the switch reg​ is​ tered be​ fore it even ar​ rives to your data cen​ ter. choose Fabric Membership. 6 Click Submit. choose Fabric. In the pop-up window. refer to the fol​ low​ ing white paper: . add a Rack Name name. but the switch will im​ me​ di​ ately be​ come a mem​ ber of the fab​ ric once it ar​ rives and is ca​ bled. The fol​ low​ ing steps walk you through switch pre-pro​ vi​ sion​ ing for both leaves and spines: 1 On the menu bar. b. perform the following actions: a. 2 In the Navigation pane. Note: Repeat this process for all switches you wish to pre-provision.394 Hardware Expansion and Replacement 4 When the new switch appears. This is commonly used to identify the physical location of the switch in the data center. 5 Optionally. To be proac​ tive. As a best practice. Assign a Node ID and a Switch Name. You will need to know the se​ ri​ al num​ ber of the switch you will re​ ceive to pre-pro​ vi​ sion. number leaf nodes starting with 101. Lower numbers are reserved for APICs. Double click the switch and assign a Node ID and a Node Name.

b. as the switch will not for​ ward traf​ fic after de​ com​ mis​ sion​ ing. choose Inventory > Pod 1. c. 4 Click Submit. choose either Regular or Remove from Controller. The switch will show up under the Dis​ abled In​ ter​ faces and De​ com​ mis​ sioned Switches folder in the left nav​ i​ ga​ tion pane. Click the General tab. Per​ form the fol​ low​ ing steps to ver​ ify clus​ ter health: .​ html#_​ Toc405844675 Decommission Existing Switch De​ com​ mis​ sion​ ing a switch can ei​ ther re​ move it from the fab​ ric en​ tirely. 2 In the Navigation pane. or re​ move a switch tem​ porar​ ily to per​ form main​ te​ nance.​ cisco. En​ sure you do not have any de​ vices con​ nected. The switch will no longer show up in the fab​ ric mem​ ber​ ship as a reg​ is​ tered node and the in​ fra​ struc​ ture VTEP IP ad​ dresses it was as​ signed will be re​ moved. en​ sure each APIC in the clus​ ter is fully fit and change the clus​ ter size to re​ flect the new con​ troller you are adding to the clus​ ter. Per​ form the fol​ low​ ing steps from the APIC GUI to de​ com​ mis​ sion a switch from the ACI fab​ ric: 1 On the menu bar. while keep​ ing the switch's node ID and fab​ ric mem​ ber​ ship. and Re​ move from Con​ troller. The Re​ move from Con​ troller op​ tion will com​ pletely re​ move the switch from the ACI fab​ ric and all APICs.Hardware Expansion and Replacement 395 http://​ www. a. There are two types of switch de​ com​ mis​ sion​ ing: Reg​ ul​ ar. 3 Click the switch to decommission in the Navigation pane. Reg​ u​ lar de​ com​ mis​ sion​ ing can be used for main​ te​ nance and es​ sen​ tially si​ lences the switch from re​ port​ ing faults and send​ ing SNMP in​ for​ ma​ tion as a tem​ po​ rary so​ lu​ tion.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c11-731960. APICs Add New APIC Be​ fore mak​ ing any changes to an APIC clus​ ter. In the pop-up. choose Fabric. Chose the Actions > Decommission.

396 Hardware Expansion and Replacement 1 2 On the menu bar. 3 In the Work pane. choose System > Controllers. choose Controllers > APIC_Name > Cluster. Click the Cluster folder. Note: A cluster size of two is not permitted as that does not allow for quorum amongst APICs. and the health state of each controller is Fully Fit from the Cluster folder under the new controller. a.​ html 1 On the menu bar.​ cisco. b. The APIC controllers are added one by one and displayed in the sequential order starting with N + 1 and continuing until the target cluster size is achieved.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ troubleshooting/​ b_​ APIC_​ Troubleshooting. Verify every controller shows Fully Fit under the Heath State column. a. choose Controllers > APIC_Name > Cluster. 2 In the Navigation pane.​ cisco. Change the Target Cluster Administrative Size to reflect the new APIC(s) being added.​ html Per​ form the fol​ low​ ing steps to change the APIC clus​ ter size: 1 On the menu bar. b. Per​ form the fol​ low​ ing steps to add a new APIC to the clus​ ter: 1 Install and stage the APIC by connecting it to the fabric by following the hardware installation guide: http://​ www. Verify that the APIC controllers are in operational state. choose System Controllers. . a. choose System > Controllers. choose Controllers. Expand the first APIC in the folder. choose Actions > Change Cluster Size. In the Navigation pane. c. 2 In the Navigation pane. 4 Click Submit.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ application-policy-infrastructure-controller-apic/​ products-installationguides-list. If any of the APICs are not fully fit. refer to the fol​ low​ ing trou​ bleshoot​ ing guide: http://​ www.

Decommission Existing APIC When de​ com​ mis​ sion​ ing APICs. 6 In the Work pane. choose System > Controllers. . 5 In the Navigation pane. a. Fab​ ric op​ er​ at​ ion will con​ tinue nor​ mally. Per​ form the fol​ low​ ing steps to de​ com​ mis​ sion an APIC that needs to be re​ moved from the fab​ ric: 1 On the menu bar. choose Controllers > APIC_Name > Cluster. You can​ not de​ com​ mis​ sion a pow​ ered on fully fit APIC. Again. Note: Select an APIC that is NOT being decommissioned. Change the Target Cluster Administrative Size to reflect the new APIC(s) being added. a. Note: A cluster size of two is not permitted as that does not allow for quorum ​ amongst APICs. be​ fore mak​ ing any changes to an APIC clus​ ter. 4 Click Submit. click the APIC to be decommissioned. en​ sure each APIC in the clus​ ter is fully fit with the ex​ cep​ tion of the faulty APIC being de​ com​ mis​ sioned. choose Actions > Change Cluster Size. choose Controllers > APIC_Name > Cluster. Verify the APIC no longer appears in the Cluster folder under any of the remaining APICs. they must be de​ com​ mis​ sioned se​ quen​ tially in re​ verse order.Hardware Expansion and Replacement 397 Note: It will take sev​ eral min​ utes for the APICs to syn​ chro​ nize and join the new APIC to the clus​ ter. choose Actions > Actions > Decommission. 2 In the Navigation pane. APIC5 must be de​ com​ mis​ sioned be​ fore APIC4. Note: In the main pane. For ex​ am​ ple. 3 In the Work pane.

.

If ACME ever ex​ pe​ ri​ ences some sort of power surge or sees a com​ po​ nent of their switches go bad. or a com​ bi​ na​ tion of the two that ne​ ces​ si​ tates a leaf re​ place​ ment. Re​ gard​ ing hard​ ware. Identify Hardware Failure When a hard​ ware fail​ ure oc​ curs in the fab​ ric. Ex​ am​ ples of hot-swap​ pable com​ po​ nents on both the leaves and spines in​ clude: • Power supplies • Fan trays Ex​ am​ ples of hot-swap​ pable com​ po​ nents on the spines in​ clude: • Power supplies • Fan trays • Supervisors • System Controller cards • Linecard modules De​ spite sig​ nif​ ic ​ant ad​ vances in the above com​ po​ nents that re​ duce the MTBF. the hot-swap​ pable com​ po​ nents en​ able them to re​ place failed hard​ ware quickly and non-dis​ rup​ tively. For cases where there is a com​ po​ nent level . there are sev​ eral hot-swap​ pable com​ po​ nents on both the leaf and spine switches in ad​ di​ tion to a few com​ po​ nents that are fixed on the chas​ sis. In such an event. faults are raised in the sys​ tem dash​ board and are pre​ sented to the ad​ min​ is​ tra​ tor. the state​ less na​ ture of the ACI fab​ ric pro​ vides sig​ nif​ ic ​ant ad​ van​ tages to ad​ min​ is​ tra​ tors from an op​ er​ at​ ions stand​ point.Hardware Expansion and Replacement 399 Hardware Diagnostics and Replacement The Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric em​ ploys a com​ bi​ na​ tion of key soft​ ware and hard​ ware fea​ tures that are specif​ ic ​ally de​ signed to re​ duce the mean time be​ tween fail​ ures (MTBF) and the mean time to re​ pair (MTTR). there is al​ ways the pos​ si​ bil​ ity of a fail​ ure on a leaf switch ei​ ther in switch hard​ ware or soft​ ware.

Resolve Leaf Hardware Failure As an ex​ am​ ple of a leaf fail​ ure. After con​ firm​ ing that the leaf node has failed. there is a pos​ si​ bil​ ity that the hard​ ware has failed. such as SNMP polling at a set in​ ter​ val. The first step in re​ plac​ ing the failed switch .400 Hardware Expansion and Replacement fail​ ure with re​ dun​ dant com​ po​ nents pre​ sent in the sys​ tem. the APICs them​ selves do not have the abil​ ity to gen​ er​ ate alerts using SNMP or sys​ log. For ex​ am​ ple a power sup​ ply fail​ ure on the APIC will not gen​ er​ ate an SNMP or sys​ log mes​ sage and must be mon​ it​ ored and re​ me​ di​ ated using the APIC dash​ board. a Nexus 9396 leaf switch that is a part of the fab​ ric is un​ reach​ able. The leaf and spine switches in the ACI fab​ ric also sup​ port tra​ di​ tional meth​ ods of de​ tect​ ing fail​ ures. while the leaf and spine switches re​ port SNMP and Sys​ log mes​ sages for com​ po​ nent level fail​ ures. such as Splunk. to pro​ vide alert​ ing. you want to re​ move the failed switch and pro​ vi​ sion a new switch as part of the fab​ ric. Note: The pod health displays in the Work pane and is zero. Ex​ am​ ples of hard​ ware events that gen​ er​ ate sys​ log mes​ sages and SNMP traps in​ clude: • Linecard failure on a spine switch • Supervisor failure on a spine switch • System controller failure on a spine switch • Power supply or fan failures on a leaf or a spine switch While Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) is a cen​ tral point of man​ age​ ment for the en​ tire fab​ ric. op​ er​ at​ ions teams can lever​ age their ex​ ist​ ing NMS tools. or SNMP mes​ sages can be sent to NMS sys​ tems. 2 In the Navigation pane. Log​ ging mes​ sages can be sent to sys​ log servers. choose Pod 1. How​ ever. per​ haps due to a hard​ ware fail​ ure on the up​ link mod​ ules. choose Fabric > Inventory. such as ZenOSS. If re​ sponses are not re​ ceived from the switch in a cer​ tain time​ frame. To view the node health score: 1 On the menu bar. sys​ log mes​ sages and SNMP traps are gen​ er​ ated. You can use the GUI to de​ ter​ mine the node health to con​ firm that the leaf has failed.

Re​ plac​ ing the node is as sim​ ple as de​ com​ mis​ sion​ ing the switch and recom​ mis​ sion​ ing it. .xml?rsp-subtree-include=health&rspsubtree-filter=le(healthInst. 2 Load the correct version of code. 5 Copy the old configuration over to the switch.Hardware Expansion and Replacement 401 is to get the failed switch's unique ID (node ID). and syslog servers and the ACLs that are associated with each of them. 2 In the Navigation pane. expand Pod 1. 3 Attempt to obtain the latest version of configurations from a configuration repository server.cur. which is the ref​ er​ ence ob​ ject that al​ lows a re​ place​ ment switch with a new se​ ri​ al num​ ber to in​ herit the same state​ less con​ fig​ ur​ a​ tion that was as​ signed to the old node. update the AAA. You can also use a sin​ gle REST API call to pe​ ri​ od​ ic ​ally poll for a full list of nodes that are at or below a cer​ tain health level. 6 Bring up links one by one and verify if data traffic is flowing correctly. In an ACI fab​ ric. For example. 4 Stage the device with the right configuration file and eliminate any errors. choose Fabric Membership. as shown in the fol​ low​ ing ex​ am​ ple: {{protocol}}://{{apic}}/api/class/topSystem. choose Fabric > Inventory. the fol​ low​ ing high-level pro​ ce​ dure re​ places the switch: 1 Stand up the replacement switch. To de​ com​ mis​ sion and recom​ mis​ sion a switch: 1 On the menu bar. To view the fab​ ric node IDs using the GUI: 1 On the menu bar. you can take ad​ van​ tage of the state​ less na​ ture of the hard​ ware to in​ stan​ ti​ ate the log​ ic ​al con​ fig​ ur​ a​ tion pro​ files. 3 Right click the failed node and choose Decommission. choose Fabric > Inventory. 2 In the Navigation pane."0") In the case of a tra​ di​ tional op​ er​ at​ ions model where each switch was man​ aged as an in​ de​ pen​ dent en​ tity. Each node is as​ signed an ID in the fab​ ric. NTP.

0. choose Fabric Membership. enter the information that you recorded in this procedure. To view which ver​ sion of the firmware that the APIC will load: 1 On the menu bar. you can also reuse the same leaf name. To view the un​ reach​ able nodes: 1 On the menu bar. When the new leaf switch is com​ mis​ sioned suc​ cess​ fully. 2 In the Navigation pane. choose Unreachable Nodes. the APIC au​ to​ mat​ ic ​ally loads the cor​ rect ver​ sion of the firmware into the leaf. 9 Choose Actions > Commission Switch. 2 In the Navigation pane. You can get the name and ID by view​ ing the un​ reach​ able nodes. In most cases. 10 When prompted for the node ID. 6 In the Navigation pane. 11 Click Update.0. enter the old node's ID. starting with step 5. choose Admin > Firmware. choose Fabric > Inventory. Note: In the Work pane. If the new switch is not op​ er​ at​ ional. the new switch's name and node ID are dif​ fer​ ent from the name and ID that you en​ tered. choose Fabric > Inventory. choose Fabric Node Firmware > Firmware Groups > All. When prompted for the name and node ID.402 Hardware Expansion and Replacement 4 Replace the failed leaf switch with the new leaf switch. 5 On the menu bar. 3 Find the new switch and record its name and node ID. 8 In the Work pane. . which is automatically set to the latest firmware version. 4 Repeat the "To decommission and recommission a switch" procedure.0. 7 The new leaf appears with a node ID of 0 and an IP address of 0. you can see the target firmware version. click on the new leaf.

2 Set the IP address on the mgmt0 interface to allow connectivity between the switch and the APIC. bridge do​ mains. by lever​ ag​ ing the state​ less ob​ ject mod​ el​ ing that re​ places the tra​ di​ tional run​ ning con​ fig​ u​ ra​ tion on a de​ vice. you might need to copy the ACI switch soft​ ware image to the switch in ques​ tion. ACLs. sys​ log. switch(config)# copy running-config startup-config . To copy the ACI switch soft​ ware image to the switch: 1 Connect to the switch console. and EPGs. NTP. switch(config)# no boot nxos 7 Save the configuration. APIC au​ to​ mat​ i​ cally loads the cor​ rect run​ ning con​ fig​ u​ ra​ tion onto the de​ vice. ensure that images are copied to the standby supervisor in case of a full chassis replacement by using the command: # copy bootflash:aci_image bootflash://sup-standby/ 6 Configure the switch not to boot from Cisco NX-OS. such as AAA. SNMP.Hardware Expansion and Replacement 403 In ad​ di​ tion. In the event that the re​ place​ ment switch runs stand​ alone NX-OS soft​ ware in​ stead of ACI switch soft​ ware. 3 Enable SCP services: # feature scp-server 4 Copy the firmware image from APIC to the switch: # scp -r /firmware/fwrepos/fwrepo/switch_image_name admin@switch_ip_address:switch_image_name 5 For dual supervisor systems.

the correct certificates are installed. admin@apic1:aci> openssl asn1parse /securedata/ssl/server.404 Hardware Expansion and Replacement 8 Boot the active and standby supervisor modules with the ACI image. . Once you have con​ firmed the that cer​ tifi​ cate is in​ stalled and the switch is in ACI mode. From the GUI of an op​ er​ at​ ional APIC.crt 13 Look for PRINTABLESTRING in the command output. If something else is listed. switch(config)# boot aci bootflash:aci-image-name 9 Verify the integrity of the file by displaying the MD5 checksum. the switch should ap​ pear as an un​ man​ aged fab​ ric node when con​ nected to the fab​ ric. contact TAC to generate and install the correct certificates for your device. 1 On the menu bar choose System > Controllers. Resolve APIC Hardware Failure In this ex​ am​ ple. switch(config)# show file bootflash:aci-image-name md5sum 10 Reload the switch. Login: admin 12 Verify whether you must install certificates for your device. you must iden​ tify and re​ me​ di​ ate a hard​ ware fail​ ure on one of the APICs in your APIC clus​ ter. If "Cisco Manufacturing CA" is listed. switch(config)# reload 11 Log in to the switch as an administrator.

4 In the Work pane. node ID of the failed APIC. choose Controllers > apic_name > Cluster. and the TEP address space.Hardware Expansion and Replacement 405 2 In the Navigation pane. click the new APIC to select it. The new APIC should boot to the initial setup script. click the failed APIC to select it. Failure to configure the APIC with the same settings could result in the fabric entering a partially diverged state. . Decommissioning a Failed APIC 6 Remove the failed APIC from your rack and install the replacement. in the Navigation pane. 5 Choose Actions > Decommission. You can choose any APIC. The APIC changes to an "Out of Service" admin state. This information is also available through the acidiag avread command on APIC’s CLI. choose Controllers > apic_name > Cluster. 9 In the Work pane. Note: In the Work pane. target size. you should see the failed APIC in the "Unavailable" operational state. 7 Proceed through the setup script and enter the values of the failed APIC that you recorded in step 3. 3 Record the fabric name. 8 Once the new APIC finishes booting.

Recommissioning an APIC 11 The new APIC will receive an IP address. Diagnose Equipment Failures The ACI fab​ ric pro​ vides bootup. run​ time and on-de​ mand di​ ag​ nos​ tics to help as​ sess the hard​ ware health of sev​ eral sub-sys​ tems on each leaf and spine switch. The new APIC might also cycle between the Available and Unavailable operational states before becoming Fully Fit. which will be reflected in the APIC GUI. It might take 5 to 10 minutes for this to occur. .406 Hardware Expansion and Replacement 10 Choose Actions > Commission. Waiting for Cluster Convergence 12 On the command line of the new APIC. you can verify that it has joined the fabric by logging in using the credentials that are configured for the rest of the fabric.

Viewing Diagnostic Monitoring Policies in the GUI . To look at the de​ fault di​ ag​ nos​ tic poli​ cies. Comes with default set of tests that can be modified. These are typically ONLY disruptive tests. 2 Health (aka On-going) tests run periodically. Deployed via selectors. Can only run non-disruptive tests. By de​ fault.Hardware Expansion and Replacement 407 1 Boot-up tests run when switch. tests are log​ i​ cally grouped into col​ lec​ tions. there are no defaults. click fab​ ric > fab​ ric poli​ cies > Mon​ i​ tor​ ing poli​ cies > de​ fault > di​ ag​ nos​ tics pol​ icy In the work pane se​ lect the fab​ ric el​ e​ ment that you would like to view the di​ ag​ nos​ tic mon​ i​ tor​ ing pol​ icy for. and they can be disruptive. card boots up. Comes with default set of tests that can be modified and are deployed via selectors 3 On-Demand Tests are to be run on specific ports or cards for troubleshooting.

Create new on-demand diagnostic test As a part of op​ er​ a​ tional pro​ ce​ dures. di​ ag​ nos​ tic re​ sults are avail​ able for all the su​ per​ vi​ sor. . mod​ ules. in the work pane se​ lect the Trou​ bleshoot​ ing tab to view GOLD di​ ag​ nos​ tic re​ sults for the su​ per​ vi​ sor. it may be nec​ es​ sary to val​ i​ date a sys​ tem is healthy by run​ ning non-dis​ rup​ tive tests on the sys​ tem. Viewing GOLD Diagnostics Information in the GUI In a mod​ u​ lar chas​ sis-based sys​ tem such as the Cisco Nexus 9500 se​ ries switch.408 Hardware Expansion and Replacement Test re​ sults are view​ able by click​ ing on Fab​ ric > in​ ven​ tory > Pod-1 > Leaf-xx or Spine-xx > Chas​ sis > Su​ per​ vi​ sor mod​ ules > Slot-1 AND Fab​ ric > in​ ven​ tory > Pod-1 > Leaf-xx or Spine-xx > Chas​ sis > Line mod​ ules > Slot-1 Once there. sys​ tem con​ troller and the fab​ ric mod​ ules in the sys​ tem.

Enter a name for the test. the op​ er​ a​ tor se​ lects the op​ tions for a non-dis​ rup​ tive test to be done on leaf-2. under the di​ ag​ nos​ tic tests. se​ lect a test con​ fig (de​ fault is "no tests") and se​ lect the fab​ ric ports that the test would need to be run on. right click on the test or test set you would like to run on de​ mand. the APIC GUI al​ lows cre​ ation of an On-de​ mand di​ ag​ nos​ tic test. In the ex​ am​ ple below.Hardware Expansion and Replacement 409 In order to do this. Creating a On-Demand Diag Policy in the GUI Once you click cre​ ate. In this case. nav​ i​ gate to Fab​ ric > Fab​ ric Poli​ cies > Trou​ bleshoot Poli​ cies > On de​ mand di​ ag​ nos​ tics Once there. an op​ er​ a​ tor is cre​ at​ ing a fab​ ric-port on-de​ mand diag pol​ icy that al​ lows the abil​ ity to test the leaf up​ link ports going to the spine. en​ sure you check the box against "in​ clude dis​ rup​ tive tests" if this is the in​ tent. To do this. fab​ ric-port 1/97. . se​ lect the admin state. and click on Cre​ ate.

click "SUB​ MIT" to sub​ mit the pol​ icy. Note that APIC dis​ plays a warn​ ing mes​ sage in cases where a non-dis​ rup​ tive tests are se​ lected to the ef​ fect of below. you can kick off the di​ ag​ nos​ tic test by click​ ing on the test it​ self and click​ ing sub​ mit. Once sub​ mit​ ted.410 Hardware Expansion and Replacement Creating a Fabric Port On-Demand Diag Policy in the GUI After ver​ i​ fy​ ing the pa​ ra​ me​ ters above. .

Hardware Expansion and Replacement 411 GUI Warning for Executing a Disruptive Diag Policy Once you con​ firm the test run. Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-2 > Line-mod​ ules > Slot-1 > Fab​ ric ports > 1/97 Viewing On-Demand Diag Policy Test Results in the GUI Note the on-de​ mand re​ sults in the right hand cor​ ner after the test has com​ pleted its run. the di​ ag​ nos​ tic re​ sults can be ob​ tained from the same lo​ ca​ tion as the lo​ ca​ tion for di​ ag​ nos​ tics that are run at bootup or are on​ go​ ing. .

.

413 Appendix .

.

The list below is a sub​ set of the full list of the avail​ able classes.Appendix 415 Classes The Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) classes are cru​ cial from an op​ er​ at​ ional per​ spec​ tive to un​ der​ stand how sys​ tem events and faults re​ late to ob​ jects within the ob​ ject model. You can in​ voke an API func​ tion by send​ ing an HTTP/1. and/or sta​ tis​ tics. The APIC REST API is a pro​ gram​ matic in​ ter​ face to the APIC that uses a REST ar​ chi​ tec​ ture. con​ fir​ ma​ tion of a re​ quested ac​ tion. You can use any pro​ gram​ ming lan​ guage to gen​ er​ ate the mes​ sages. Each event and/or fault in the sys​ tem is a unique ob​ ject that can be ac​ cessed for con​ fig​ ur​ a​ tion. fault. The API ac​ cepts and re​ turns HTTP or HTTPS mes​ sages that con​ tain JSON or XML doc​ um ​ents. All the phys​ ic ​al and log​ ic ​al com​ po​ nents that com​ prise the Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture fab​ ric are rep​ re​ sented in a hi​ er​ ar​ chi​ cal man​ age​ ment in​ for​ ma​ tion tree (MIT). Each node in the tree rep​ re​ sents a man​ aged ob​ ject (MO) or group of ob​ jects that con​ tains its ad​ min​ is​ tra​ tive state and its op​ er​ at​ ional state. health. The HTML body of the POST mes​ sage con​ tains a JSON or XML data struc​ ture that de​ scribes an MO or an API method. GET. point to the APIC and ref​ er​ ence the doc/html di​ rec​ tory at the end of the URL: https://apic_ip_address/doc/html/ . The HTML body of the re​ sponse mes​ sage con​ tains a JSON or XML struc​ ture that con​ tains re​ quested data. The fol​ low​ ing sec​ tion is a rep​ re​ sen​ ta​ tion of use​ ful classes for es​ tab​ lish​ ing a foun​ da​ tion for mon​ it​ or​ ing and man​ age​ ment of the fab​ ric. or error in​ for​ ma​ tion. or DELETE mes​ sage to the APIC. To ac​ cess the com​ plete list of classes. and the JSON or XML doc​ um ​ents that con​ tain the API meth​ ods or man​ aged ob​ ject (MO) de​ scrip​ tions.1 or HTTPS POST.

This ob​ ject is at​ tached as a child of the ob​ ject on which the fault con​ di​ tion oc​ curred.96.16. in​ clud​ ing con​ trollers. Usage: The fab​ ric​ N​ ode class can be used to de​ rive ob​ ject prop​ er​ ties in​ clud​ ing node se​ ri​ al num​ bers. fabricNode REST :: https://172. One in​ stance ob​ ject is cre​ ated for each fault con​ di​ tion of the par​ ent ob​ ject.2/api/node/class/topSystem. in​ clud​ ing con​ trollers.416 Appendix Fabric Monitoring topSystem Name: top:Sys​ tem De​ scrip​ tion: Pro​ vides a list of all the de​ vices within the fab​ ric. as​ signed node ids. topSystem REST :: https://172. leafs and spines. cur​ rent time. node model num​ bers and de​ vice roles. sys​ tem up​ time and cur​ rent state. leafs and spines. Usage: The top​ Sys​ tem class can be used to de​ rive ob​ ject prop​ er​ ties in​ clud​ ing inb/oob man​ age​ ment de​ tails.json faultInst Name: fault:Inst De​ scrip​ tion: Con​ tains de​ tailed in​ for​ ma​ tion of the fault. . ten​ ant or in​ di​ vid​ ual man​ aged ob​ jects within the APIC. Usage: The fault​ Inst class can be used to de​ rive all faults as​ so​ ci​ ated with the fab​ ric.json fabricNode Name: fab​ ric:Node De​ scrip​ tion: Pro​ vides a list of all the nodes that are part of the fab​ ric.96.2/api/node/class/fabricNode.16. A fault in​ stance ob​ ject is iden​ ti​ fied by a fault code.

2/api/node/class/fabricHealthTotal.96. fab​ ric:Health​ To​ tal The fab​ ric total health score in​ stance.96.json fabricHealthTotal Name: De​ scrip​ tion: Usage: health.2/api/node/class/faultInst.96. ap​ pli​ ca​ tion pro​ file and end point group.json fvRsCEpToPathEp Name: De​ scrip​ tion: fv:RsCEp​ ToPathEp This is an in​ ter​ nal ob​ ject that pro​ vides a re​ la​ tion to a path end​ point. fvRsCEpToPathEp REST :: https://172.json fvCEp Name: De​ scrip​ tion: fv:CEp A client end​ point at​ tach​ ing to the net​ work.16. Usage: The fvRsCEp​ ToPathEp class can be used to de​ rive path fab​ ric de​ tails such as the node and port as well as the ten​ ant de​ tails such as the ten​ ant name.16.16.json eqptFabP .Appendix 417 faultInst REST :: https://172. The fab​ ricHealth​ To​ tal class can be used to de​ rive the over​ all sys​ tem fabricHealthTotal REST :: https://172. fvCEp REST :: https://172.2/api/node/class/fvCEp.2/api/node/class/fvRsCEpToPathEp. Usage: The fvCEp class can be used to de​ rive a list of end points at​ tached to the fab​ ric and the as​ so​ ci​ ated ip/mac ad​ dress and en​ cap​ su​ la​ tion for each ob​ ject.16.96.

json eqptLeafP Name: De​ scrip​ tion: eqpt:LeafP Fab​ ric port.16. Usage: The eqpt​ FabP class can be used to de​ rive a list of fab​ ric port and the as​ so​ ci​ ated de​ tails such as the line card and chas​ sis place​ ment.json eqptLC Name: eqpt:LCA .96.16.96.2/api/node/class/eqptFabP.2/api/node/class/eqptCh.2/api/node/class/eqptLeafP. Usage: The eqptCh class can be used to de​ rive a chas​ sis list and the as​ so​ ci​ ated de​ tails such as the op​ er​ at​ ional state. se​ ri​ al num​ ber and model num​ ber. eqptCh REST :: https://172. the fab​ ric fac​ ing ex​ ter​ nal IO port. the non-fab​ ric fac​ ing ex​ ter​ nal leaf IO port.96.16. Usage: The eqpt​ FabP class can be used to de​ rive a list of non-fab​ ric port and the as​ so​ ci​ ated de​ tails such as the line card and chas​ sis place​ ment.json eqptCh Name: De​ scrip​ tion: eqpt:ChA The hard​ ware chas​ sis con​ tainer. eqptLeafP REST :: https://172. eqptFabP REST :: https://172.418 Appendix eqptFabP Name: De​ scrip​ tion: eqpt:FabP Fab​ ric port.

2/api/node/class/eqptFt. model. eqptLC REST :: https://172. se​ ri​ al num​ ber. eqptPsu REST :: https://172.96. se​ r​ ial num​ ber and hard​ ware ver​ sion.16.96. and the volt​ age source. which con​ tains the CPU run​ ning con​ trol plane.16. Usage: The eqptFt class can be used to de​ rive a list of fan trays and the as​ so​ ci​ ated de​ tails such as the op​ er​ a​ tional sta​ tus. eqptFt REST :: https://172. se​ ri​ al num​ bers and the num​ ber of ports.json eqptPsu Name: De​ scrip​ tion: eqpt:Psu The power sup​ ply unit.96. Usage: The eqptFt class can be used to de​ rive a list of power sup​ plies within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the model num​ ber.2/api/node/class/eqptPsu.Appendix 419 De​ scrip​ tion: Usage: The line card (IO card). The eqptLC class can be used to de​ rive a list of line cards de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the re​ dun​ dancy state. op​ er​ at​ ional sta​ tus.json eqptFt Name: De​ scrip​ tion: eqpt:Ft The in​ ven​ to​ ried fan tray. model num​ ber. con​ tain​ ing IO ports.json eqptSupC Name: De​ scrip​ tion: eqpt:SupC The su​ per​ vi​ sor card.16. .2/api/node/class/eqptLC.

json ethpmPhysIf Name: De​ scrip​ tion: ethpm:PhysIf The phys​ ic ​al in​ ter​ face in​ for​ ma​ tion holder.16.96. dbg:Epg​ ToEp​ gRslt The end​ point group to end​ point group atomic counter. se​ ri​ al num​ ber.json dbgEpgToEpgRslt Name: De​ scrip​ tion: entry.96. ethpmPhysIf REST :: https://172. du​ plex.2/api/node/class/dbgAcTrail. on-de​ mand.json dbgAcTrail Name: De​ scrip​ tion: dbg:Ac​ Trail The atomic counter trail. Usage: The eth​ pm​ PhysIf class can be used to de​ rive a list of phys​ ic ​al in​ ter​ faces in the fab​ ric and the as​ so​ ci​ ated de​ tails such as a the speed.96.2/api/node/class/eqptSupC.16. and usage state. dbgAcTrail REST :: https://172. . op​ er​ at​ ional sta​ tus and re​ dun​ dancy state. eqptSupC REST :: https://172.2/api/node/class/ethpmPhysIf.16.420 Appendix Usage: The eqptFt class can be used to de​ rive a list of su​ per​ vi​ sor cards de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the model num​ ber. Usage: The db​ gAc​ Trail class can be used to de​ rive a list of the atomic coun​ ters de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts. op​ er​ at​ ional sta​ tus.

compVm REST :: https://172. Entry.Appendix 421 Usage: The dbgEpg​ ToEp​ gR​ sIt class can be used to de​ rive a list of the EPG to EPG atomic coun​ ters de​ ployed within the fab​ ric.96. dbgEpToEpTsIt REST :: https://172.json dbgEpToEpRslt Name: De​ scrip​ tion: Usage: dbg:Ep​ ToEpRslt The end​ point to end​ point atomic counter.16. Usage: The com​ pVm class can be used to de​ rive a list of vir​ tual ma​ chines de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the name and state. . and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts. The dbgEp​ ToEpT​ sIt class can be used to de​ rive a list of the end​ point to end​ point atomic coun​ ters de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts. On-de​ mand.json VMM Monitoring compVm Name: De​ scrip​ tion: comp:Vm The Vir​ tual ma​ chine ob​ ject.96. dbgEpgToEpgRsIt REST :: https://172.16.2/api/node/class/dbgEpToEpRslt.16.2/api/node/class/dbgEpgToEpgRslt.2/api/node/class/compVm.96.json compHv Name: De​ scrip​ tion: comp:Hv An ob​ ject rep​ re​ sent​ ing the com​ pute hy​ per​ vi​ sor.

For ex​ am​ ple. .json fvRsHyper Name: De​ scrip​ tion: fv:RsHy​ per A re​ la​ tion to the hy​ per​ vi​ sor that con​ trols and mon​ it​ ors the APIC VMs.96.16. the VMM con​ troller pro​ file could be a pol​ icy to con​ nect a VMware vCen​ ter that is part a VMM do​ main.2/api/node/class/compHv.json fvRsVm Name: De​ scrip​ tion: ter​ nal ob​ ject. compHv REST :: https://172.2/api/node/class/fvRsVm. which spec​ if​ ies how to con​ nect to a sin​ gle VM man​ age​ ment con​ troller that is part of con​ tain​ ing pol​ icy en​ force​ ment do​ main.96.16.422 Appendix Usage: The com​ pVm class can be used to de​ rive a list of com​ pute hy​ per​ vi​ sor de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the name and sta​ tus.96.16. Usage: The fvR​ sHy​ per class can be used to de​ rive the re​ la​ tion​ ship of the hy​ per​ vi​ sor that con​ trols and mon​ it​ ors the APIC VMs.json vmmCtrlrP Name: vmm:CtrlrP De​ scrip​ tion: The VMM con​ troller pro​ file. This is an in​ - Usage: The fvRsVm class can be used to de​ rive the re​ la​ tion​ ship of the vir​ tual ma​ chines con​ nected to the hy​ per​ vi​ sor. This is an in​ ter​ nal ob​ ject.2/api/node/class/fvRsHyper. fvRsHyper REST :: https://172. vRsVm REST :: https://172. fv:RsVm A re​ la​ tion to a vir​ tual ma​ chine con​ nected to a hy​ per​ vi​ sor.

json Layer 4 to Layer 7 Monitoring vnsAbsGraph Name: vns​ Ab​ s​ Graph De​ scrip​ tion: The ab​ stract graph is made up of ab​ stract nodes and used to de​ fine the traf​ fic flow through a ser​ vice func​ tion such as load bal​ anc​ ing.96.json .2/api/node/class/vnsLDevVip.96.16. vmmCtrlrP REST :: https://172. Ab​ stract nodes are com​ prised of ser​ vice nodes such as a ser​ vice node bal​ ancer (SLB) or fire​ wall (FW). or fire​ wall. along with their prop​ er​ ties. which is rep​ re​ sented by a sin​ gle vir​ tual IP (VIP).Appendix 423 Usage: The vmm​ C​ trlrP class can be used to de​ rive the ip ad​ dress and the dat​ a-​ cen​ ter name of the con​ nected VM do​ main. vnsAbsGraph REST :: https://172.json vnsLDevVip Name: vnsLDevVip De​ scrip​ tion: An L4-L7 de​ vice clus​ ter.16.96. ab​ stract term nodes (the nodes that are con​ nected to end​ point groups). and con​ nec​ tions.2/api/node/class/vnsAbsGraph.2/api/node/class/vmmCtrlrP. Usage: The class vns​ Ab​ s​ Graph can be used to de​ rive a list of ser​ vice graph tem​ - plates con​ fig​ ured on the APIC. The con​ fig​ ur​ a​ tion is pushed down to the VIP ad​ dress. SSL of​ fload. Usage: The class vnsLDevVip can be used to de​ rive all the VIPs con​ fig​ ured for the log​ ic ​al de​ vice clus​ ters in the fab​ ric vnsLDevVip REST :: https://172.16.

and func​ tion label or names.json vnsLif Name: vnsLif De​ scrip​ tion: The log​ ic ​al in​ ter​ face. Usage: The class vnsLDe​ vCtx can be used to de​ rive the node and con​ tract name.424 Appendix vnsCDev Name: vn​ sCDev De​ scrip​ tion: The in​ di​ vid​ ual ser​ vice de​ vice. Usage: The class vnsLif can be used to de​ rive the con​ nec​ tion be​ tween a ser​ vice graph and de​ vice in​ ter​ faces.16. To spec​ ify a wild card.json vnsRsLDevCtxToLDev . which is used to de​ fine a con​ crete l4-l7 ser​ vice de​ vice.json vnsLDevCtx Name: vnsLDe​ vCtx De​ scrip​ tion: A de​ vice clus​ ter con​ text.2/api/node/class/vnsLIf. which points to the de​ vice clus​ ter used to pick a spe​ cific de​ vice based on con​ tract.2/api/node/class/vnsLDevCtx.96. which is as​ so​ ci​ ated with a set of con​ crete in​ ter​ - faces from the L4-L7 de​ vice clus​ ter. sub​ ject. set the name to Any. Usage: The class vn​ sCDev can be used to de​ rive a list of con​ crete de​ vices con​ fig​ ured as part of the L4-7 ser​ vice in​ te​ gra​ tion vnsCDev REST :: https://172.16.96.2/api/node/class/vnsCDev.96.16. vnsLif REST :: https://172. nsLDevCtx REST :: https://172.

2/api/node/class/compRcvdErrPkts1h.Appendix 425 vnsRsLDevCtxToLDev Name: De​ scrip​ tion: vn​ sRsLDe​ vC​ tx​ ToLDev A source re​ la​ tion to the ab​ strac​ tion of a ser​ vice de​ vice clus​ ter or of a proxy ob​ ject for a log​ ic ​al de​ vice clus​ ter in the ten​ ant.json Statistics compHostStats1h Name: comp:Host​ Stat​ s1h De​ scrip​ tion: A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for host in a 1 hour sam​ pling in​ ter​ val. Usage: The class vn​ sRsLDe​ vC​ tx​ ToLDev can be used to de​ rive the re​ la​ tion​ ship be​ tween vnsLDe​ vCtx and vnsLDev.json compRcvdErrPkts1h Name: comp:RcvdEr​ rP​ kt​ s​ 1h De​ scrip​ tion: A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for re​ ceived error pack​ ets in a 1 hour sam​ pling in​ ter​ val. Usage: The com​ pHost​ Stat​ s1h class can be used to de​ rive the sta​ tis​ tics as​ so​ ci​ ated with the com​ pute hy​ per​ vi​ sor. Usage: The com​ pRcvdEr​ rP​ kt​ s​ 1h class can be used to de​ rive the most cur​ rent sta​ tis​ tics for re​ ceived error pack​ ets. This class up​ dates every 15 min​ utes. This class up​ dates every 15 min​ utes.96.16.16.json .2/api/node/class/compHostStats1h.96.2/api/node/class/vnsRsLDevCtxToLDev. compRcvdErrPkts1h REST :: https://172. vnsRsLDevCtxToLDev REST :: https://172.16.96. compHostStats1h REST :: https://172.

compTrnsmtdErrPkts1h REST :: https://172.json . Usage: The aaaUser class can be used to de​ rive a list of user ac​ counts de​ ployed within the fab​ ric.2/api/node/class/aaaUser.16. Usage: The aaaModLR class can be used to de​ rive a fab​ ric based audit log for all changes and events.16.16. A log record is au​ to​ mat​ ic ​ally gen​ er​ ated when​ ever a user mod​ if​ ies an ob​ ject.426 Appendix compTrnsmtdErrPkts1h Name: De​ scrip​ tion: comp:Trnsmt​ dEr​ rP​ kt​ s​ 1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for trans​ mit​ ted error pack​ ets in a 1 hour sam​ pling in​ ter​ val. aaaModLR REST :: https://172. and Accounting aaaModLR Name: aaa:ModLR De​ scrip​ tion: The AAA audit log record.json Authentication. Authorization.2/api/node/class/aaaModLR.96.json aaaUser Name: De​ scrip​ tion: aaa:User A lo​ cally-au​ then​ ti​ cated user ac​ count.2/api/node/class/compTrnsmtdErrPkts1h.96. Usage: The compTrnsmt​ dEr​ rP​ kt​ s​ 1h class can be used to de​ rive the most cur​ rent sta​ tis​ tics for trans​ mit​ ted error pack​ ets.96. This class up​ dates every 15 min​ utes. aaaUser REST :: https://172.

eqptcapacityPolEntry5min REST :: http://172.2/api/class/eqptcapacityPolEntry5min. . A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for pol​ icy entry in a 5 minute sam​ pling in​ ter​ val.96. This class up​ dates every 10 sec​ onds. Usage: The eqpt​ ca​ pac​ ityL3En​ try5min class can be used to de​ rive the cur​ rent value as​ so​ ci​ ated with the Pre​ fix TCAM usage.json Fabric Capacity Policy TCAM Name: eqpt​ ca​ pac​ it​ y​ Po​ lEn​ try5min De​ scrip​ tion: Pol​ icy CAM entry sta​ tis​ tics.json Prefix TCAM Name: eqpt​ ca​ pac​ ityL3En​ try5min De​ scrip​ tion: Lay​ er3 entry sta​ tis​ tics.16. A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for lay​ er3 entry in a 5 minute sam​ pling in​ ter​ val. Usage: The eqpt​ ca​ pac​ it​ y​ Po​ lEn​ try5min class can be used to de​ rive the cur​ rent value as​ so​ ci​ ated with the Pol​ icy TCAM usage.96. Usage: The aaaUser class can be used to de​ rive a list of re​ mote user ac​ counts de​ ployed within the fab​ ric. This class up​ dates every 10 sec​ onds.16.Appendix 427 aaaRemoteUser Name: aaa:Re​ mo​ teUser De​ scrip​ tion: A re​ mote user login ac​ count. aaaRemoteUser REST :: https://172.2/api/node/class/aaaRemoteUser.

​ html .2/api/node/class/snmpTrapDest. De​ scrip​ tion: Usage: The sn​ mpTrapDest class can be used to de​ rive the cur​ rent list of snmp trap des​ ti​ na​ tions im​ ple​ mented within the fab​ ric.16. syslogRemoteDest REST :: https://172. The Cisco APIC Com​ mand-Line In​ ter​ face User Guide may also be help​ ful for un​ der​ stand​ ing the fol​ low​ ing sec​ tions .16.96.16.​ cisco.2/api/node/class/syslogRemoteDest. Usage: The sys​ lo​ gRe​ mot​ eDest class can be used to de​ rive the cur​ rent list of sys​ log re​ mote des​ ti​ na​ tions im​ ple​ mented within the fab​ ric.json SNMP/SYSLOG SNMP Trap Destination Name: sn​ mpTrapDest A des​ ti​ na​ tion to which traps and in​ forms are sent.json Prefix TCAM Name: sys​ lo​ gRe​ mot​ eDest De​ scrip​ tion: The sys​ log re​ mote des​ ti​ na​ tion host en​ ables you to spec​ ify sys​ log servers to which mes​ sages from the APIC and fab​ ric nodes should be for​ warded.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ cli/​ b_​ APIC_​ CLI_​ User_​ Guide.428 Appendix eqptcapacityL3Entry5min REST :: https://172.json Use Cases The class fault​ Inst used in Use Case #1 and Use Case #2 below can be re​ placed with any of the man​ aged ob​ ject classses dis​ cussed above or spec​ if​ ied within the APIC doc​ u-​ men​ ta​ tion.see: http://​ www.96.96.2/api/class/eqptcapacityL3Entry5min. snmpTrapDest REST :: https://172.

mit.mit.session from cobra. user the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/class/faultInst. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query. Vi​ sore.mit. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -c faultInst From a Vi​ sore per​ spec​ tive.session import LoginSession . POST​ MAN and/or Cobra. From a CLI per​ spec​ tive.query(classQuery): print fault. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions.name Sam​ ple Cobra script to cap​ ture faults within the fab​ ric: #!/usr/bin/env python import cobra. The user has the op​ tion of col​ lect​ ing the re​ sults via CLI.: Class or DN :: faultInst Property :: n/a Op :: n/a Value :: n/a From a POST​ MAN per​ spec​ tive.access import cobra. This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to ob​ tain the list of cur​ rent faults in the fab​ ric. use the fol​ low​ ing class query: # Class Query classQuery= ClassQuery('faultInst') for fault in md.xml From a Cobra per​ spec​ tive.Appendix 429 Case 1: Creating an application script to retrieve the current list of faults in the fabric.

POST​ MAN and/or Cobra.faultInst. use the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/class/faultInst."config-failure")) . <username>. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions. secure=False) md = cobra.mit.cause.access.login() # Class Query classQuery= ClassQuery('faultInst') for fault in md.name Case 2: Creating an application script to retrieve the current list of faults in the fabric that have been caused by a failed configuration. The user has an op​ tion of col​ lect​ ing the re​ sults via CLI. Vi​ sore.session. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query: Class or DN :: faultInst Property :: cause Op :: == Value :: config-failure From a POST​ MAN per​ spec​ tive.430 Appendix from cobra.query(classQuery): print fault.cause=="config-failure"' From a Vi​ sore per​ spec​ tive.LoginSession('https://'<your apic ip address>. <password>.mit.request import ClassQuery ls = cobra.MoDirectory(ls) md.mit. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -c faultInst -f 'fv. This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cob​ tain the list of cur​ rent faults in the fab​ ric.xml?query-targetfilter=and(eq(faultInst. From a CLI per​ spec​ tive.

name Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures.mit.session from cobra.query(classQuery): print fault.mit.format('config-failure') for fault in md.MoDirectory(ls) md.login() # Class Query classQuery= ClassQuery('faultInst') for fault in md.name Case 3: Creating an application script to retrieve the properties for a specific managed object. secure=False) md = cobra. "{0}")'. <password>.mit. DN This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to ob​ tain the prop​ er​ ties of the ten​ ant name Com​ mon. From a CLI per​ spec​ tive.session import LoginSession from cobra. POST​ MAN and/or Cobra. <username>. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions. #!/usr/bin/env python import cobra.mit.access import cobra.access.request import ClassQuery ls = cobra. use the fol​ low​ ing class query: # Class Query classQuery= ClassQuery('faultInst') classQuery.Appendix 431 From a Cobra per​ spec​ tive.query(classQuery): print fault.mit.session. Vi​ sore.propFilter = 'wcard(faultInst.mit.LoginSession('https://'<your apic ip address>. The user has an op​ tion of col​ lect​ ing the re​ sults via CLI.cause. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -d uni/tn-common .

<password>.session from cobra.access.MoDirectory(ls) md. <username>.432 Appendix From a Vi​ sore per​ spec​ tive.xml?query-target=self From a Cobra per​ spec​ tive. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query: Class or DN :: uni/tn-common Property :: n/a Op :: n/a Value :: n/a From a POST​ MAN per​ spec​ tive.login() # DN Query dnQuery= DnQuery('uni/tn-common') for results in md.mit.dn .session import LoginSession from cobra.session.mit.mit.mit. #!/usr/bin/env python import cobra. use the fol​ low​ ing class query: # DN Query dnQuery= DnQuery('uni/tn-common') for results in md.query(dnQuery): print results. use the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/mo/uni/tn-common.dn Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures. secure=False) md = cobra.access import cobra.mit.LoginSession('https://'<your apic ip address>.request import DnQuery ls = cobra.query(dnQuery): print results.mit.

mit. secure=False) md = MoDirectory(ls) md.subtree = 'children' q.mit. #!/usr/bin/env python from cobra.session.access import MoDirectory from cobra.dn .subtreeClassFilter = 'fvRsCEpToPathEp' mos = md.login() q = ClassQuery('fvCEp') q.LoginSession('https://'<your apic ip address>. <password>.Appendix 433 Case 4: Creating an application script to retrieve the current list of endpoints (mac-addresses) attached to the fabric This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate an ap​ pli​ ca​ tion script to cap​ ture the list of cur​ rent end​ points at​ tached to the fab​ ric along with the node de​ tails per​ tain​ ing to each end​ point.mit. <username>. Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures.mit.query(q) for mo in mos: for child in mo.request import ClassQuery lls = cobra.rscEpToPathEp: print child.session import LoginSession from cobra.

.

Here are some de​ scrip​ tions of com​ monly used ab​ bre​ vi​ at​ ions.Appendix 435 Package Decoder There are sev​ eral ab​ bre​ vi​ at​ ions used in the names of classes in the ACI ob​ ject model. au​ tho​ riza​ tion. ac​ count​ ing ac: atomic coun​ ters actrl: ac​ cess con​ trol ac​ trl​ cap: ac​ cess con​ trol ca​ pa​ bil​ ity adcom: ap​ pli​ ance di​ rec​ tor com​ mu​ ni​ ca​ tion aib: ad​ ja​ cency in​ for​ ma​ tion base arp: ad​ dress res​ o​ lu​ tion pro​ to​ col bgp: bor​ der gate​ way pro​ to​ col call​ home: Cisco smart call home ser​ vices cap: ca​ pa​ bil​ ity cdp: Cisco dis​ cov​ ery pro​ to​ col cnw: node clus​ ter comm: com​ mu​ ni​ ca​ tion pol​ icy comp: com​ pute . which may help when de​ ci​ pher​ ing what class ob​ jects are when using them with REST calls. Package Decoder Aaa: au​ then​ ti​ ca​ tion.

436 Appendix com​ pat: com​ pat​ ib ​il​ ity con​ di​ tion: health pol​ icy con​ fig: con​ fig​ ur​ a​ tion pol​ icy coop: Coun​ cil of Or​ ac ​les pro​ to​ col copp: con​ trol plane polic​ ing pol​ icy: con​ tains set of rules de​ scrib​ ing po​ licer rates ctrlr: con​ troller ctx: con​ text date​ time: date/time pol​ icy dbg: debug dbgac: debug atomic coun​ ters dbg​ exp: debug ex​ port pol​ icy dhcp: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col dhcptlv: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col type length value dhcptlvpol: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col type length value pol​ icy dns: do​ main name ser​ vice draw: graph vi​ su​ al​ iza​ tion for GUI epm: end​ point man​ ager eqpt: equip​ ment eqpt​ cap: equip​ ment ca​ pa​ bil​ ity eqpt​ ca​ pac​ ity: equip​ ment ca​ pac​ ity .

Appendix 437 eqpt​ diag: equip​ ment di​ ag​ nos​ tics eqpt​ di​ agp: equip​ ment di​ ag​ nos​ tics pol​ icy ethpm: eth​ er​ net pol​ icy man​ ager event: event pol​ icy extnw: ex​ ter​ nal net​ work fab​ ric: fab​ ric fault: fault pol​ icy. coun​ ters file: file path. con​ fig im​ port/ex​ port pol​ icy firmware: firmware fm​ cast: fab​ ric mul​ ti​ cast fsm: fi​ nite state ma​ chine fv: fab​ ric vir​ tu​ al​ iza​ tion fvns: fab​ ric vir​ tu​ al​ iza​ tion name​ space fv​ topo: fab​ ric vir​ tu​ al​ iza​ tion topol​ ogy geo: ge​ olo​ ca​ tion glean: glean ad​ ja​ cency ha: high avail​ abil​ ity health: health score hvs: hy​ per​ vi​ sors vir​ tual switch icmp: in​ ter​ net con​ trol pro​ to​ col .

438 Appendix icm​ pv4: in​ ter​ net con​ trol pro​ to​ col ver​ sion 4 icm​ pv6: in​ ter​ net con​ trol pro​ to​ col ver​ sion 6 ident: iden​ tity igmp: in​ ter​ net group man​ age​ ment pro​ to​ col igmp​ snoop: in​ ter​ net group man​ age​ ment pro​ to​ col snoop​ ing im: in​ ter​ face man​ ager mod​ ule im​ gin​ stall: image in​ stall infra: in​ fra​ struc​ ture ip: in​ ter​ net pro​ to​ col ipv4: in​ ter​ net pro​ to​ col ver​ sion 4 ipv6: in​ ter​ net pro​ to​ col ver​ sion 6 isis: in​ ter​ me​ di​ ate sys​ tem to in​ ter​ me​ di​ ate sys​ tem isistlv: in​ ter​ me​ di​ ate sys​ tem to in​ ter​ me​ di​ ate sys​ tem type length value l1: layer 1 l1cap: layer 1 ca​ pa​ bil​ ity l2: layer 2 l2cap: layer 2 ca​ pa​ bil​ ity l2ext: layer 2 ex​ ter​ nal l3: layer 3 l3cap: layer 3 ca​ pa​ bil​ ity .

not in the fab​ ric) lldp: link layer dis​ cov​ ery pro​ to​ col lldptlv: link layer dis​ cov​ ery pro​ to​ col type length value lldptlvpol: link layer dis​ cov​ ery pro​ to​ col type length value pol​ icy maint: main​ te​ nance mcast: mul​ ti​ cast mcp: mas​ ter con​ trol proces​ sor mem​ ory: mem​ ory sta​ tis​ tics mgmt: man​ age​ ment mo: man​ aged ob​ ject mock: mock (ob​ jects used on the sim​ ul​ a​ tor mostly for show​ ing stats/faults/etc) mon: mon​ it​ or​ ing mon​ i​ tor: mon​ it​ or (SPAN) nam​ ing: ab​ stract for ob​ jects with names nd: neigh​ bor dis​ cov​ ery nw: net​ work .Appendix 439 l3ext: layer 3 ex​ ter​ nal l3vm: Layer 3 Vir​ tual Ma​ chine lacp: link ag​ gre​ ga​ tion pro​ to​ col lbp: load bal​ anc​ ing pol​ icy leqpt: loose equip​ ment (un​ man​ aged nodes.

and mem​ ory uti​ liza​ tion sta​ tis​ tics psu: power sup​ ply unit pol​ icy qos: qual​ ity of ser​ vice pol​ icy qosm: qos sta​ tis​ tics qosp: qos/ 802. logs/his​ tory opflex: OpFlex os: op​ er​ at​ ing sys​ tem ospf: open short​ est path first pc: port chan​ nel pcons: **gen​ er​ ated and used by in​ ter​ nal processes** phys: phys​ ic ​al do​ main pro​ file ping: ping ex​ e​ cu​ tion and re​ sults pki: pub​ lic key in​ fra​ struc​ ture pol: pol​ icy de​ f​ in ​i​ t​ ion po​ licer: traf​ fic polic​ ing (rate lim​ it​ ing) pool: ob​ ject pool pres: **gen​ er​ ated and used by in​ ter​ nal processes** proc: sys​ tem load. ad​ min​ is​ tra​ tions and man​ age​ ment ob​ server: ob​ server for sta​ tis​ tics. fault. cpu.1p rbqm: de​ bug​ ging . state. health.440 Appendix oam: eth​ er​ net op​ er​ at​ ions.

Appendix 441 regress: re​ gres​ sion reln: **gen​ er​ ated and used by in​ ter​ nal processes** repl: **gen​ er​ ated and used by in​ ter​ nal processes** res: **gen​ er​ ated and used by in​ ter​ nal processes** rib: rout​ ing in​ for​ ma​ tion base rmon: re​ mote net​ work mon​ it​ or​ ing/ in​ ter​ face stats/coun​ ters rpm: route pol​ icy map rtcom: route con​ trol com​ mu​ nity list rtc​ trl: route con​ trol rtextcom: router ex​ tended com​ mu​ nity rtflt: route fil​ ter rtleak: route leak rtmap: RPM route map rtpfx: route pre​ fix list rtreg​ com: route reg​ ul​ ar com​ mu​ nity list rtsum: route sum​ ma​ riza​ tion ad​ dress/pol​ icy satm: satel​ lite man​ ager snmp: sim​ ple net​ work man​ age​ ment pro​ to​ col span: switched port an​ al​ yzer stats: sta​ tis​ tics col​ lec​ tion poli​ cies .

tags (group mul​ ti​ ple ob​ jects by a de​ scrip​ tive name) task: task ex​ e​ cu​ tion. sub​ ject. in​ stance.442 Appendix stat​ store: sta​ tis​ tics data hold​ ers storm​ c​ trl: storm con​ trol (traf​ fic sup​ pres​ sion) pol​ icy stp: span​ ning tree pro​ to​ col de​ f​ in ​i​ t​ ions and pol​ icy sts: Ser​ vice Tag Switch​ ing (used for ser​ vices in​ ser​ tion) svc​ core: core pol​ icy svi: switched vir​ tual in​ ter​ face/ routed VLAN in​ ter​ face syn​ thetic: syn​ thetic ob​ jects (for test​ ing) sys​ de​ bug: sys​ tem debug sys​ file: sys​ tem files syshist: sys​ tem cards reset records/his​ tory sys​ log: sys​ log pol​ icy sys​ mgr: sys​ tem man​ ager (firmware. etc) sys​ m​ grp: con​ tainer for cores pol​ icy & ab​ stract class for all qos pol​ icy de​ f​ in ​i​ t​ ions tag: alias (use de​ scrip​ tive name for dn). and re​ sult test: ab​ stract class for test rule. length. and re​ sult testin​ fralab: test in​ fra​ struc​ ture tlv: type. su​ per​ vi​ sor. value sys​ tem struc​ tures top: sys​ tem task man​ ager for proces​ sor ac​ tiv​ ity . sys​ tem states.

e. vmm pol​ icy and de​ f​ in ​i​ t​ ions) vns: vir​ tual net​ work ser​ vice (L4-L7 pol​ icy and de​ f​ in ​i​ t​ ions) vpc: vir​ tual port chan​ nel (vpc pol​ icy and de​ f​ in ​i​ t​ ions) vsvc: ser​ vice la​ bels (provider/con​ sumer) vtap: trans​ lated ad​ dress of ex​ ter​ nal node (NATed IP of ser​ vice node) vxlan: Vir​ tu​ ally ex​ ten​ si​ ble LAN de​ f​ in ​i​ t​ ions vz: vir​ tual zones (for​ mer name of the pol​ icy con​ trols) i.Appendix 443 topoc​ trl: topol​ ogy con​ trol pol​ icy (shard​ ing. fab​ ric LB. Con​ tracts Model Naming schemes Rs: Re​ la​ tion​ ship source Rt: Re​ la​ tion​ ship tar​ get Ag: Ag​ gre​ gated stats BrCP: Bi​ nary Con​ tract Pro​ file . etc) tracer​ oute: tracer​ oute ex​ e​ cu​ tion and re​ sults tracer​ outep: tracer​ oute end points trig: trig​ ger​ ing pol​ icy tun​ nel: tun​ nel​ ing uribv4: ipv4 uni​ cast rout​ ing in​ for​ ma​ tion base en​ tity vlan: vlan in​ stances vlan​ mgr: vlan man​ ager con​ trol plane vmm: vir​ tual macine man​ ager (con​ troller. fab​ ric VxLan.

.

and un​ der​ stand​ ing those new terms and con​ cepts will help those work​ ing on ACI com​ mu​ ni​ cate with one an​ other about the con​ structs used in ACI to trans​ mit those bits. Au​ tho​ riza​ tion.Appendix 445 Acronyms and Definitions Overview This sec​ tion is de​ signed to pro​ vide a high level de​ scrip​ tion of terms and con​ cepts that get brought up in this book. While ACI does not change how pack​ ets are trans​ mit​ ted on a wire. where end​ points fall out​ side of the man​ aged scope of the fab​ ric. This means that if one part of a trans​ ac​ tion fails the en​ tire trans​ ac​ tion fails. and Ac​ count​ ing. Dura​ bil​ ity – prop​ er​ ties of trans​ ac​ tions that en​ sure con​ sis​ tency in data​ base trans​ ac​ tions. As​ so​ ci​ ated new acronyms are also pro​ vided. ACI Ex​ ter​ nal Con​ nec​ tiv​ ity: Any con​ nec​ tiv​ ity to and from the fab​ ric that uses an ex​ ter​ nal routed or switched in​ ter​ me​ di​ ary sys​ tem. or which would be rel​ e​ vant to the trou​ bleshoot​ ing ex​ er​ cises that were cov​ ered in the trou​ bleshoot​ ing sce​ nar​ ios dis​ cussed. AEPs are also the mech​ an ​ism that ties the phys​ ic ​al port to the do​ main (phys​ ic ​al or vir​ tual) to a switch pol​ icy. An AEP rep​ re​ sents a group of ex​ ter​ nal en​ ti​ ties with sim​ il​ ar in​ fra​ struc​ ture pol​ icy re​ quire​ ments. Trans​ ac​ tions to APIC de​ vices in an ACI clus​ ter are con​ sid​ ered ACID. Iso​ la​ tion. only the key ones that may not be a part of the com​ mon ver​ nac​ ul​ ar. . This is not meant to be an ex​ haus​ tive list nor a com​ pletely de​ tailed dic​ tio​ nary of all of the terms and con​ cepts. AEP: At​ tach​ able En​ tity Pro​ file – this is a con​ fig​ ur​ a​ tion pro​ file of the in​ ter​ face that gets ap​ plied when an en​ tity at​ taches to the fab​ ric. A AAA: acronym for Au​ then​ ti​ ca​ tion. Con​ sis​ tency. there are some new terms and con​ cepts em​ ployed. ACID trans​ ac​ tions: ACID is an acronym for Atom​ ic​ ity. to en​ sure that data​ base con​ sis​ tency is main​ tained. ALE: Ap​ pli​ ca​ tion Leaf En​ gine. an ASIC on a leaf switch.

. ASE: Ap​ pli​ ca​ tion Spine En​ gine. the de​ fault for​ ward​ ing pol​ icy is to not allow any com​ mu​ ni​ ca​ tion be​ tween EPGs. ARP flood​ ing. Con​ tracts: A log​ ic ​al con​ tainer for the sub​ jects which re​ late to the fil​ ters that gov​ ern the rules for com​ mu​ ni​ ca​ tion be​ tween end​ point groups. C CLOS fab​ ric: A multi-tier non​ block​ ing leaf-spine ar​ chi​ tec​ ture net​ work. In the ACI ob​ ject model. Clus​ ter: Set of de​ vices that work to​ gether as a sin​ gle sys​ tem to pro​ vide an iden​ ti​ cal or sim​ il​ ar set of func​ tions. Ap​ pli​ ca​ tion Pro​ file: Term used to ref​ er​ ence an ap​ pli​ ca​ tion pro​ file-man​ aged ob​ ject ref​ er​ ence that mod​ els the log​ ic ​al com​ po​ nents of an ap​ pli​ ca​ tion and how those com​ po​ nents com​ mu​ ni​ cate. etc. Bridge Do​ mains are also a con​ tainer for IP sub​ nets and are where fab​ ric Layer 3 gate​ way func​ tion​ al​ ity is con​ fig​ ured. With​ out a con​ tract. The APIC con​ fig​ ures the in​ tended state of the pol​ icy to the fab​ ric. a BD is a child of a Pri​ vate Layer 3 or con​ text. an ASIC on a Spine switch. ACI works on a white list pol​ icy model. and In​ ter​ nal BGP is used to peer the fab​ ric with ex​ ter​ nal Layer 3 de​ vices.) for each unique Layer 2 for​ ward​ ing do​ main. but com​ mu​ ni​ ca​ tion within an EPG is al​ lowed.446 Appendix APIC: Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller is a cen​ tral​ ized pol​ icy man​ age​ ment con​ troller clus​ ter. The AP is the key ob​ ject used to rep​ re​ sent an ap​ pli​ ca​ tion and is also the an​ chor point for the au​ to​ mated in​ fra​ struc​ ture man​ age​ ment in an ACI fab​ ric. API: Ap​ pli​ ca​ tion Pro​ gram​ ming In​ ter​ face used for pro​ gram​ ma​ ble ex​ ten​ si​ bil​ ity. on the ACI fab​ ric Multi-Pro​ to​ col BGP is used to dis​ trib​ ute reach​ ab ​il​ ity in​ for​ ma​ tion within the fab​ ric. B BGP: Bor​ der Gate​ way Pro​ to​ col. Bridge Do​ main: An ACI con​ struct that de​ fines Layer 2 for​ ward​ ing be​ hav​ iors (Broad​ cast. BDs can em​ u​ late the be​ hav​ ior of a tra​ di​ tional VLAN but are not con​ strained by for​ ward​ ing scale lim​ i​ ta​ tions.

a ser​ vice that runs on the APIC that man​ ages data for the data model. DME: Data Man​ age​ ment En​ gine. the sys​ tem cre​ ates a fault-man​ aged ob​ ject for the fault. servers. A fault con​ tains the con​ di​ tions. dMIT: dis​ trib​ uted Man​ age​ ment In​ for​ ma​ tion Tree. etc. E EP: End​ point . End​ points have spe​ cific prop​ er​ ties like an ad​ dress. Dn: Dis​ tin​ guished name – a fully qual​ if​ ied name that rep​ re​ sents a spe​ cific ob​ ject within the ACI man​ age​ ment in​ for​ ma​ tion tree as well as the spe​ cific lo​ ca​ tion in​ for​ ma​ tion in the tree. D DLB: Dy​ namic Load Bal​ anc​ ing – a net​ work traf​ fic load-bal​ anc​ ing mech​ an ​ism in the ACI fab​ ric based on flowlet switch​ ing. named com​ merce work​ space within a Ten​ ant named Prod. if pol​ icy ob​ ject of type Ap​ pli​ ca​ tion Pro​ file is cre​ ated. A col​ lec​ tion of end​ points that can be grouped based on com​ mon re​ quire​ ments for a com​ mon pol​ icy.Appendix 447 Con​ text: A Layer 3 for​ ward​ ing do​ main. End​ point groups can be dy​ namic or sta​ tic. The tree con​ tains all as​ pects of the ob​ ject model that rep​ re​ sent an ACI fab​ ric. Ex​ am​ ples in​ clude vir​ tual-ma​ chines. equiv​ al​ ent to a VRF. in​ for​ ma​ tion about the op​ er​ at​ ional state of the af​ fected ob​ ject and po​ ten​ tial res​ ol​ u​ tions for the prob​ lem.Any log​ ic ​al or phys​ ic ​al de​ vice con​ nected di​ rectly or in​ di​ rectly to a port on a leaf switch that is not a fab​ ric fac​ ing port. lo​ ca​ tion. stor​ age de​ vices. . It is made up of a con​ cate​ na​ tion of all of the rel​ at​ ive names from it​ self back to the root of the tree. and in ACI ver​ nac​ ul​ ar a Pri​ vate Layer 3. a rep​ re​ sen​ ta​ tion of the ACI ob​ ject model with the root of the tree at the top and the leaves of the tree at the bot​ tom. or po​ ten​ tially some other at​ tribute. F Fault: When a fail​ ure oc​ curs or an alarm is raised. EPG: End Point Group. which is used to iden​ tify the end​ point. the dn would be ex​ pressed as uni/tn-Prod/ap-com​ merce​ work​ space. As an ex​ am​ ple.

a markup lan​ guage that fo​ cuses on the for​ mat​ ting of web pages.448 Appendix Fab​ ric: The col​ lec​ tive end​ points as​ so​ ci​ ated with an ACI so​ lu​ tion (Leaf. Flowlet switch​ ing: An op​ ti​ mized. H HTML: Hy​ per​ Text Markup Lan​ guage. Used for com​ mu​ ni​ ca​ tion be​ tween dif​ fer​ ent de​ vices on the ACI fab​ ric. I IFM: In​ tra-Fab​ ric Mes​ sages. Hy​ per​ vi​ sor in​ te​ gra​ tion: Ex​ ten​ sion of ACI Fab​ ric con​ nec​ tiv​ ity to a vir​ tual ma​ chine man​ ager to pro​ vide the APIC with a mech​ an ​ism for vir​ tual ma​ chine vis​ ib ​il​ ity and pol​ icy en​ force​ ment. and split​ ting traf​ fic across mul​ ti​ ple par​ al​ lel paths with​ out re​ quir​ ing packet re​ order​ ing. load-bal​ anc​ ing method​ ol​ ogy based on re​ search from MIT in 2004. the man​ age​ ment cat​ e​ gories Fil​ ters: Fil​ ters de​ fine the rules out​ lin​ ing the Layer 2 to layer 4 fields that will be matched by a con​ tract. Spine and Vir​ tual Switches plus APICs) fines net​ work man​ age​ ment tasks. se​ cu​ rity. mul​ ti​ path. con​ fig​ ur​ a​ tion. Flowlet Switch​ ing is a way to use TCP’s own bursty na​ ture to more ef​ fi​ ciently for​ ward TCP flows by dy​ nam​ ic ​ally split​ ting flows into flowlets. FCAPS is n acronym for FCAPS: The ISO model de​ fault. . per​ for​ mance. Hy​ per​ vi​ sor: Soft​ ware that ab​ stracts the hard​ ware on a host ma​ chine and al​ lows the host ma​ chine to run mul​ ti​ ple vir​ tual ma​ chines. G GUI: Graph​ ic ​al User In​ ter​ face. ac​ count​ ing.

. . DLP. Loop​ back and VTEP ad​ dresses are in​ ter​ nally ad​ ver​ tised over IS-IS. Leaf: Net​ work node in fab​ ric pro​ vid​ ing host and bor​ der con​ nec​ tiv​ ity. L4-L7 Ser​ vice In​ ser​ tion: The in​ ser​ tion and stitch​ ing of VLANs/Layer 3 con​ structs of vir​ tual or phys​ ic ​al ser​ vice ap​ pli​ ances (Fire​ wall. L Layer 2 Out (l2out): Layer 2 con​ nec​ tiv​ ity to an ex​ ter​ nal net​ work that ex​ ists out​ side of the ACI fab​ ric. where as net​ work​ ing el​ e​ ments (i..e. J JSON: JavaScript Ob​ ject No​ ta​ tion. IS-IS: Link local rout​ ing pro​ to​ col lever​ aged by the fab​ ric for in​ fra​ struc​ ture topol​ ogy. Leafs con​ nect only to hosts and spines. the fab​ ric) op​ er​ ate at lay​ ers 1-3). Ser​ vice nodes op​ er​ ate be​ tween Lay​ ers 4 and Layer 7 of the OSI model. IS-IS an​ nounces the cre​ ation of tun​ nels from leaf nodes to all other nodes in fab​ ric. Con​ nec​ tiv​ ity using an in​ band man​ age​ ment con​ fig​ ur​ a​ tion. This uses a front panel (data plane) port of a leaf switch for ex​ ter​ nal man​ age​ ment con​ nec​ tiv​ ity for the fab​ ric and APICs. etc. a data en​ cap​ su​ la​ tion for​ mat that uses human read​ able text to en​ cap​ su​ late data ob​ jects in at​ tribute and value pairs. Leafs never con​ nect to each other.) into the flow of traf​ fic. Layer 3 Out (l3out): Layer 3 con​ nec​ tiv​ ity to an ex​ ter​ nal net​ work that ex​ ists out​ side of the ACI fab​ ric.Appendix 449 In​ band Man​ age​ ment (INB): In​ band Man​ age​ ment. M MO: Man​ aged Ob​ ject – every con​ fig​ urable com​ po​ nent of the ACI pol​ icy model man​ aged in the MIT is called a MO. IDS/IPS. Load Bal​ ancers. La​ bels: Used for clas​ si​ fy​ ing which ob​ jects can and can​ not com​ mu​ ni​ cate with each other..

In ACI the ob​ ject model is rep​ re​ sented as a tree known as the dis​ trib​ uted man​ age​ ment in​ for​ ma​ tion tree (dMIT). or Rep​ re​ sen​ ta​ tional State Trans​ fer. The lo​ ca​ tion that the client ac​ cesses usu​ ally de​ fines the data the client is try​ ing to ac​ cess from the ser​ vice. then using those roles in the process of grant​ ing or deny​ ing ac​ cess to de​ vices. R RBAC: Role Based Ac​ cess Con​ trol. REST​ ful: An API that uses REST. Rep​ re​ sen​ ta​ tional State Trans​ fer (REST): a state​ less pro​ to​ col usu​ ally run over HTTP that al​ lows a client to ac​ cess a server-side or cloud-based API with​ out hav​ ing to write a local client for the host ac​ cess​ ing the API. Multi-tier Ap​ pli​ ca​ tion: Client–server ar​ chi​ tec​ ture in which pre​ sen​ ta​ tion. . which is a method of man​ ag​ ing se​ cure ac​ cess to in​ fra​ struc​ ture by as​ sign​ ing roles to users. Out-of-Band man​ age​ ment (OOB man​ age​ ment): Ex​ ter​ nal con​ nec​ tiv​ ity using a spe​ cific out-of-band man​ age​ ment in​ ter​ face on every switch and APIC. O Ob​ ject Model: A col​ lec​ tion of ob​ jects and classes are used to ex​ am​ ine and ma​ nip​ ul​ ate the con​ fig​ ur​ a​ tion and run​ ning state of the sys​ tem that is ex​ pos​ ing that ob​ ject model.450 Appendix Model: A model is a con​ cept which rep​ re​ sents en​ ti​ ties and the re​ la​ tion​ ships that exist be​ tween them. and data​ base man​ age​ ment func​ tions re​ quire phys​ ic ​al or log​ ic ​al sep​ ar​ a​ tion and re​ quire net​ work​ ing func​ tions to com​ mu​ ni​ cate with the other tiers for ap​ pli​ ca​ tion func​ tion​ al​ ity. P Port Chan​ nel: A Port link ag​ gre​ ga​ tion tech​ nol​ ogy that binds mul​ ti​ ple phys​ ic ​al in​ ter​ faces into a sin​ gle log​ ic ​al in​ ter​ face and pro​ vides more ag​ gre​ gate band​ width and link fail​ ure re​ cov​ ery with​ out a topol​ ogy change. Data is usu​ ally ac​ cessed and re​ turned in ei​ ther XML or JSON for​ mat. ob​ jects and priv​ il​ ege lev​ els. ap​ pli​ ca​ tion logic.

Appendix 451 Rn: Rel​ at​ ive name. S Ser​ vice graph: A mech​ an ​ism within ACI that au​ to​ mates redi​ rec​ tion of traf​ fic and VLAN stitch​ ing based on de​ fined pa​ ra​ me​ ters. Spine: Net​ work node in fab​ ric car​ ry​ ing ag​ gre​ gate host traf​ fic from leafs. but with​ out con​ text. Sub​ jects: Con​ tained by con​ tracts and cre​ ate the re​ la​ tion​ ship be​ tween fil​ ters and con​ tracts. a name of a spe​ cific ob​ ject within the ACI man​ age​ ment in​ for​ ma​ tion tree that is not fully qual​ if​ ied. See also the Dn de​ f​ in ​i​ t​ ion. and rep​ re​ sent each func​ tion as a node. As an ex​ am​ ple. a sub​ net de​ fines the IP ad​ dress range that can be used within the bridge do​ main. T Ten​ ants: The log​ ic ​al con​ tainer to group all poli​ cies for ap​ pli​ ca​ tion poli​ cies. Su​ per​ vi​ sor: Switch mod​ ule that pro​ vides the con​ trol plane for the 95xx switches. A Rn is sig​ nif​ ic ​ant to the in​ di​ vid​ ual ob​ ject. Spine/Leaf topol​ ogy: A clos-based fab​ ric topol​ ogy in which spine nodes con​ nect to leaf nodes. which then be​ comes use​ ful for nav​ ig ​a​ tion. A Rn would need to be con​ cate​ nated with all the rel​ at​ ive names from it​ self back up to the root to make a dis​ tin​ guished name. Any ser​ vices that are re​ quired are treated as a ser​ vice graph that is in​ stan​ ti​ ated on the ACI fab​ ric from the APIC. it’s not very use​ ful in nav​ ig ​a​ tion. if an Ap​ pli​ ca​ tion Pro​ file ob​ ject is cre​ ated named “com​ merce​ work​ space”. . con​ nected only to leafs in the fab​ ric and no other de​ vice types. leaf nodes con​ nect to hosts and ex​ ter​ nal net​ works. the Rn would be “ap-com​ merce​ work​ space” be​ cause Ap​ pli​ ca​ tion Pro​ file rel​ at​ ive names are all pref​ aced with the let​ ters “ap-”. Sub​ nets: Con​ tained by a bridge do​ main or an EPG. Ser​ vice graphs iden​ tify the set of net​ work or ser​ vice func​ tions that are needed by the ap​ pli​ ca​ tion.

X XML: eX​ ten​ si​ ble Markup Lan​ guage.A Layer 3 name​ space iso​ la​ tion method​ ol​ ogy to allow for mul​ ti​ ple con​ texts to be de​ ployed on a sin​ gle de​ vice or in​ fra​ struc​ ture. . VXLAN: VXLAN is a Layer 2 over​ lay scheme trans​ ported across a Layer 3 net​ work. A 24bit VXLAN seg​ ment ID (SID) or VXLAN net​ work iden​ ti​ fier (VNID) is in​ cluded in the en​ cap​ su​ la​ tion to pro​ vide up to 16 mil​ lion VXLAN seg​ ments for traf​ fic iso​ la​ tion or seg​ men​ ta​ tion. VRF: Vir​ tual Rout​ ing and For​ ward​ ing . An ACI VXLAN header is used to iden​ tify the pol​ icy at​ trib​ utes if the ap​ pli​ ca​ tion end​ point within the fab​ ric. and every packet car​ ries these pol​ icy at​ trib​ utes. in which a port chan​ nel is cre​ ated for link ag​ gre​ ga​ tion.452 Appendix V Vir​ tu​ al​ iza​ tion: Tech​ nol​ ogy used to ab​ stract hard​ ware re​ sources into vir​ tual rep​ re​ sen​ ta​ tions and al​ low​ ing soft​ ware con​ fig​ ura​ bil​ ity. vPC: vir​ tual Port Chan​ nel. but is spread across no more or less than 2 phys​ ic ​al switches. a markup lan​ guage that fo​ cuses on en​ cod​ ing data for doc​ um ​ents rather than the for​ mat​ ting of the data for those doc​ um ​ents. Each seg​ ment rep​ re​ sents a unique Layer 2 broad​ cast do​ main.

​ cisco.​ html .​ html#_​ Toc405844675 ACI Trou​ bleshoot​ ing Guides http://​ www.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-installation-guides-list.Appendix 453 Reference Material Top​ ics that are out​ side of the scope of this op​ er​ at​ ions guide may be doc​ um ​ented in other places.​ io/​ aci-troubleshooting-book/​ ACI White Pa​ pers http://​ www.​ html ACI Get​ ting Started .​ cisco.​ cisco.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-troubleshooting-guides-list.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-installation-and-configurationguides-list. ACI Install and Upgrade Guides http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c11-731960.​ github.​ cisco.​ html https://​ datacenter.​ html ACI De​ sign Guide http://​ www.Fab​ ric Ini​ tial​ iza​ tion http://​ www.​ cisco. This sec​ tion in​ cludes links to other help​ ful ref​ er​ ence doc​ um ​en​ ta​ tion for fur​ ther read​ ing and view​ ing.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ white-paper-listing.

​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ solution-overview-c22-732445.​ io/​ acitoolkit/​ ACI Com​ pata​ bil​ ity Tool http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ application-centricinfrastructure/​ customer-case-study-listing.​ cisco.​ cisco.​ html ACI So​ lu​ tions Overview http://​ www.​ cisco.​ cisco.​ html ACI Toolkit http://​ datacenter.​ github.​ cisco.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ application-centricinfrastructure/​ sales-resources-list.​ cisco.454 Appendix ACI Case Stud​ ies www.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ presentations-listings.​ html AVS Con​ fig​ u​ ra​ tion and Scal​ a​ bil​ ity Guides http://​ www.​ html .​ com/​ c/​ en/​ us/​ support/​ switches/​ application-virtual-switch/​ products-installation-and-configuration-guides-list. Pre​ sen​ ta​ tions and Train​ ings www.​ com/​ web/​ techdoc/​ aci/​ acimatrix/​ matrix.​ html ACI Ecosys​ tem Com​ pata​ bil​ ity List http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ solution-overview-listing.​ html ACI Part​ ners and Cus​ tomers Pre​ sen​ ta​ tions http://​ www.​ html ACI Demos.

​ html APIC Com​ mand-Line In​ ter​ face User Guide http://​ www.​ org/​ en/​ latest/​ Cobra GitHub http://​ github.​ com/​ watch?​ v=_​ iQvoC9zQ_​ A Nexus CLI to Cisco APIC Map​ ping http://​ www.​ html Cobra Docs http://​ cobra.Appendix 455 AVS Topolo​ gies and So​ lu​ tion Guide http://​ www.​ html APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide http://​ www.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-command-reference-list.​ cisco.​ com/​ c/​ en/​ us/​ support/​ switches/​ application-virtual-switch/​ products-technical-reference-list.​ readthedocs.​ html Fab​ ric Con​ nec​ tiv​ ity Video https://​ www.​ com/​ datacenter/​ cobra Con​ nect​ ing ACI to Out​ side Layer 2 and 3 Net​ works http://​ www.​ cisco.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy.​ youtube.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-configuration-examples-list.​ html .​ cisco.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c07-732033.​ cisco.

456 Appendix POST​ man getpostman.​ html .​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.​ Sup​ ported SNMP MIB List http://​ www.​ cisco.​ com http://www.