You are on page 1of 466

Operating Cisco

Application Centric
Infrastructure

Alejandra Sanchez, Andres Vega, Arvind Chari, Carly Stoughton,
Christopher Stokes, Gabriel Fontenot, Jonathan Cornell, Ken Fee,
Kevin Corbin, Lauren Malhoit, Loy Evans, Lucien Avramov,
Paul Lesiak, Steven Lym, Rafael Muller, Robert Burns

Operating Cisco
Application Centric
Infrastructure

Copyright
Op​
er​
at​
ing Cisco Ap​
pli​
ca​
tion Cen​
tric In​
fra​
struc​
ture
Ale​
jan​
dra Sanchez, An​
dres Vega, Arvind Chari, Carly Stoughton, Christo​
pher Stokes,
Gabriel Fontenot, Jonathan Cor​
nell, Ken Fee, Kevin Corbin, Lau​
ren Mal​
hoit, Loy Evans,
Lu​
cien Avramov, Paul Lesiak, Rafael Muller, Robert Burns. Steven Lym
Copy​
right © 2015 Cisco Sys​
tems, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this
file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or
agreed to in writing, software distributed under the License is distributed on an "AS IS"
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing permissions and
limitations under the License.
Cisco Press logo is a trade​
mark of Cisco Sys​
tems, Inc.
Pri​
vately pub​
lished by Cisco Sys​
tems, Inc.

Warning and Disclaimer
This book is de​
signed to pro​
vide in​
for​
ma​
tion about Cisco ACI. Every ef​
fort has been
made to make this book as com​
plete and as ac​
cu​
rate as pos​
si​
ble, but no war​
ranty or
fit​
ness is im​
plied.
The in​
for​
ma​
tion is pro​
vided on an “as is” basis. The au​
thors, and Cisco Sys​
tems, Inc.
shall have nei​
ther li​
ab
​il​
ity nor re​
spon​
si​
bil​
ity to any per​
son or en​
tity with re​
spect to any
loss or dam​
ages aris​
ing from the in​
for​
ma​
tion con​
tained in this doc​
um
​ent.
The opin​
ions ex​
pressed in this book be​
long to the au​
thor and are not nec​
es​
sar​
ily those
of Cisco Sys​
tems, Inc.

Feed​
back In​
for​
ma​
tion
Read​
ers’ feed​
back is a nat​
ural con​
tin​
ua
​t​
ion of this process. If you have any com​
ments
re​
gard​
ing how we could im​
prove the qual​
ity of this book, or oth​
er​
wise alter it to bet​
ter
suit your needs, you can contact
​ us through email at ops-booksprint@​
cisco.​
com . ​

Contents

. . . . . Prologue
........................................................................

1

. . . . . Abstract
...................................................................

3

. . . . . Authors
...................................................................

5

. . . . . Book
. . . . . Writing
. . . . . . . .Methodology
......................................................

7

. . . . . Hardware
. . . . . . . . . .and
. . . .Software
. . . . . . . . .Included
. . . . . . . . in
. . the
. . . .Book
..............................

9

. . . . . Introduction
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
. . . . . The
. . . . Story
. . . . . .of
. . ACME
. . . . . . Inc.
.................................................

13

. . . . . The
. . . . Why,
. . . . . .Who,
. . . . .What,
. . . . . .When
. . . . . .and
. . . .How
....................................

15

. . . . . Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
. . . . . APIC
. . . . . Overview
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
. . . . . Configuring
. . . . . . . . . . . .Management
. . . . . . . . . . . . Protocols
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
. . . . . Role-Based
. . . . . . . . . . . Access
. . . . . . .Control
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
. . . . . Import
. . . . . . . and
. . . . Export
. . . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

. . . . . Upgrading
. . . . . . . . . . .and
. . . Downgrading
. . . . . . . . . . . . . Firmware
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
. . . . . Section
. . . . . . . .Content
...........................................................

51

. . . . . Firmware
. . . . . . . . . .Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
. . . . . Upgrading
. . . . . . . . . . .and
. . . Downgrading
. . . . . . . . . . . . . Considerations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
. . . . . Upgrading
. . . . . . . . . . .the
. . . Fabric
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

. . . . . Fabric
. . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
. . . . . Section
. . . . . . . .Content
...........................................................

71

. . . . . Understanding
. . . . . . . . . . . . . . .Fabric
. . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
. . . . . Adding
. . . . . . . New
. . . . .Devices
. . . . . . . .to
. .the
. . . .Fabric
.........................................

81

. . . . . Server
. . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
. . . . . Virtual
. . . . . . . Machine
. . . . . . . . Networking
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
. . . . . Deploying
. . . . . . . . . . the
. . . .Application
. . . . . . . . . . .Virtual
. . . . . . Switch
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
. . . . . External
. . . . . . . . .Connectivity
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
. . . . . Application
. . . . . . . . . . . Migration
. . . . . . . . . .Use
. . . .Case
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

. . . . . Tenants
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
. . . . . ACI
. . . . Tenancy
. . . . . . . . .Models
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
. . . . . Application
. . . . . . . . . . . Profile
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
. . . . . Endpoint
. . . . . . . . . Group
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
. . . . . Endpoint
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

. . . . . Private
. . . . . . . Network
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
. . . . . Bridge
. . . . . . .Domain
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
. . . . . Tenant
. . . . . . . Networking
. . . . . . . . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

. . . . . Working
. . . . . . . . .with
. . . . Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
. . . . . Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
. . . . . Filters
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
. . . . . Taboo
. . . . . . .Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
. . . . . Inter-Tenant
. . . . . . . . . . . . .Contracts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
. . . . . Contracts
. . . . . . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

. . . . . Layer
. . . . . .4. .to
. .Layer
. . . . . .7. Services
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .249
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
. . . . . Understanding
. . . . . . . . . . . . . . .Layer
. . . . . 4. .to
. . Layer
. . . . . .7. Integration
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
. . . . . Services
. . . . . . . . .Deployment
. . . . . . . . . . . Guide
. . . . . . Reference
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
. . . . . Service
. . . . . . . .Graph
. . . . . .Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
. . . . . ASAv
. . . . . Sample
. . . . . . . .Configuration
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

. . . . . Health
. . . . . . .Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .269
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
. . . . . Understanding
. . . . . . . . . . . . . . .Health
. . . . . . Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
. . . . . Understanding
. . . . . . . . . . . . . . .Faults
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
. . . . . How
. . . . . Health
. . . . . . .Scores
. . . . . . Are
. . . . Calculated
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
. . . . . Health
. . . . . . .Score
. . . . . .Use
. . . .Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

. . . . . Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . -. .Tenant
. . . . . . .and
. . . .Fabric
. . . . . .Policies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . -. .Infrastructure
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
. . . . . Proactive
. . . . . . . . . .Monitoring
. . . . . . . . . . Use
. . . . Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . Tools
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
. . . . . Reactive
. . . . . . . . .Monitoring
. . . . . . . . . . Use
. . . . Cases
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

. . . . . Scripting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .349
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
. . . . . Leveraging
. . . . . . . . . . .Network
. . . . . . . . Programmability
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
. . . . . ACI
. . . . and
. . . . Scripting
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
. . . . . API
. . . .Inspector
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
. . . . . Development
. . . . . . . . . . . . . Techniques
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

. . . . . STman
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
. . . . . Cobra
. . . . . . SDK
. . . . .and
. . . .Arya
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
. . . . . ACI
. . . . Toolkit
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
. . . . . GitHub
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

. . . . . Hardware
. . . . . . . . . .Expansion
. . . . . . . . . .and
. . . .Replacement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .389
. . . . . Section
. . . . . . . .Content
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
. . . . . Expanding
. . . . . . . . . . .and
. . . Shrinking
. . . . . . . . . .the
. . . Fabric
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
. . . . . Hardware
. . . . . . . . . .Diagnostics
. . . . . . . . . . . and
. . . .Replacement
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

. . . . . Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .413
. . . . . Classes
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
. . . . . Package
. . . . . . . . Decoder
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
. . . . . Acronyms
. . . . . . . . . . and
. . . .Definitions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
. . . . . Reference
. . . . . . . . . . Material
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
....

1 Prologue .

.

Hav​ ing the tools to change how in​ fra​ struc​ ture is built is one thing. method​ olo​ gies. but being able to ef​ fec​ tively op​ er​ ate the in​ fra​ struc​ ture be​ yond the Day Zero build ac​ tiv​ i​ ties is cru​ cial to long term ef​ fec​ tive​ ness and ef​ fi​ ciency. To ef​ fec​ tively har​ ness the power of ACI. .Prologue 3 Abstract Cisco's Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) pro​ vides pow​ er​ ful new ways to dy​ nam​ i​ cally man​ age in​ fra​ struc​ ture in the mod​ ern world of IT au​ toma​ tion and De​ vOps. and processes that can be em​ ployed to sup​ port day 1+ op​ er​ a​ tions within an ACI-based fab​ ric. or​ ga​ ni​ za​ tions will need to un​ der​ stand how to in​ cor​ po​ rate ACI into their daily op​ er​ a​ tions. This book ex​ am​ ines some of the com​ mon op​ er​ a​ tional ac​ tiv​ i​ ties that IT teams use to pro​ vide con​ tin​ ued in​ fra​ struc​ ture op​ er​ a​ tions and gives the reader ex​ po​ sure to the tools.

.

Tech​ ni​ cal So​ lu​ tions Ar​ chi​ tect. Authors Ale​ jan​ dra Sanchez . Tech​ ni​ cal Ser​ vices.So​ lu​ tions Ar​ chi​ tect.Cus​ tomer Sup​ port En​ gi​ neer. Sales Lu​ cien Avramov . Ad​ vanced Ser​ vices Jonathan Cor​ nell . Sys​ tems En​ gi​ neer​ ing Lau​ ren Mal​ hoit .Con​ sult​ ing Sys​ tem En​ gi​ neer. Sales Ken Fee . So​ lu​ tion Val​ id ​a​ tion Test​ ing.Tech​ ni​ cal Writer.So​ lu​ tions Ar​ chi​ tect.Net​ work Con​ sult​ ing En​ gi​ neer.Prologue 5 Authors This book rep​ re​ sents a joint in​ tense col​ lab​ or​ a​ tive ef​ fort over the course of a week. INSBU Christo​ pher Stokes . INSBU . IT.Tech​ ni​ cal Leader. Tech​ ni​ cal Ser​ vices Arvind Chari .Sys​ tems En​ gi​ neer. Ad​ vanced Ser​ vices Rafael Muller . So​ lu​ tion Val​ id ​a​ tion Ser​ vices Robert Burns . INSBU Loy Evans . Tech​ ni​ cal Ser​ vices Steven Lym . Busi​ ness Tech​ nol​ ogy Ar​ chi​ tects Kevin Corbin . INSBU Paul Lesiak . with par​ tic​ ip ​a​ tion of a num​ ber of Cisco func​ tional or​ ga​ ni​ za​ tions in​ clud​ ing Cisco Ad​ vanced Ser​ vices. and Sales.Tech​ ni​ cal Mar​ ket​ ing En​ gi​ neer.Tech​ ni​ cal Mar​ ket​ ing En​ gi​ neer.Tech​ ni​ cal Leader.Mar​ ket​ ing Con​ sul​ tant. Tech​ ni​ cal Ser​ vices An​ dres Vega .Con​ sult​ ing Ar​ chi​ tect.Cisco IT Net​ work Con​ sult​ ing En​ gi​ neer Gabriel Fontenot .Cus​ tomer Sup​ port En​ gi​ neer. Prod​ uct Mar​ ket​ ing. Ad​ vanced Ser​ vices Carly Stoughton .

6 Prologue Book Sprint Facilitation Bar​ bara Rühling Faith Bosworth Illustrations Hen​ rik van Leuwen Book Production Julien Taquet Clean Up Raewyn Whyte .

.Prologue 7 Book Writing Methodology The Book Sprint (booksprints. the com​ plete book was writ​ ten in only five days. and in​ cor​ po​ rate it into a com​ plete nar​ ra​ tive in a short amount of time. By lever​ ag​ ing the input of many ex​ perts. Book Sprints are strongly fa​ cil​ it​ ated and lever​ age team-ori​ ented in​ spi​ ra​ tion and mo​ ti​ va​ tion to rapidly de​ liver large amounts of well-au​ thored and re​ viewed con​ tent. The Book Sprint method​ ol​ ogy is an in​ no​ v​ at​ ive new style of co​ op​ er​ at​ ive and col​ lab​ o​ ra​ tive book pro​ duc​ tion. al​ low​ ing for ex​ tremely high qual​ ity in a very short pro​ duc​ tion time pe​ riod. and in​ cluded thou​ sands of ex​ pe​ ri​ enced en​ gi​ neer​ ing hours. but in​ volved hun​ dreds of au​ thor​ ing per​ son-hours.​ net) method​ ol​ ogy was used for writ​ ing this book.

.

the fol​ low​ ing hard​ ware de​ vices were used: • Cisco Application Policy Infrastructure Controller (APIC) • ACI Spine switches.Prologue 9 Hardware and Software Included in the Book The Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) com​ bines hard​ ware. including KVM. ap​ pli​ ca​ tion. and 93128TX • Cisco Application Virtual Switch (AVS) • Cisco UCS B and C series servers • Several models of switches and routers. For the pur​ pose of writ​ ing this book. . including Cisco Nexus 5000 switches and Cisco Integrated Services Routers (ISR) • A variety of hypervisors. 9396TX.1 soft​ ware re​ lease. Microsoft Hyper-V. soft​ ware. and vir​ tu​ al​ iza​ tion. se​ cu​ rity. including Cisco Nexus 9508 and 9336PQ • ACI Leaf switches. and ASIC in​ no​ va​ tions into an in​ te​ grated sys​ tems ap​ proach and pro​ vides a com​ mon man​ age​ ment frame​ work for net​ work. including Cisco Nexus 9396PX. and VMware • IXIA IxVM product family traffic generators vSphere This book was writ​ ten based on the Cisco ACI 1.

.

11 Introduction .

.

caused cus​ tomer-im​ pact​ ing out​ ages. and the new ap​ pli​ ca​ tion plat​ form is to be de​ vel​ oped in this man​ ner. To sup​ port this.Introduction 13 The Story of ACME Inc. and taken longer to ac​ com​ plish than the busi​ ness lead​ ers would have liked. Where they have used cus​ tom soft​ ware in the past. can be tested along​ side the ac​ tual ap​ pli​ ca​ tion. can be checked into ver​ sion con​ trol. while in​ cor​ po​ rat​ ing an on​ go​ ing im​ prove​ ment cycle so they can react to chang​ ing mar​ ket dy​ nam​ ics in a more nim​ ble fash​ ion. is a multi​ na​ tional cor​ po​ ra​ tion that spe​ cial​ izes in man​ uf​ ac​ tur​ ing. ACME Inc. and have pre​ vi​ ously main​ tained sep​ ar​ ate in​ fra​ struc​ ture and ap​ pli​ ca​ tions. These prod​ uct groups op​ er​ ate as sep​ ar​ ate busi​ ness groups within the com​ pany. the in​ fra​ struc​ ture com​ po​ nents need to be ca​ pa​ ble of map​ ping to these new par​ a-​ digms in a way that is not pos​ si​ ble using tra​ di​ tional con​ cepts. ACME busi​ ness units have lever​ aged third party soft​ ware com​ pa​ nies and com​ mer​ cially avail​ able soft​ ware to meet their IT de​ mands. and there​ fore ACME is look​ ing for a new ap​ proach to both ap​ pli​ ca​ tion and in​ fra​ struc​ ture life​ cy​ cle man​ age​ ment. ACME has un​ der​ taken a pro​ ject to build a mo​ bile ap​ pli​ ca​ tion plat​ form to sup​ port or​ der​ ing and lo​ gis​ tics for prod​ uct de​ liv​ ery to their cus​ tomers for their en​ tire port​ fo​ lio. . sales. ACME Inc. and var​ io ​us ex​ plo​ sive ma​ te​ ri​ als. For this rea​ son. jetpro​ pelled uni​ cy​ cles. Tra​ di​ tion​ ally. The ap​ pli​ ca​ tion de​ vel​ op​ ers have been look​ ing at new ap​ pli​ ca​ tion de​ vel​ op​ ment trends such as Con​ tin​ uo ​us De​ liv​ ery and Con​ tin​ uo ​us In​ te​ gra​ tion. has de​ cided to change by cre​ at​ ing an en​ vi​ ron​ ment where in​ fra​ struc​ ture ar​ ti​ facts are treated as part of the ap​ pli​ ca​ tion. but would like to cre​ ate a more in​ ti​ mate re​ la​ tion​ ship with their con​ sumers and be able to take feed​ back on the plat​ form di​ rectly from those users. but have re​ cently de​ cided to pur​ sue a more di​ rect-to-con​ sumer busi​ ness model due to in​ tense pres​ sure from new com​ peti​ tors who have dom​ in ​ated the on​ line sales chan​ nels. in​ clud​ ing rocket-pow​ ered roller skates. They have largely fo​ cused on re​ tail routes to mar​ ket. they have lever​ aged a tra​ di​ tional in​ fra​ struc​ ture and soft​ ware model that does not allow them to keep up with the chang​ ing re​ quire​ ments. This has led to sev​ eral sit​ u​ at​ ions where ap​ pli​ ca​ tion de​ ploy​ ments have meant long week​ end hours for all of the teams. and dis​ tri​ bu ​t​ ion of a di​ verse prod​ uct port​ fo​ lio. and can con​ tin​ ua ​lly im​ prove. One of the largest chal​ lenges ACME has his​ tor​ ic ​ally faced is that op​ er​ at​ ions and in​ fra​ struc​ ture has been an af​ ter​ thought to prod​ uct de​ vel​ op​ ment. In an ef​ fort to be more com​ pet​ it​ ive.

This is the type of con​ sump​ tion that ACME Inc. this runs con​ trary to how most op​ er​ at​ ions groups exist in the re​ la​ tion​ ship be​ tween busi​ ness and tech​ nol​ ogy. ACME Inc. Every com​ pany is a tech​ nol​ ogy com​ pany. In​ no​ va​ tion at this level will pro​ vide more op​ por​ tu​ ni​ ties for ex​ pand​ ing the tools with which users in​ ter​ act. con​ sump​ tion mod​ els of tech​ nol​ ogy de​ liv​ ery have the abil​ ity to adapt tech​ nol​ ogy more quickly to rapid busi​ ness re​ quire​ ments changes. is a fic​ ti​ tious com​ pany. ACME is also in​ ter​ ested in cre​ at​ ing a foun​ da​ tion on which it can grow to de​ liver a com​ mon pool of in​ fra​ struc​ ture that is shared across all busi​ ness groups and op​ er​ ated in a multi-ten​ ant fash​ ion to in​ crease ef​ fi​ ciency. and if you don't adapt. After an​ al​ yz​ ing cur​ rent of​ fers from var​ io ​us tech​ nol​ ogy ven​ dors. as well as portable across the var​ io ​us data cen​ ter lo​ ca​ tions cur​ rently main​ tained by ACME. is highly de​ sir​ able. While ACME Inc. John Cham​ bers. How​ ever. thus al​ low​ ing IT to op​ er​ ate and man​ age at the speed of busi​ ness. the CEO of Cisco Sys​ tems. This is where the ful​ crum will tilt in the favor of IT and in​ fra​ struc​ ture being more dy​ namic. you'll get left be​ hind. but con​ trol can be a bar​ rier to a pure con​ sump​ tion model. and doubt. test. Con​ trol of op​ er​ at​ ions is what op​ er​ at​ ions groups are fo​ cused on. this is the true story of every com​ pany.'s busi​ ness own​ ers need. told ACME: "The world is chang​ ing.14 Introduction While ACME is in​ tensely fo​ cused on de​ liv​ er​ ing the new ap​ pli​ ca​ tion plat​ form in a timely man​ ner." As ev​ id ​enced by the suc​ cess of cloud plat​ forms. such as Ama​ zon Web Ser​ vices (AWS) and Open​ stack. un​ cer​ tainty. se​ lected Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI). and prod en​ vi​ ron​ ments. This book will at​ tempt to bring some level of com​ fort and fa​ mil​ iar​ ity with op​ er​ at​ ions ac​ tiv​ i-​ ties within an ACI fab​ ric. the only other way to scale is by break​ ing the human level com​ po​ nent. Work​ ers in the IT in​ dus​ try need to adapt to keep up with the rapid change of the busi​ ness. The abil​ ity to ab​ stract all phys​ ic ​al and vir​ tual in​ fra​ struc​ ture con​ fig​ ur​ a​ tion into a sin​ gle con​ fig​ ur​ a​ tion that is con​ sis​ tent across dev. and just im​ por​ tant this is the story of the em​ ploy​ ees of those com​ pa​ nies. Un​ less com​ pa​ nies make in​ vest​ ments in tech​ nolo​ gies that allow for con​ sump​ tion of au​ to​ mated com​ po​ nents. ACI has been built from the ground up to change the sub​ struc​ ture used to build net​ work de​ vices and pro​ to​ cols. and few peo​ ple would re​ ally choose to work for that type of com​ pany. "Why fix what is al​ ready work​ ing?" . The thought is. At an ex​ ec​ ut​ ive brief​ ing. How​ ever. Most IT op​ er​ at​ ions groups in​ vest a lot of time in the tools needed to de​ liver ser​ vices today and there is an or​ ganic re​ sis​ tance to re-in​ vest​ ing. with a change of this na​ ture comes fear.

Au​ toma​ tion adds speed to repet​ it​ ive tasks and elim​ in ​ates er​ rors or missed steps. ACME must ad​ dress fun​ da​ men​ tal ques​ tions in​ clud​ ing Who man​ ages What. these groups can have dis​ tinct or​ ga​ ni​ za​ tional bound​ aries. When and How Within ACI. tra​ di​ tion​ ally had dif​ fer​ ent types of stake​ hold​ ers in​ volved in mak​ ing any IT ini​ tia​ tive suc​ cess​ ful. au​ toma​ tion can be a scary propo​ si​ tion for some of the key stake​ hold​ ers. and How they go about their tasks. the key suc​ cess cri​ te​ ria is to stream​ line processes and pro​ ce​ dures re​ lated to the de​ ploy​ ment of in​ fra​ struc​ ture re​ quired to sup​ port the ap​ pli​ ca​ tion ini​ tia​ tives. a high de​ gree of au​ toma​ tion is re​ quired. Who As with most or​ ga​ ni​ za​ tions. and each has a spe​ cific el​ e​ ment of the in​ fra​ struc​ ture in which they have spe​ cific ex​ per​ tise and about which they care most. At the macro level. and this in​ fra​ struc​ ture are new to ACME Inc. but rec​ og​ nizes that this ini​ tia​ tive. but more tac​ ti​ cal and pointin-time-rel​ e​ vant. Who. ACME is look​ ing to sim​ plify how it op​ er​ ates in​ fra​ struc​ ture. Quite the op​ po​ site. What. the fact that these dif​ fer​ ent or​ ga​ ni​ za​ tions exist should . as it could be looked at as a threat to their own job. In the case of ACME Inc. Also. and Where they go to per​ form these op​ er​ at​ ions. this ap​ pli​ ca​ tion. When dif​ fer​ ent groups per​ form reg​ ul​ ar op​ er​ at​ ions.Introduction 15 The Why. are also con​ sid​ er​ at​ ions. Ini​ tially. but more likely the bound​ aries are blurred to some de​ gree. Look​ ing at why an au​ to​ mated fab​ ric is ben​ e​ fi​ cial to an or​ ga​ ni​ za​ tion is im​ por​ tant for set​ ting ex​ pec​ ta​ tions of re​ turn on in​ vest​ ment. ACME Inc. but keep in mind that some of these char​ ac​ ter​ is​ tics might be com​ bined. repet​ it​ ive tasks. Listed below are some char​ ac​ ter​ is​ tics of these groups. au​ toma​ tion is about mak​ ing work more en​ joy​ able for all team mem​ bers. al​ low​ ing them the free​ dom to in​ no​ vate and add value. Why "Why" is the most im​ por​ tant as​ pect of what should be con​ sid​ ered in op​ er​ at​ ional​ iz​ ing an ACI fab​ ric. In any IT or​ ga​ ni​ za​ tion. This sec​ tion dis​ cusses the rel​ e​ vant as​ pects of these monikers as it re​ lates to ACI fab​ ric op​ er​ at​ ions and how a com​ pany such as ACME Inc. while re​ mov​ ing mun​ dane. To achieve the de​ sired re​ sult. look​ ing at why an op​ er​ at​ ional prac​ tice is done a spe​ cific way can help with fram​ ing the tools and processes that are em​ ployed. can di​ vide the work​ load.

The stor​ age team is con​ cerned with pro​ tect​ ing the data in terms of avail​ abil​ ity. With ACI. as well as mak​ ing sure that sen​ si​ tive data is se​ cure. the team had to worry about de​ liv​ er​ ing a stor​ age fab​ ric in ad​ di​ tion to man​ ag​ ing stor​ age de​ vices them​ selves. and as​ sist​ ing in the en​ force​ ment of in​ for​ ma​ tion se​ cu​ rity. while al​ low​ ing other groups to lever​ age their spe​ cific ex​ per​ tise to ma​ nip​ ul​ ate se​ cu​ rity and ap​ pli​ ca​ tion level poli​ cies. The De​ vel​ op​ ment part of the team will be writ​ ing the mo​ bile ap​ pli​ ca​ tion soft​ ware plat​ form. ACME's Net​ work Team is pri​ mar​ ily fo​ cused on cre​ at​ ing and man​ ag​ ing net​ work​ ing con​ structs to for​ ward pack​ ets at Layer 2 (MAC/switch​ ing) and Layer 3 (IP rout​ ing). The ca​ pa​ bil​ it​ ies pro​ vided by the ACI fab​ ric allow them to con​ fi​ dently de​ ploy newer IP-based stor​ age and clus​ ter​ ing tech​ nolo​ gies. In​ stead. man​ ag​ ing SLA. What the team needs to know is how to con​ fig​ ure the net​ work​ ing con​ structs. ACME's Stor​ age Team is pri​ mar​ ily fo​ cused on de​ liv​ ery of data stor​ age re​ sources to the or​ ga​ ni​ za​ tion. ACI will pro​ vide the stor​ age team with the vis​ ib ​il​ ity they will re​ quire. how to tie Layer 2 to Layer 3. The team is also in​ ter​ ested in al​ low​ ing more trans​ parency in the per​ for​ mance of the net​ work for​ ward​ ing. Both parts of this team will need to work closely with the other teams in this sec​ tion to en​ able the best de​ sign. The team is also very in​ ter​ ested in being able to see how the stor​ age ac​ cess is per​ form​ ing and would like to be no​ ti​ fied in the event of con​ tention. The Ap​ pli​ ca​ tion part of the team con​ tains ap​ pli​ ca​ tion own​ ers and sub​ ject mat​ ter ex​ perts that en​ sure other busi​ ness units are able to do their jobs by uti​ liz​ ing the busi​ ness ap​ pli​ ca​ tions avail​ able. per​ for​ mance. and avail​ abil​ ity of ap​ pli​ ca​ tions for the end users. ACME's De​ vel​ op​ ment and Ap​ pli​ ca​ tion Team is fo​ cused on the soft​ ware and ap​ pli​ ca​ tions the com​ pany uses in​ ter​ nally and the soft​ ware that it de​ liv​ ers to its cus​ tomers. the team is the most in​ ter​ ested in de​ cou​ pling over​ loaded net​ work con​ structs and re​ turn​ ing to the spe​ cific net​ work prob​ lems that the team was in​ tended to solve. The stor​ age team has been very suc​ cess​ ful in main​ tain​ ing very tight SLAs and has tra​ di​ tion​ ally man​ aged sep​ ar​ ate in​ fra​ struc​ ture for stor​ age ac​ cess. These ca​ pa​ bil​ it​ ies are pri​ mar​ ily dis​ cussed in the mon​ it​ or​ ing sec​ tions. all while main​ tain​ ing high lev​ els of avail​ abil​ ity. The team is chal​ lenged with jug​ gling ap​ pli​ ca​ tion re​ quire​ ments. and the team is mak​ ing key met​ rics avail​ able on de​ mand in a self-ser​ vice ca​ pac​ ity. . and so on. His​ tor​ ic ​ally. the en​ tire or​ ga​ ni​ za​ tion should be seen as one team with a com​ mon goal of de​ liv​ er​ ing value to their or​ ga​ ni​ za​ tion. The team typ​ ic ​ally has some spe​ cific re​ quire​ ments around QoS. and how to trou​ bleshoot net​ work for​ ward​ ing as​ pects in the fab​ ric. multi-pathing. how to ver​ ify for​ ward​ ing.16 Introduction not be ev​ id ​ent to the end-user.

Role Based Ac​ cess Con​ trol (RBAC). to man​ age net​ work op​ er​ at​ ions. For ex​ am​ ple. The Com​ pute and Vir​ tu​ al​ iza​ tion teams are in​ ter​ ested in ACI for its abil​ ity to pro​ vide com​ mon ac​ cess to phys​ ic ​al and vir​ tual servers. Ad​ di​ tion​ ally. CLI. This is timely as the ap​ pli​ ca​ tion roll​ out will have both vir​ tu​ al​ ized and non-vir​ tu​ al​ ized work​ loads. These ca​ pa​ bil​ it​ ies are dis​ cussed fur​ ther in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter. Due to the sen​ si​ tiv​ ity of this in​ for​ ma​ tion and the se​ cu​ rity as​ pects of the ACI fab​ ric. With this base. What The as​ pect of "what" can be looked at in many dif​ fer​ ent ways. and Layer 4 to Layer 7 ser​ vices. if an op​ er​ at​ ions staff is try​ ing to gather a bunch of in​ for​ ma​ tion across a num​ ber of in​ ter​ faces and switches or is man​ ag​ ing the con​ fig​ ur​ a​ - . but the fab​ ric man​ age​ ment is rooted in an ab​ stracted ob​ ject model that pro​ vides a more flex​ ib ​le base. you have some tra​ di​ tional tools. and these tools in​ te​ grate into man​ age​ ment plat​ forms and con​ fig​ ur​ a​ tion and man​ age​ ment processes. script​ ing. How a tool is se​ lected in ACI will often be a prod​ uct of what is being done and the as​ pects of how the tool is used.Introduction 17 The Com​ pute and Vir​ tu​ al​ iza​ tion Team at ACME Inc. the ap​ pli​ ca​ tion de​ vel​ op​ ers are in​ creas​ ingly in​ ter​ ested in lever​ ag​ ing Linux con​ tainer tech​ nolo​ gies to allow for even greater ap​ pli​ ca​ tion porta​ bil​ ity. such as CLI and SNMP. but the main con​ cept in the con​ text of this book is what tools are used to man​ age op​ er​ at​ ions of an ACI fab​ ric. is wrap​ ping up a major ini​ tia​ tive to vir​ tu​ al​ ize the server farms that it is re​ spon​ si​ ble for main​ tain​ ing. the new ap​ pli​ ca​ tion will be stor​ ing sen​ si​ tive cus​ tomer in​ for​ ma​ tion. With the cur​ rent pro​ ject. the In​ for​ ma​ tion Se​ cu​ rity Team is able to pro​ vide input ear​ lier in the process and avoid re-do​ ing work be​ cause of se​ cu​ rity or com​ pli​ ance is​ sues. pro​ gram​ ming. and has been re​ spon​ si​ ble for per​ form​ ing vul​ ner​ ab ​il​ ity as​ sess​ ment and data clas​ si​ fi​ ca​ tion ef​ forts. The In​ for​ ma​ tion Se​ cu​ rity Team at ACME Inc. such as GUI. mon​ it​ or​ ing. The team also re​ cently em​ ployed new con​ fig​ ur​ a​ tion man​ age​ ment tools to ac​ count for new work​ loads that fell out​ side of the vir​ tu​ al​ iza​ tion ef​ fort to get sim​ il​ ar agility for bare metal servers that the team gained from its vir​ tu​ al​ iza​ tion ef​ forts. the op​ er​ at​ or of the fab​ ric can choose from mul​ ti​ ple modes of man​ age​ ment. has tra​ di​ tion​ ally been en​ gaged late in an ap​ pli​ ca​ tion de​ ploy​ ment process. The In​ for​ ma​ tion Se​ cu​ rity Team is in​ ter​ ested in the op​ er​ at​ ional as​ pects of the ACI se​ cu​ rity model as it re​ lates to the fol​ low​ ing ca​ pa​ bil​ it​ ies: ten​ ancy. In ACI there are some el​ e​ ments of the tra​ di​ tional tools. in​ clud​ ing credit card num​ bers. In a tra​ di​ tional net​ work. or some com​ bi​ na​ tion of these. API in​ te​ gra​ tion. al​ low​ ing the team to pub​ lish end​ point groups to vir​ tu​ al​ iza​ tion clus​ ters from a cen​ tral​ ized place across mul​ ti​ ple hy​ per​ vi​ sors.

This is a key dif​ fer​ ence be​ tween ACI and tra​ di​ tional processes that were very se​ ri​ al in na​ ture. There are mech​ an ​isms for backup and re​ store that will be dis​ cussed in fol​ low-on chap​ ters. . We will also dis​ cuss the model and which ob​ jects af​ fect the ten​ ants and the fab​ ric as a whole. There are a num​ ber of change/IT man​ age​ ment prin​ ci​ ples (Cisco Life​ cy​ cle ser​ vices. The col​ lab​ o​ ra​ tive na​ ture of ACI al​ lows for a high de​ gree of par​ al​ leliza​ tion of work flow. When "When" refers to when the teams listed above are in​ volved in the plan​ ning. script​ ing might be more ef​ fi​ cient. FCAPS. As a base​ line. The ACI pol​ icy model has been de​ signed to re​ duce the over​ all size of a fault do​ main and pro​ vide a mech​ an ​ism for in​ cre​ men​ tal change. It is a good idea to in​ volve the dif​ fer​ ent teams early when build​ ing poli​ cies and processes for how the fab​ ric is im​ ple​ mented and then man​ aged.Introduction 18 tion of many dif​ fer​ ent ob​ jects at once. re​ sult​ ing in a longer de​ ploy​ ment time for ap​ pli​ ca​ tions and a higher mean-time to res​ ol​ u​ tion when is​ sues arise. whereas sim​ ple dash​ board mon​ it​ or​ ing might be more suited to a GUI. Change con​ trol is a fact of life in the mis​ sioncrit​ ic ​al en​ vi​ ron​ ments that ACI has been de​ signed to sup​ port. An eval​ ua ​t​ ion of cur​ rent change con​ trol and con​ tin​ uo ​us in​ te​ gra​ tion/de​ liv​ ery strate​ gies is war​ ranted as op​ er​ at​ ional pro​ ce​ dures evolve. an im​ por​ tant con​ sid​ er​ at​ ion is change con​ trol. How "How" an​ swers the fol​ low​ ing basic ques​ tions: • How does a networking person go about configuring the network forwarding? • How does the compute team get information from the infrastructure to make • How do the application team track performance and usage metrics? • How does a storage team track the access to storage subsystems and ensure optimal workload placement decisions? that it is performant? When "how" in​ volves mak​ ing a change to the con​ fig​ ur​ a​ tion of an en​ vi​ ron​ ment. most or​ ga​ ni​ za​ tions are im​ ple​ ment​ ing some kind of struc​ tured changecon​ trol method​ ol​ ogy to mit​ ig ​ate busi​ ness risk and en​ hance sys​ tem avail​ abil​ ity. Through​ out this book we will high​ light the meth​ ods and pro​ ce​ dures to proac​ tively and re​ ac​ tively man​ age the fab​ ric.

Con​ tin​ uo ​us de​ liv​ ery does ex​ actly this by en​ sur​ ing that re​ leases are per​ formed reg​ ul​ arly from early on in the de​ liv​ ery process. In the In​ for​ ma​ tion Man​ age​ ment Sys​ tems world. which starts with the cre​ ation of a re​ quest for change which is then re​ viewed. there are three fun​ da​ men​ tal kinds of changes: • Emergency changes • Normal • Standard Emer​ gency changes are by de​ f​ i​ n​ i​ tion a re​ sponse to some kind of tech​ ni​ cal out​ age (hard​ ware.Introduction 19 and ITIL) that are good guides from which to start. and pro​ vi​ sion​ ing. Nor​ mal changes are those that go through the reg​ ul​ ar change man​ age​ ment process. and then (as​ sum​ ing it is au​ tho​ rized) planned and im​ ple​ mented. see the RBAC sec​ tion of this book. Ap​ ply​ ing change man​ age​ ment prin​ ci​ ples based on tech​ nol​ ogy from five years ago would not en​ able the rapid de​ ploy​ ment of tech​ nol​ ogy that ACI can de​ liver. soft​ ware. A com​ mon sense ap​ proach to change man​ age​ ment and con​ tin​ uo ​us in​ te​ gra​ tion should be a premise that is dis​ cussed early in the de​ sign and im​ ple​ men​ ta​ tion cycle be​ fore hand​ ing the fab​ ric to the op​ er​ a-​ tions teams for day-to-day main​ te​ nance. and en​ sur​ ing that de​ liv​ ery teams are work​ ing on the most valu​ able thing they could be at any given time. Train​ ing op​ er​ a​ tions teams on norms (a stated goal of this book) is also key. as​ sessed. Ul​ ti​ mately each change must be eval​ ua ​ted pri​ mar​ ily in terms of both its risk and value to the busi​ ness. The multi-ten​ ant and role-based ac​ cess con​ trol fea​ tures in​ her​ ent to the ACI so​ lu​ tion allow the iso​ la​ tion or draw​ ing of a very clean box around the scope and im​ pact of the changes that can be made. in​ fra​ struc​ ture) and are per​ formed to re​ store ser​ vice to af​ fected sys​ tems. mon​ it​ or​ ing. For more de​ tails. based on feed​ back from users. In an ACI en​ vi​ ron​ ment a nor​ mal change could apply to any​ thing within the fol​ low​ ing com​ po​ nents: • Fabric Policies (fabric internal and access will be discussed in detail later) • Configuration objects in the Common tenant that are shared with all other tenants (things that affect the entire fabric) Private Networks . A way to en​ able a low-over​ head change man​ age​ ment process is to re​ duce the risk of each change and in​ crease its value. and then ei​ ther au​ tho​ rized or re​ jected.

This is a key trou​ bleshoot​ ing tool for "when some​ thing mag​ ic ​ally stops work​ ing". In the ACI en​ vi​ ron​ ment some ex​ am​ ples of "stan​ dard" changes could be: • Tenant creation • Application profile creation • Endpoint group (EPG) creation • Contracts scoped at a tenant level • Layer 4 to Layer 7 service graphs • Domain associations for EPGs The items men​ tioned above are not in​ tended to be all-in​ clu​ sive. Im​ me​ di​ ate ac​ tion should be to check the audit log as it will tell who made what change and when. The abil​ ity to audit changes that are hap​ pen​ ing to the en​ vi​ ron​ ment is a re​ quire​ ment for ACME Inc. they must still be recorded and ap​ proved. cor​ re​ lat​ ing this to any faults that re​ sult from said change. . but are rep​ re​ sen​ ta​ tive of com​ mon tasks per​ formed day-to-day and week-to-week. Each or​ ga​ ni​ za​ tion will de​ cide the kind of stan​ dard changes that they allow. who is al​ lowed to ap​ prove them. APIC main​ tains an audit log for all con​ fig​ ur​ a​ tion changes to the sys​ tem. This en​ ables the change to be re​ verted quickly. the cri​ te​ ria for a change to be con​ sid​ ered "stan​ dard".Introduction 20 Bridge Domains Subnets • Virtual Machine Manager (VMM) integrations • Layer 4 to Layer 7 devices Device packages Creation of logical devices Creation of concrete devices • Layer 2 or Layer 3 external configuration • Attachable Entity Profile (AEP) creation • Server or external network attachment • Changes to currently deployed contracts and filters that would materially change the way traffic flows Stan​ dard changes are low-risk changes that are pre-au​ tho​ rized. As with nor​ mal changes. and the process for man​ ag​ ing them.

user. and this book il​ lus​ trates how the stake​ hold​ ers can work to​ gether in a more col​ lab​ o​ ra​ tive man​ ner than they have in the past. While some script​ ing op​ por​ tu​ ni​ ties are called out through​ out the book. the most im​ por​ tant thing is for these groups to work in​ sieme (to​ gether). to pro​ vide the great​ est value to the cus​ tomer. How​ ever.Introduction 21 A more in-depth dis​ cus​ sion of con​ tin​ u​ ous de​ liv​ ery in the con​ text of in​ fra​ struc​ ture man​ age​ ment is out​ side of the scope of this book. The re​ main​ der of this book an​ swers these ques​ tions. pro​ vid​ ing you with a frame​ work of how to take the con​ cepts and pro​ ce​ dures and apply them to sim​ i​ lar ini​ tia​ tives within your or​ ga​ ni​ za​ tions. ACI addresses IT requirements from across the organization . The book is laid out in a spe​ cific order. to com​ plete these tasks in par​ al​ lel with the var​ i​ ous stake​ hold​ ers who are high​ lighted through​ out. and ul​ ti​ mately the busi​ ness. While or​ ga​ ni​ za​ tional struc​ tures might be siloed into these teams. ACI en​ ables ACME Inc. there is a more in-depth sec​ tion at the end that ex​ plains how to use the ACI API to au​ to​ mate most op​ er​ a​ tional tasks.

.

23 Management .

.

Management Section Content • APIC Overview • Configuring Management Protocols Cisco Discovery Protocol Link Layer Discovery Protocol Time Synchronization and NTP Out-of-Band Management NTP In-Band Management NTP Verify NTP Operation Verifying that the NTP Policy Deployed to Each Node Domain Name Services (DNS) Verifying DNS Operation • Role-Based Access Control Multiple Tenant Support User Roles Security Domains Creation of a Security Domain Adding Users Remote Authentication • Import and Export Policies Configuration Export (Backup) Add a Remote Location (SCP) Create a One Time Export Policy Verify Export Policy was Successful Extract and View Configuration Files Configuration Import (Restore/Merge) 25 .

.

The un​ der​ ly​ ing in​ ter​ face for all ac​ cess meth​ ods is pro​ vided through a REST-based API. Each node re​ ceives a unique node iden​ ti​ fier. This re​ sults in a clean and pre​ dictable tran​ si​ tion be​ tween the in​ ter​ faces. This con​ troller-based ar​ chi​ tec​ ture also makes pos​ si​ ble a state​ less con​ fig​ ur​ a​ tion model that de​ cou​ ples the hard​ ware from the con​ fig​ ur​ a​ tion run​ ning on it. com​ mand line in​ ter​ face (CLI). The de​ vice can also be sub​ sti​ tuted in a state​ less fash​ ion. re​ duce the num​ ber of touch​ points. and health func​ tions. nor from a con​ fig​ ur​ a​ tion file re​ sid​ ing on the de​ vices. and ap​ pli​ ca​ tion pro​ gram​ ming in​ ter​ face (API). and not from the se​ ri​ al num​ ber of the chas​ sis. mean​ ing that hard​ ware swaps can be faster. This con​ troller pro​ vides ac​ cess to all con​ fig​ ur​ a​ tion. with no risk of in​ con​ sis​ tency be​ tween the var​ io ​us data in​ ter​ faces. man​ age​ ment. These changes in​ clude: • Single point of management controller-based architecture • Stateless hardware • Desired state-driven eventual consistency model The sin​ gle point of man​ age​ ment within the ACI ar​ chi​ tec​ ture is known as the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC). based on a con​ cept known as the promise .Management 27 APIC Overview There are a num​ ber of fun​ da​ men​ tal dif​ fer​ ences be​ tween the op​ er​ at​ ions of tra​ di​ tional net​ work​ ing hard​ ware and an ACI fab​ ric. which al​ lows for the de​ vice to down​ load the cor​ rect con​ fig​ ur​ a​ tion at​ trib​ utes from the con​ troller. The de​ sired state model for con​ fig​ ur​ a​ tion fur​ ther com​ ple​ ments these con​ cepts of con​ troller-based man​ age​ ment and state​ less​ ness by tak​ ing ad​ van​ tage of a con​ cept known as de​ clar​ at​ ive con​ trol-based man​ age​ ment. and de​ cou​ ple the switch​ ing hard​ ware from the de​ sired con​ fig​ ur​ a​ tion in​ tent. topol​ ogy changes are less im​ pact​ ful. This trans​ lates to an APIC clus​ ter that man​ ages in​ di​ vid​ ual fab​ ric nodes of leaf and spine switches that de​ rive their iden​ tity from what the con​ troller de​ fines as being the de​ sired in​ tent. Hav​ ing a cen​ tral​ ized con​ troller with an ap​ pli​ ca​ tion pro​ gram​ ming in​ ter​ face (API) means that all func​ tions con​ fig​ ured or ac​ cessed through the fab​ ric can be uni​ formly ap​ proached through a graph​ ic ​al user in​ ter​ face (GUI). These dif​ fer​ ences serve to sim​ plify the man​ age​ ment greatly. which mod​ if​ ies the con​ tents of a syn​ chro​ nized data​ base that is repli​ cated across APICs in a clus​ ter and pro​ vides an ab​ strac​ tion layer be​ tween all of the in​ ter​ faces. mon​ it​ or​ ing. and net​ work man​ age​ ment is sim​ pli​ fied.

This stands in con​ trast with the tra​ di​ tional model of im​ per​ at​ ive con​ trol. be told how to do it. dic​ tated by the man​ ag​ ing con​ troller. . A sys​ tem based on de​ clar​ at​ ive con​ trol is able to scale much more ef​ fi​ ciently than an im​ per​ at​ ive-based sys​ tem. where each man​ aged el​ e​ ment must be told pre​ cisely what to do. and take into ac​ count the spe​ cific sit​ ua ​t​ ional as​ pects that will im​ pact its abil​ ity to get from its cur​ rent state to the con​ fig​ ured state.28 Management the​ ory. De​ clar​ at​ ive con​ trol dic​ tates that each ob​ ject is asked to achieve a de​ sired state and makes a "promise" to reach this state with​ out being told pre​ cisely how to do so. since each en​ tity within the do​ main is re​ spon​ si​ ble for know​ ing its cur​ rent state and the steps re​ quired to get to the de​ sired state.

Maintaining accurate time across the application logs and infrastructure components is extremely important for being able to correlate events.Management 29 Configuring Management Protocols For the op​ ti​ miza​ tion of ACME's Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. man​ ua ​lly cre​ ate a “CDP-EN​ ABLED" pol​ icy with CDP en​ abled. there may be times when you must pro​ vi​ sion a par​ tic​ ul​ ar host-fac​ ing port man​ ua ​lly with a CDP on or off pol​ icy. maintaining accurate time is of increased importance as accurate time is a prerequisite for features such as atomic counters. such as when to cre​ ate switch pro​ files to con​ nect to Cisco UCS B Se​ ries servers. the net​ work team cre​ ated the fol​ low​ ing man​ age​ ment pro​ to​ col poli​ cies: • Cisco Discovery Protocol (CDP) . as CDP is the standard for their existing network equipment. these policies will be available for consumption by all of the teams. It is also im​ por​ tant to note that CDP might be re​ quired for cer​ tain con​ fig​ ur​ a​ tions. LLDP is a standards-based protocol for discovering topology relationships. With ACI.These policies are primarily consumed by the network team. • Network Time Protocol (NTP) . and storage arrays.providing name resolution to fabric components consistent with the application and server teams. ACME is deploying LLDP on servers. As a rec​ om​ mended prac​ tice. • Link Layer Discovery Protocol (LLDP) . Therefore.Discovery protocols are no longer limited to just network devices. In the course of ACI op​ er​ at​ ions. the important ACI components are also registered with ACME's DNS server so teams can easily access them in their management tasks. as well as a “CDP-DIS​ ABLED” pol​ icy with CDP dis​ abled that can be ref​ er​ enced through​ out all in​ ter​ face pol​ icy con​ fig​ ur​ a​ tions. • Domain Name Services (DNS) . Layer 4 to Layer 7 services devices. Cisco Discovery Protocol Cisco Dis​ cov​ ery Pro​ to​ col (CDP) is a valu​ able source of in​ for​ ma​ tion in trou​ bleshoot​ ing phys​ ic ​al and data-link layer is​ sues. By de​ fault. In addition to be able to resolve names within the fabric. ACME has existing NTP servers and will leverage them for maintaining time in their ACI fabric deployment. . the ACI fab​ ric pro​ vides a de​ fault pol​ icy where CDP is dis​ abled.

In the Name field enter the name of the policy.Management 30 To cre​ ate CDP poli​ cies: 1 On the menu bar. 4 In the Create LLDP Interface Policy dialog box. 5 6 In the Work pane. For the Admin State radio buttons. choose Actions > Create CDP Interface Policy. c. choose Actions > Create CDP Interface Policy. For the Admin State radio buttons. choose Interface Policies > Policies > LLDP Interface. In the course of ACI op​ er​ at​ ions. b. click Enabled. perform the following actions: a. To cre​ ate an LLDP pol​ icy: 1 On the menu bar. b. you must dis​ able LLDP. by de​ fault. 3 In the Work pane. 3 In the Work pane. 4 In the Create CDP Interface Policy dialog box. Cisco rec​ om​ mends that you pre-pro​ vi​ sion LLDP en​ able and dis​ able pro​ to​ col poli​ cies to make fu​ ture in​ ter​ face pol​ icy de​ ploy​ ment de​ ci​ sions stream​ lined and eas​ ily con​ fig​ ured. Click Submit. choose Fabric > Access Policies. perform the following actions: . 2 In the Navigation pane. 2 In the Navigation pane. choose Fabric > Access Policies. In the Name field enter the name of the policy. For ex​ am​ ple. In the Create CDP Interface Policy dialog box. There might be times in the op​ er​ at​ ion of the ACI fab​ ric in which you must man​ ua ​lly ad​ just the LLDP pro​ to​ col to con​ form with in​ ter​ op​ er​ abil​ ity re​ quire​ ments of end-host de​ vices. click Disabled. Link Layer Discovery Protocol Link Layer Dis​ cov​ ery Pro​ to​ col (LLDP) is a valu​ able source of in​ for​ ma​ tion for trou​ bleshoot​ ing phys​ ic ​al and data-link layer is​ sues. such as "CDP-ENABLED". when con​ nect​ ing to Cisco Uni​ fied Com​ put​ ing Sys​ tem (UCS) Blade servers. perform the following actions: a. Your CDP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric in​ ter​ faces. the LLDP fea​ ture is en​ abled on the fab​ ric. choose Actions > Create LLDP Interface Policy. such as "CDP-DISABLED". Click Submit. choose Interface Policies > Policies > CDP Interface. c.

d. click Disabled. time syn​ chro​ niza​ tion is a cru​ cial ca​ pa​ bil​ ity upon which many of the mon​ it​ or​ ing. choose Actions > Create LLDP Interface Policy. click Disabled. click Disabled. click Enabled. In the Name field. For the Receive State radio buttons. b. Clock syn​ chro​ niza​ tion is im​ por​ tant for proper analy​ sis of traf​ fic flows as well as for cor​ re​ lat​ ing debug and fault time​ stamps across mul​ ti​ ple fab​ ric nodes. In ad​ di​ tion.Management 31 a. c. enter "LLDP-TX-ON-RX-ON". c. Click Submit. d. In the Name field. 5 6 In the Work pane. op​ er​ at​ ional. 9 In the Create LLDP Interface Policy dialog box. perform the following actions: a. For the Transmit State radio buttons. 7 8 In the Work pane. For the Transmit State radio buttons. perform the following actions: a. For the Receive State radio buttons. An off​ set pre​ sent on one or more de​ vices can ham​ per the abil​ ity to prop​ erly di​ ag​ nose and re​ solve many com​ mon op​ er​ at​ ional is​ sues. c. b. c. perform the following actions: a. d. click Disabled. enter "LLDP-TX-OFF-RX-OFF". and trou​ bleshoot​ ing tasks de​ pend. In the Name field. Nonex​ is​ tent or im​ proper con​ fig​ ur​ a​ tion of time syn​ chro​ niza​ tion does not nec​ es​ sar​ ily trig​ ger a fault or a low health score. b. For the Receive State radio buttons. You . For the Transmit State radio buttons. In the Create LLDP Interface Policy dialog box. d. Time Synchronization and NTP Within the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. b. For the Transmit State radio buttons. choose Actions > Create LLDP Interface Policy. click Enabled. enter "LLDP-TX-ON-RX-OFF". In the Name field. Click Submit. Click Submit. Your LLDP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric in​ ter​ faces. For the Receive State radio buttons. Click Submit. In the Create LLDP Interface Policy dialog box. clock syn​ chro​ niza​ tion al​ lows for the full uti​ liza​ tion of the atomic counter ca​ pa​ bil​ ity that is built into the ACI upon which the ap​ pli​ ca​ tion health scores de​ pend. click Enabled. enter "LLDP-TX-OFF-RX-ON". click Enabled.

. leaves and all mem​ bers of the APIC clus​ ter. Provide a name for the policy to easily distinguish between your environment's different NTP configurations. only one pod per ACI fab​ ric is al​ lowed. c. To con​ fig​ ure NTP. The most widely adapted method for syn​ chro​ niz​ ing a de​ vice clock is to use Net​ work Time Pro​ to​ col (NTP). choose Pod Policies > Policies. Prior to con​ fig​ ur​ ing NTP. in-band man​ age​ ment and/or out-of-band man​ age​ ment. 3 In the Work pane. In the Create Providers dialog box. in​ clu​ sive of spines. choose Actions > Create Date and Time Policy. such as "Production-NTP". The re​ li​ ab ​il​ ity of the source must be care​ fully con​ sid​ ered when de​ ter​ min​ ing if you will use a pri​ vate in​ ter​ nal clock or an ex​ ter​ nal pub​ lic clock. con​ sider what man​ age​ ment IP ad​ dress scheme is in place within the ACI fab​ ric. This IP reach​ ab ​il​ ity will be lever​ aged so that each node can in​ di​ vid​ u​ ally query the same NTP server as a con​ sis​ tent clock source. con​ fig​ ur​ a​ tion of NTP will vary. An​ other con​ sid​ er​ at​ ion in de​ ploy​ ing time syn​ chro​ niza​ tion where the time source is lo​ cated. There are two op​ tions for con​ fig​ ur​ ing man​ age​ ment of all ACI nodes and Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ trollers (APICs). 2 In the Navigation pane. Description. see the Re​ ac​ tive Mon​ it​ or​ ing Tools chap​ ter. Date and Time poli​ cies are con​ fined to a sin​ gle pod and must be de​ ployed across all pods pro​ vi​ sioned in the ACI fab​ ric. b. 1 On the menu bar. Out-of-Band Management NTP When an ACI fab​ ric is de​ ployed with out-of-band man​ age​ ment. each node of the fab​ ric. For more in​ for​ ma​ tion on atomic coun​ ters. including the following fields: Name. 4 In the Create Date and Time Policy dialog box. and Maximum Polling Intervals. perform the following actions: a. choose Fabric > Fabric Policies. is man​ aged from out​ side the ACI fab​ ric. Click the + sign to specify the NTP server information (provider) to be used. enter all relevant information.Management 32 should con​ fig​ ure time syn​ chro​ niza​ tion be​ fore de​ ploy​ ing a full fab​ ric or ap​ pli​ ca​ tions so as to en​ able proper usage of these fea​ tures. a Date and Time pol​ icy must be cre​ ated that ref​ er​ ences an out-ofband man​ age​ ment end​ point group. De​ pend​ ing on which man​ age​ ment op​ tion was cho​ sen for the fab​ ric. and Minimum Polling Intervals. As of the pub​ lish​ ing of this doc​ um ​ent. d. Click Next.

Click OK. choose Fabric > Fabric Policies. click the Preferred check box for the most reliable NTP source. In the Management EPG drop-down list. con​ sider the reach​ ab ​il​ ity of the NTP server from within the ACI in-band man​ age​ ment net​ work. Verifying NTP Operation Full ver​ if​ i​ ca​ tion of NTP func​ tion​ al​ ity is best ac​ com​ plished by lever​ ag​ ing both the ACI CLI and the APIC GUI.Management 33 i. verify the details of the server. iii. if the NTP server is reachable by all nodes on the fabric through out-of-band management. e. see the "In-Band Management NTP" section. choose Out-ofBand. Pol​ icy con​ fig​ ur​ a​ tion can be ver​ if​ ied by these steps: 1 On the menu bar. you must con​ struct a pol​ icy to en​ able this com​ mu​ ni​ ca​ tion. In-Band Management NTP When an ACI Fab​ ric is de​ ployed with in-band man​ age​ ment. Your NTP pol​ icy is now ready for de​ ploy​ ment to the ACI fab​ ric nodes. Repeat steps c and d for each provider that you want to create. If you have deployed in-band management. 2 In the Navigation pane. choose Pod Policies > Policies > Date and Time > ntp_policy > server_name. If you are creating multiple providers. ii. To lever​ age an NTP server ex​ ter​ nal to the fab​ ric with in-band man​ age​ ment. In-band IP ad​ dress​ ing used within the ACI fab​ ric is. not reach​ able from any​ where out​ side the fab​ ric. Once you have es​ tab​ lished a pol​ icy to allow com​ mu​ ni​ ca​ tion. by de​ f​ in ​i​ t​ ion. you can cre​ ate the NTP pol​ icy as es​ tab​ lished in the "Out-of-Band Man​ age​ ment NTP" sec​ tion. . The steps used to con​ fig​ ure in-band man​ age​ ment poli​ cies are iden​ ti​ cal to those used to es​ tab​ lish an out-of-band man​ age​ ment pol​ icy: the dis​ tinc​ tion is around how to allow the fab​ ric to con​ nect to the NTP server. 3 In the Work pane. Start ver​ if​ y​ ing time syn​ chro​ niza​ tion con​ fig​ ur​ a​ tion by en​ sur​ ing that all poli​ cies are con​ fig​ ured prop​ erly in​ side of the APIC GUI. The ntp_policy is the previously created policy.

3 In the Work pane.Management 34 Verifying that the NTP Policy Deployed to Each Node To ver​ ify that the pol​ icy has been suc​ cess​ fully de​ ployed to each node. 1 SSH to an APIC in the fabric. 1 On the menu bar. 5 Repeat steps 3 and 4 for each node on the fabric. choose Global Policies > DNS Profiles > default. From this prompt. . choose the appropriate management EPG. and the delay is a non-zero value. 2 In the Navigation pane. Domain Name Services (DNS) Set​ ting up a DNS server al​ lows the APIC to re​ solve var​ io ​us host​ names to IP ad​ dresses. 2 Press the Tab key twice after the entering the attach command to list all of the available node names: admin@apic1:~> attach <Tab> <Tab> 3 Log in to one of the nodes using the same password that you used to access the APIC. in the Management EPG drop-down list. To ac​ cess the NX-OS Vir​ tual Shell. admin@apic1:~> attach node_name 4 View the NTP peer status: leaf-1# show ntp peer-status A reach​ able NTP server has its IP ad​ dress pre​ fixed by an as​ ter​ isk (*). open an SSH ses​ sion to the out-of-band man​ age​ ment IP in​ ter​ face of any of the APIC nodes. choose Fabric > Fabric Policies. you should use the NX-OS Vir​ tual Shell that is pre​ sent in the APIC. ex​ e​ cute the "ver​ sion" com​ mand to ob​ tain a con​ sol​ id ​ated list of node host​ names. This is use​ ful when in​ te​ grat​ ing VMM do​ mains or other Layer 4 to Layer 7 de​ vices and the host​ name is ref​ er​ enced.

conf file from the APIC CLI: admin@apic1:~> cat /etc/resolv.com nameserver 171. c. c. enter the provider address. In the Address field. enter the domain name. Click Update.131. such as "cisco. click the check box to make this domain the default domain. Note: You can have only one preferred provider. a. 7 Repeat step 6 for each additional DNS Domain suffix.com". click the check box if you want to have this address as the preferred provider.168.conf # Generated by IFC search cisco. In the Preferred field.70. 6 Click + next to DNS Domains to add a DNS domain. 8 Click Submit. b.183 nameserver 173.36. Note: You can have only one domain name as the default.Management 35 Note: The default is default (Out-of-Band). 4 Click + next to DNS Providers to add a DNS provider. In the Name field. In the Default field . Click Update. 5 Repeat step 4 for each additional DNS provider.10 . a. b. Verifying DNS Operation 1 Check the resolv.

163.4.4 ms 64 bytes from www1.161) 56(84) bytes of data.com (72.161): icmp_seq=2 ttl=241 time=42.com (72.4.163.9 ms ^C --. 64 bytes from www1.038/43.cisco.com (72.com (72.163.cisco.619 ms admin@apic1:~> .161): icmp_seq=3 ttl=241 time=43.161): icmp_seq=1 ttl=241 time=42. 3 received.Management 36 2 Ping a host by DNS name that will be reachable from the APIC out-of-band management.903/0.163. admin@apic1:~> ping cisco.cisco.cisco.485/43. time 2102ms rtt min/avg/max/mdev = 42.4.7 ms 64 bytes from www1.com ping statistics --3 packets transmitted. 0% packet loss.4.com PING cisco.

For ex​ am​ ple. In an ACI fab​ ric. ricThe role clas​ si​ fi​ ca​ tion is an es​ tab​ lished group​ ing of per​ mis​ sions. and the se​ cu​ rity do​ main de​ fines Where the user can per​ form these ac​ tions. What. privileges and privilege types (which can be "no access". Each user can be as​ signed to a set of roles. Role-based ac​ cess and the con​ cept of ten​ ancy are two core foun​ da​ tions upon which the ACI man​ age​ ment and pol​ icy mod​ els are built. and Where. ACME's IT or​ ga​ ni​ za​ tion fits per​ fectly into this ac​ cess con​ trol model. and a se​ cu​ rity do​ main. and is un​ able to man​ age other user roles at a fab​ ric wide level. or "read-write") information tree (MIT) that a user can access . The role is the Who (log​ ic ​al col​ lec​ tion of priv​ il​ eges). priv​ il​ eges de​ fine What func​ tions a user can per​ form.Management 37 Role-Based Access Control Role-based ac​ cess con​ trol (RBAC) and ten​ ancy are im​ por​ tant con​ cepts to mas​ ter when op​ er​ at​ ing a Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. a fab​ ad​ min can have ac​ cess to the en​ tire fab​ ric as well as to as​ sign​ ing other user roles and as​ so​ ci​ ate them to se​ cu​ rity do​ mains. When com​ bin​ ing these ob​ jects. "read- • One or more security domain tags that identify the portions of the management only". whereas a ten​ ant-ad​ min can only have ac​ cess to the com​ po​ nents within the ten​ ant to which they're as​ so​ ci​ ated. the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) grants ac​ cess to all ob​ jects based on a user's es​ tab​ lished role in the con​ fig​ ur​ a​ tion. Think of this in terms of Who. a priv​ il​ ege type. you can im​ ple​ ment very gran​ ul​ ar ac​ cess con​ trol in the sys​ tem. An ACI fab​ ric user is as​ so​ ci​ ated with the fol​ low​ ing: • A set of roles • For each role. The APIC pro​ vides ac​ cess ac​ cord​ ing to a user’s role through RBAC.

APIC poli​ cies man​ age the ac​ cess. Multiple Tenant Support A core APIC in​ ter​ nal data ac​ cess con​ trol sys​ tem pro​ vides multi-ten​ ant iso​ la​ tion and pre​ vents in​ for​ ma​ tion pri​ vacy from being com​ pro​ mised across ten​ ants. This flag is set by the APIC on all ob​ jects that cor​ re​ spond to equip​ ment in the phys​ i​ cal fab​ ric. These con​ fig​ u​ ra​ tions can be im​ ple​ mented using the REST API. The com​ bi​ na​ tion of user priv​ i​ leges. Read/write re​ stric​ tions pre​ vent any ten​ ant from see​ ing any other ten​ ant’s con​ fig​ u​ ra​ tion. au​ then​ ti​ ca​ tion. and ac​ count​ ing (AAA) func​ tions of the Cisco ACI fab​ ric. sta​ tis​ tics. roles. A priv​ i​ lege is an MO that en​ ables or re​ stricts ac​ cess to a par​ tic​ u​ lar func​ tion within the sys​ tem. . CLI.38 Management Authenticated user creation lifecycle The ACI fab​ ric man​ ages ac​ cess priv​ i​ leges at the man​ aged ob​ ject (MO) level. and se​ cu​ rity do​ mains with ac​ cess rights in​ her​ i​ tance en​ ables ad​ min​ is​ tra​ tors to con​ fig​ ure AAA func​ tions at the man​ aged ob​ ject level in a very gran​ u​ lar fash​ ion. fab​ ric-equip​ ment is a priv​ i​ lege flag. For ex​ am​ ple. or GUI.

1 On the menu bar. ten​ ants are re​ stricted from read​ ing fab​ ric con​ fig​ ur​ a​ tion. Changes in the "all" do​ main will af​ fect ac​ cess per​ mis​ sions for all users. you can see each built-in role and the associated privileges. Using the ten​ ancy ca​ pa​ bil​ it​ ies of the ACI fab​ ric in con​ junc​ tion with prop​ erly con​ fig​ ured se​ cu​ rity do​ mains. 2 In the Navigation pane. or events. you can create a custom role from the Actions menu. A key con​ cept to keep in mind when con​ fig​ ur​ ing se​ cu​ rity do​ mains is that the con​ fig​ u-​ ra​ tion only ap​ plies at a ten​ ant level. The poli​ cies can​ not cur​ rently be ap​ plied to in​ di​ vid​ ual ob​ jects de​ spite ref​ er​ ences at the ob​ ject level in​ side of the RBAC screen in the APIC. By using se​ cu​ rity do​ mains. it will be pos​ si​ ble to com​ pletely sep​ ar​ ate con​ fig​ ur​ a​ tion of ap​ pli​ ca​ tion work​ loads while only pro​ vid​ ing ac​ cess to those who need it. users can be or​ ga​ nized into var​ io ​us per​ mis​ sion struc​ tures. sta​ tis​ tics. the users con​ tained in the se​ cu​ rity do​ main can ac​ cess the cor​ re​ spond​ ing VMM do​ main. Un​ less the ad​ min​ is​ tra​ tor as​ signs per​ mis​ sions to do so. choose Security Management > Roles. choose Admin > AAA. Note: From here. Security Domains The se​ cu​ rity do​ main con​ cept is cru​ cial to proper op​ er​ at​ ion of the ACI fab​ ric. most com​ monly ap​ plied to ten​ ants. then users in the solar ten​ ant can ac​ cess the VMM do​ main ac​ cord​ ing to their ac​ cess rights. Cisco does not rec​ om​ mend that you pro​ vide user ac​ cess con​ fig​ u-​ ra​ tion by mod​ if​ ing per​ mis​ sions of the "all" do​ main.Management 39 faults. or event data. If you need to make se​ lec​ tive changes to allow ac​ cess out​ side of a user's se​ cu​ rity do​ main. For ex​ am​ ple. Additionally. be sure to set up a dis​ crete user ac​ cess pol​ icy just for that com​ mu​ ni​ ca​ tion. User Roles You can view the built-in user roles as well as cre​ ate cus​ tom roles to meet spe​ cific re​ quire​ ments. . If a vir​ tual ma​ chine man​ age​ ment (VMM) do​ main is tagged as a se​ cu​ rity do​ main. Cisco rec​ om​ mends that you con​ fig​ ure se​ cu​ rity do​ mains and ac​ cess lev​ els prior to de​ ploy​ ment of ten​ ants. if a ten​ ant named solar is tagged with the se​ cu​ rity do​ main called sun and a VMM do​ main is also tagged with the se​ cu​ rity do​ main called sun. poli​ cies. faults.

Give the security domain a name and an optional description. 2 In the Navigation pane. choose Local and click Submit if Local is not already chosen. choose Security Management > Security Management. ACI can have local users man​ u-​ ally en​ tered by an admin. choose AAA Authentication. choose Admin > AAA. 2 In the Navigation pane.​ html To cre​ ate a se​ cu​ rity do​ main: 1 On the menu bar. perform the following actions: a. Con​ fig​ ure Local Users: 1 On the menu bar.Management 40 Creation of a Security Domain There are three se​ cu​ rity do​ mains that are pro​ vi​ sioned by de​ fault on the ACI fab​ ric: "all". choose Actions > Create a Security Domain. for a multi-ten​ ant en​ vi​ ron​ ment. or TACACS+ to spec​ ify users that will be al​ lowed to man​ age cer​ tain parts of the ACI net​ work. 3 In the Work pane. The "all" se​ cu​ rity do​ main usu​ ally in​ cludes ac​ cess to every​ thing within the man​ age​ ment in​ for​ ma​ tion tree (MIT). Adding Users You might need to add new users within the ACI fab​ ric. RA​ DIUS. or ten​ ants. choose Admin > AAA. For more in​ for​ ma​ tion about the MIT.​ cisco. in the Realm drop-down list. RBAC poli​ cies are con​ fig​ ured under the "Admin" tab of the APIC GUI. "com​ mon".​ com/​ c/​ en/​ us/​ products/​ collateral/​ cloud-systems-management/​ aci-fabric-controller/​ white-paper-c11-729586. You can add se​ cu​ rity do​ mains as nec​ es​ sary. 3 In the Work pane. For ex​ am​ ple. 4 In the Create a Security Domain dialog box. Ac​ tive Di​ rec​ tory. "Com​ mon" is usu​ ally used when there is a need for shared re​ sources be​ tween ten​ ants. or use some​ thing such as LDAP. such as DNS or di​ rec​ tory ser​ vices. see the doc​ um ​ent at the fol​ low​ ing URL: http://​ www. The "mgmt" se​ cu​ rity do​ main is for man​ age​ ment traf​ fic poli​ cies. . a se​ cu​ rity do​ main would be cre​ ated for each ten​ ant. Users are then cre​ ated and as​ signed to cer​ tain se​ cu​ rity do​ mains. and "mgmt".

The IP ad​ dress or host name of the LDAP server is needed. Specify any security information that is necessary and click Next. The same is true for RA​ DIUS and TACACS+. choose Security Management. de​ tail in the Cisco ACI Fun​ .Management 4 In the Navigation pane. Specify login information for the user and 7 Click Finish. Select the roles to be given to this user. then the LDAP provider must be con​ fig​ ured first. and click Next. If cre​ at​ ing an LDAP ac​ count. c. such as Read/Write for admin or tenant admin. 5 In the Work pane. Re​ mote au​ then​ ti​ ca​ tion is cov​ ered in da​ men​ tals Guide. perform the following actions: a. 6 41 In the Create Local User dialog box. choose Actions > Create Local User. The next op​ tion is to con​ fig​ ure groups from which it is al​ lowed to read data and grab it for the pur​ poses of se​ lect​ ing re​ mote users granted ac​ cess to the ACI net​ work. b. Remote Authentication Cre​ at​ ing re​ mote user ac​ counts in the ACI is sim​ il​ ar to most other data cen​ ter sys​ tems. along with the port it uses to com​ mu​ ni​ cate as well as any other rel​ e​ vant in​ for​ ma​ tion that will allow con​ nec​ tion to that server.

.

The backup and re​ store process does not re​ quire backup of in​ di​ vid​ ual com​ po​ nents. but the ad​ min​ is​ tra​ tor can op​ tion​ ally spec​ ify only a spe​ cific sub​ tree of the man​ age​ ment in​ for​ ma​ tion tree. For de​ tails on tech​ ni​ cal sup​ port ex​ port poli​ cies. Back​ ups are con​ fig​ urable through an ex​ port pol​ icy that al​ lows ei​ ther sched​ uled or im​ me​ di​ ate back​ ups to a re​ mote server (pre​ ferred) or. Con​ fig​ ur​ a​ tion Im​ port poli​ cies (pol​ icy re​ store) will be dis​ cussed later in this sec​ tion. Cisco rec​ om​ mends that a cur​ rent Backup be per​ formed be​ fore mak​ ing any major sys​ tem con​ fig​ ur​ a​ tion changes or ap​ ply​ ing soft​ ware up​ dates. in the case where an ex​ ter​ nal SCP/FTP server is not avail​ able. This can be done from any ac​ tive and fully fit APIC within the ACI fab​ ric. ACI is a good choice on this front as well.Management 43 Import and Export Policies All of the stake​ hold​ ers at ACME in​ volved in the de​ ploy​ ment and ad​ min​ is​ tra​ tion of the new mo​ bile ap​ pli​ ca​ tion plat​ form need to know that they will be able to eas​ ily re​ cover from any loss of con​ fig​ ur​ a​ tion quickly and eas​ ily. An​ other Ex​ port pol​ icy re​ quired on oc​ ca​ sion is for gath​ er​ ing tech​ ni​ cal sup​ port in​ for​ ma​ tion. This is use​ ful in multi-ten​ ant de​ ploy​ ments where in​ di​ vid​ ual ten​ ants wish to backup or re​ store their re​ spec​ tive poli​ cies. As with all things APIC. and so the bun​ dles will not be cov​ ered in de​ tail bleshoot​ here. and of​ fline analy​ sis. trou​ bleshoot​ ing. Tech​ ni​ cal sup​ port bun​ dles are con​ fig​ ured very sim​ i-​ lar to con​ fig​ ur​ a​ tion ex​ port poli​ cies. By de​ fault. back​ ups to be writ​ ten to the local APIC file sys​ tem. If is​ sues are en​ coun​ tered within the sys​ tem. all poli​ cies and ten​ ants are backed up. Since all APIC poli​ cies and con​ fig​ ur​ a​ tion data can be ex​ ported to cre​ ate back​ ups and tech sup​ port files for Dis​ as​ ter Re​ cov​ ery. you might need to con​ tact the Cisco Tech​ ni​ cal As​ sis​ tance Cen​ ter (TAC). Pro​ vid​ ing a tech​ ni​ cal sup​ port bun​ dle will give Cisco sup​ port en​ gi​ neers the in​ for​ ma​ tion needed to help iden​ tify and sug​ gest re​ me​ di​ at​ ion to is​ sues. a pol​ icy needs to be con​ fig​ ured to ex​ port or backup the con​ fig​ ur​ a​ tion poli​ cies. see the Cisco APIC Trou​ ing Guide that is ref​ er​ enced in the Ap​ pen​ dix of this book. Tech​ ni​ cal sup​ port ex​ port poli​ cies can be con​ fig​ ured to run on-de​ mand or sched​ uled for re​ cur​ ring pur​ poses. Configuration Export (Backup) . Backup/ex​ port poli​ cies can be con​ fig​ ured to be run on-de​ mand or based on a re​ cur​ ring sched​ ule.

Enter a Remote path e. 3 In the Work pane. a re​ mote lo​ ca​ tion is cre​ ated (the ex​ ter​ nal server to which you want to ex​ port in​ for​ ma​ tion). The in​ di​ vid​ ual XML files can be ex​ tracted and viewed after the con​ fig​ u​ ra​ tion bun​ dle is trans​ ferred to the desk​ top. choose Actions > Create Remote Location 4 In the Create Remote Location dialog box. Enter a Remote location name b.Management 44 Configuration Export (Backup) Since ACI is pol​ icy dri​ ven. First. Enter a Password h. and op​ tion​ ally a Sched​ uler (for re​ cur​ ring tasks). choose Admin > Import/Export. Configuration Export Policy Add a Remote Location (SCP) 1 On the menu bar. 2 In the Navigation pane. Choose a Management EPG Note: default (Out-of-Band) 5 Click Submit. then an Ex​ port pol​ icy task. perform the following actions: a. more than a backup job is re​ quired. Enter a Username g. Enter a Hostname/IP address c. . choose Remote Locations.​ gz file that is cre​ ated. The pro​ ce​ dure below de​ tails how to cre​ ate a re​ mote lo​ ca​ tion and ex​ port pol​ icy in ACI to trans​ fer the con​ fig​ u​ ra​ tion file bun​ dle to an SCP server. An ex​ trac​ tion util​ ity is needed to de​ com​ press the tar. Enter a Remote port f. Choose a Protocol d.

Export Destination = Choose_the_Remote_location_created_above 5 Click Submit. and spec​ if​ y​ ing a spe​ cific Dis​ tin​ guished Name (DN) if you want to backup only a sub​ set of the Man​ age​ ment In​ for​ ma​ tion Tree (MIT). Two op​ tional con​ fig​ ur​ a​ tions are ap​ ply​ ing a Sched​ uler pol​ icy if you want to setup a re​ cur​ ring op​ er​ at​ ion. perform the following actions: a. b. 2 In the Navigation pane. choose Export Policies > Configuration. choose Admin > Import/Export. 3 In the Work pane. Name = Export_Policy_Name b. 3 In the Work pane. choose Admin > Import/Export. Start Now = Yes d. Format = XML c. (Optional) Confirm on the SCP server that the backup filename exists. . Verify Export Policy was Successful 1 On the menu bar. choose Export Policies > Configuration > Export_Name. The State should change from "pending" to "success" when the export completes correctly. choose the Operational tab. 2 In the Navigation pane. a. but the pro​ ce​ dure for a tech​ ni​ cal sup​ port ex​ port pol​ icy is very sim​ il​ ar. choose Actions > Create Configuration Export Policy 4 In the Create Configuration Export Policy dialog box.Management 45 Create a One Time Export Policy The pro​ ce​ dure below de​ tails a con​ fig​ ur​ a​ tion ex​ port pol​ icy. 1 On the menu bar.

Do not include the file path. b.tar. 1 From the workstation where the exported bundle has been copied. 3 In the Work pane. Will not overrite any existing policies. 3 Double-click one of the XML files to view the contents in a browser. choose Admin > Import/Export. 4 Examine the various XML configuration files for parameters that have been configured. 1 On the menu bar. choose Actions > Create Import Configuration Policy. Configuration Import (Restore/Merge) To re​ store a con​ fig​ ur​ a​ tion from a pre​ vi​ ous backup: If the re​ mote lo​ ca​ tion does not exist. select the archive utility of choice.Will replace all configurations with those only from the backup file. highlight the file. cre​ ate a re​ mote lo​ ca​ tion per the "Adding a Re​ mote Lo​ ca​ tion (SCP)" sec​ tion. such as "backupfile. 2 Navigate to the folder where the config export is located (tar.Management 46 Extract and View Configuration Files A con​ fig​ ur​ a​ tion ex​ port task ex​ ports a com​ pressed bun​ dle of in​ di​ vid​ ual XML files. . 4 In the Create Import Configuration Policy dialog box. Enter a name and the filename for the import policy.gz". choose Import Policies > Configuration. perform the following actions: a. 2 In the Navigation pane.gz). and then select Extract. These XML con​ fig​ ur​ a​ tion files can be ex​ tracted and viewed on an​ other work​ sta​ tion.Will merge backup configuration with existing running configurations. Choose the import type: Merge . Replace .

d. The configuration data is imported per shard with each shard holding a certain part of the system configuration objects. The import mode must be specified when you attempt to perform a Merge import. skipping any invalid objects atomic . .Each shard is imported. Choose the Remote Location that was previously created in the "Adding a Remote Location (SCP)" section. A shard's entire configuration must be valid to be imported in this mode. Choose Start Now.Attempts to import each shard. The Replace Import Mode can be either: best-effort . e. The default is best-effort.Management 47 c. skipping those which contain an ​ invalid configuration.

.

49 Upgrading and Downgrading Firmware .

.

Upgrading and Downgrading Firmware Section Content • Firmware Management Firmware Versions Firmware Components Firmware Policies Firmware Groups Maintenance Groups Controller Firmware Catalogue Firmware • Upgrading and Downgrading Considerations • Upgrading the Fabric Downloading the Firmware Images Upgrading the APIC Controller Software Upgrading the Switch Software Using the GUI Upgrading the APIC Controller Software Using the CLI Upgrading the Switch Software Using the CLI Verifying Cluster Convergence Troubleshooting Failures During the Upgrade Process 51 .

.

• mntnc .. • minor . and the ma​ tu​ rity of the soft​ ware re​ leases. They have se​ lected a tar​ get ver​ sion of soft​ ware for their de​ ploy​ ment. This changes when there are fixes for product defects in the software. but no additional new features.minor(mntnc) • major . For ex​ am​ ple. Re​ lease notes for the APIC ver​ sions ref​ er​ ence the cor​ re​ spond​ ing switch ver​ sions and vice-versa. in part​ ner​ ship with Cisco. the sup​ port for the hard​ ware plat​ forms they have se​ lected.0(2j) cor​ re​ sponds to switch soft​ ware 11. Ad​ di​ tion​ ally.Represents major changes in the product architecture.0(2j). should be on the same . platform. All com​ po​ nents of the ACI in​ fra​ struc​ ture in​ clud​ ing the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC).1(1d) Both the soft​ ware for the APIC and the fab​ ric nodes are de​ noted by the same ver​ sion​ ing scheme. has eval​ ua ​ted the re​ quire​ ments for their de​ ploy​ ment based on the soft​ ware fea​ tures re​ quired. they have put a proac​ tive plan in place to re​ visit this de​ ci​ sion pe​ ri​ od​ ic ​ally to de​ ter​ mine if fu​ ture up​ grades are re​ quired. Ex​ am​ ple: APIC version: 1. leaf switches. or features content. and spine switches.Represents a minor release with new software features.Upgrading and Downgrading Firmware 53 Firmware Management ACME Inc. APIC re​ lease 1.Represents bug fixes to a feature release of APIC. Firmware Versions The soft​ ware ver​ sions for ACI are listed in the fol​ low​ ing for​ mat: major.

mak​ ing it pos​ si​ ble to defer an up​ grade task to a busi​ ness main​ te​ nance win​ dow. such as 1. Controller Firmware The APIC con​ troller firmware pol​ icy al​ ways ap​ plies to all con​ trollers in the clus​ ter. do not op​ er​ ate the fab​ ric for ex​ tended pe​ ri​ ods of time in this state. While at the time of up​ grad​ ing. a sin​ gle firmware group is ad​ e​ quate. For most de​ ploy​ ments. When con​ sid​ er​ ing the im​ pact and risk of up​ grad​ ing.(1d) => 1.1(1d). but the up​ grade is al​ ways done se​ quen​ tially. Firmware Components There are three main com​ po​ nents that can be up​ graded: • Switches (leaf and spine) • Application Policy Infrastructure Controller (APIC) • Catalog firmware Firmware Policies Firmware Groups Firmware Group poli​ cies on the APIC de​ fine which group of nodes on which firmware will be up​ graded. Maintenance Groups Main​ te​ nance Group poli​ cies de​ fine a group of switches that will be jointly up​ graded to the as​ so​ ci​ ated firmware set.Upgrading and Downgrading Firmware 54 ver​ sion. Con​ trol over which switches up​ grade to the new ver​ sion can be de​ ter​ mined through main​ te​ nance groups. as there will be only bug fixes and no new fea​ tures added. you can as​ sume that a main​ te​ nance ver​ sion up​ grade. Main​ te​ nance groups can be up​ graded on de​ mand or ac​ cord​ ing to a sched​ ule. dis​ parate ver​ sions may exist be​ tween APIC and the switches. will have less im​ pact than a major/minor ver​ sion up​ grade.1. The APIC GUI pro​ vides real-time sta​ tus in​ for​ - .

switch types. Con​ troller Firmware poli​ cies can be up​ graded on de​ mand or also ac​ cord​ ing to a sched​ ule.Upgrading and Downgrading Firmware 55 ma​ tion about firmware up​ grades. The APIC. and switch im​ ages. which per​ forms image man​ age​ ment. has an image repos​ i​ tory for com​ pat​ i​ bil​ ity cat​ a​ logs. and mod​ els that are al​ lowed to use that firmware image. Catalogue Firmware Each firmware image in​ cludes a com​ pat​ i​ bil​ ity cat​ a​ log that iden​ ti​ fies sup​ ported switch mod​ els. APIC con​ troller firmware im​ ages. Firmware upgrade policy relationships . The APIC main​ tains a cat​ a​ log of the firmware im​ ages.

.

the controllers should be upgraded first. Verify that the health state of all of the controllers in the cluster are Fully Fit before you proceed. • Maintenance groups. To help minimize the impact to hosts during an upgrade. • Verify free space. see the Import and Export Policies chapter. con​ sider the fol​ low​ ing things: • Before starting the upgrade process. A user must have the fabric administrator role to perform firmware upgrade tasks. On the menu bar.github. • Permissions. choose System > Controllers. choose Controllers > apic_controller_name > Storage.Upgrading and Downgrading Firmware 57 Upgrading and Downgrading Considerations Be​ fore start​ ing the up​ grade or down​ grade process. Typically. To resolve issues for controllers that are not fully fit see the Troubleshooting Cisco Application Centric Infrastructure document:https://datacenter. For information about exporting configurations. • Maintenance windows. Always refer to the relevant release notes of the destination firmware version for any changes to this order. The APIC automatically extracts the image. • Upgrade order. Assuming that your hosts are dualconnected to at least one odd and one even leaf node. you should perform an upgrade during a scheduled maintenance window according to your change control policy. In the navigation pane. choose System > Controllers. always export your configuration to an external source. In the navigation pane. Although it is possible to upgrade the fabric without impacting the dataplane. and allocate enough time to troubleshoot or perform a rollback. choose Controllers > apic_controller_name > Cluster. A common separation is by odd and even node IDs. there should not be any . followed by the switch nodes. Confirm that the /firmware partition is not filled beyond 75%. you might be required to remove some unused firmware files from the repository to accommodate the compressed image as well as provide adequate space to extract the image. This window should account for any unforseen issues that might arise during the upgrade. your controllers should be in good health.io/acitroubleshooting-book/ • Configuration backup. On the menu bar. Before starting any upgrade. you should set up at least two separate maintenance groups. If the partition is filled beyond 75%.

Maintenance group creation is covered in detail later in the chapter. software is not specifically tied to the APIC or switch software version.Upgrading and Downgrading Firmware 58 impact to your hosts. You can confirm the device compatability for Layer 4 to Layer 7 devices using the online Application Centric Infrastructure (ACI) Compatability tool: http://www.html . The AVS • Device packages. Device packages are not always tied to the APIC software.com/web/techdoc/aci/acimatrix/matrix.cisco. Another consideration is that your leaf VPC pairs should contain one odd and one even node. • Upgrading a fabric with the Application Virtual Switch (AVS) deployed.

​ com.3d.1. click the Secure copy or HTTP radio button.0. Click Submit. choose Firmware Repository. choose Admin > Firmware.iso For SCP. Note: In the Work pane.Upgrading and Downgrading Firmware 59 Upgrading the Fabric Downloading the Firmware Images You must down​ load both the con​ troller soft​ ware pack​ age and switch soft​ ware pack​ age for the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) from Cisco. To down​ load the firmware im​ ages using the APIC GUI: 1 On the menu bar. In the Source Name field. choose Download Tasks. enter your username and password. the list of all switches in the fabric and the status of when the firmware was last upgraded are displayed. For the Protocol radio buttons. 3 In the Navigation pane. 4 In the Work pane. 8 Repeat the steps for the switch image. In the URL field. choose Fabric Node Firmware. choose Actions > Create Outside Firmware Source. b. 2 In the Navigation pane.168.50/aci-apic-dk9. such as "apic_1. c. perform the following actions: a.3d.50:/tmp/aci-firmware/aci-apicdk9.0.1. enter the URL from where the image must be downloaded. 5 In the Create Outside Firmware Source dialog box.1. • HTTP Example: http://192.1.168.3d".iso • SCP Example: 192.1. in the Navigation pane. choose the Operational tab to view the download status of the images. choose Download Tasks. 9 Once the download reaches 100%. . 7 In the Work pane. 6 In the Navigation pane. d. enter a name for the switch image.

1d.Upgrading and Downgrading Firmware 60 10 In the Work pane.iso is added to the repos​ it​ ory. To down​ load the firmware im​ ages using the APIC CLI: 1 SSH to an APIC and log in as "admin". click Controller Firmware. You do not need to up​ grade the cat​ al​ og firmware image sep​ ar​ ately. choose Actions > Upgrade Controller Firmware Policy.bin Type : controller Version : 1.1. choose Admin > Firmware. perform the following actions: . 4 In the Upgrade Controller Firmware Policy dialog box.1(1d) Upgrading the APIC Controller Software The cat​ al​ og firmware image is up​ graded when an APIC con​ troller image is up​ graded. To up​ grade the APIC con​ troller soft​ ware using the GUI: 1 On the menu bar. the downloaded version numbers and image sizes are displayed. 3 In the Work pane. 2 In the Navigation pane.iso &#10! The firmware image ver_no. 4 Verify the software has been added to the respository: admin@ifc1:~> firmware list Name : aci-apic-dk9. 2 Pull the software using SCP: admin@apic1:~> scp username@IP_address_with_the_image:/absolute_path_to_image_plus_image_filename &#10! 3 Place the image into the image repository: admin@apic1:~> firmware add ver_no.1.

Once a con​ troller image is up​ graded. a Wait​ ing for Clus​ ter Con​ ver​ gence mes​ sage is dis​ played in the Sta​ tus col​ umn for each APIC as it up​ grades. click the Apply now radio button. In the Apply Policy field. choose the image version to which you want to upgrade. the up​ grade will wait until the clus​ ter con​ verges and is fully fit. it joins the clus​ ter again. Dur​ ing this pe​ riod. the APICs must have com​ pleted up​ grad​ ing and have a health state of Fully Fit. . 5 Verify the status of the upgrade in the Work pane by clicking Controller Firmware in the Navigation pane. c. Click Submit to complete the task. you can apply a schedule policy if you wish to defer the task to a specific date/time. Upgrading the Switch Software Using the GUI Be​ fore you up​ grade the switches. The Sta​ tus di​ al​ og box dis​ plays the Changes Saved Suc​ cess​ fully mes​ sage. and it re​ boots with the newer ver​ sion while the other APIC con​ trollers in the clus​ ter are still op​ er​ at​ ional. If the clus​ ter does not im​ me​ di​ ately con​ verge. 6 In the browser URL field. and is not fully fit. Each APIC con​ troller takes about 10 min​ utes to up​ grade. Then the clus​ ter con​ verges.Upgrading and Downgrading Firmware 61 a. it drops from the clus​ ter. and the up​ grade process be​ gins. enter the URL for the APIC controller that has already been upgraded. In the Target Firmware Version field. Alternately. The APIC con​ trollers are up​ graded se​ ri​ ally so that the con​ troller clus​ ter is avail​ able dur​ ing the up​ grade. To up​ grade the switch soft​ ware using the GUI: 1 On the menu bar. the browser dis​ plays an error mes​ sage. b. from the drop-down list. and the next con​ troller image starts to up​ grade. Once the con​ troller re​ boots. choose Admin > Firmware. and sign in to the APIC controller as prompted. The con​ trollers up​ grade in ran​ dom order. When the APIC con​ troller that the browser is con​ nected to is up​ graded and it re​ boots.

perform the following substeps: a. The Work pane displays details about the firmware policy that was created earlier. "102. d. To verify that the firmware group was created. For example. 108". enter the name of the firmware group. perform the following actions: i. create one group with the even-numbered devices and the other group with the odd-numbered devices. which is the default mode. choose the Policy tab. you can choose to create a schedule for upgrading or leave the drop-down list blank so that you can upgrade on demand. click Maintenance Groups. "101. For example. iii. 3 If you have not created a firmware group. enter a comma-separated list or a range of node IDs to include in the group. In the Work pane. Note: In the Work pane. In the Scheduler drop-down list. v. Choose Actions > Create Firmware Group. 106. perform the following actions: i. enter a comma-separated list or a range of node IDs to include in the group. 110". in the Navigation pane. choose Fabric Node Firmware > Firmware Groups > new_firmware_group. choose Fabric Node Firmware. In the Group Name field. In the Target Firmware Version drop-down list. the switches that are operating in the fabric are displayed. . ii. b. c. Click Submit. "Even-Nodes". In the Group Node IDs field. 103-105. 104. perform the following substeps: a. choose Action > Create Maintenance Group. In the Create Firmware Group dialog box. In the Group Name field. c. 4 If you have not created maintenance groups. iii. For example. iv.Upgrading and Downgrading Firmware 62 2 In the Navigation pane. click the Pause Upon Upgrade Failure radio button. ii. 108. iv. enter the name of the maintenance group. choose the firmware version to which you will upgrade. b. In the Create Maintenance Group dialog box. In the Navigation pane. For example. In the Work pane. For the Run Mode radio buttons. In the Group Node IDs field. Click Submit. Note: Cisco recommends that you create two maintenance groups for all of the switches.

and the controllers in the cluster will not communicate for some time with the switches in the group. Note: In the Work pane. Once the switches rejoin the cluster after rebooting. In the Current Firmware column. in the Navigation pane. the Status displays that all the switches in the group are being upgraded simultaneously. . vii. connectivity drops. the scheduler pauses and manual intervention is required by the APIC administrator. view all of the switches that are listed. Note: In the Work pane. To verify that the maintenance group was created. 5 Right-click one of the maintenance groups that you created and choose Upgrade Now. click Fabric Node Firmware. 6 In the Upgrade Now dialog box. Therefore.Upgrading and Downgrading Firmware 63 vi. click Yes. When you use the GUI. The switch upgrade takes up to 12 minutes for each group. Repeat this step for the second maintenance group. and click the name of the maintenance group that you created. for Do you want to upgrade the maintenance group policy now?. choose Fabric Node Firmware > Maintenance Groups > new_maintenance_group. 7 Click OK. Note: The Work pane displays details about the maintenance policy. The default concurrency in a group is set at 20. 8 In the Navigation pane. The switches will reboot when they upgrade. In case of any failures. For example. you will see all the switches listed under the controller node. up to 20 switches at a time will get upgraded. Verify that the switches in the fabric are upgraded to the new image. the APIC per​ forms ad​ di​ tional ver​ if​ i​ ca​ tion and in​ tegrity checks on the soft​ ware image. the upgrade process will upgrade only one switch at a time out of the two switches in a VPC domain. Cisco rec​ om​ mends that you to per​ form the firmware up​ grade from the GUI. You do not need to up​ grade the cat​ al​ og firmware image sep​ ar​ ately. view the upgrade image details listed against each switch. a group named "Odd-Nodes". If there are any VPC configurations in the cluster. and then the next set of 20 switches are upgraded. Upgrading the APIC Controller Software Using the CLI The cat​ al​ og firmware image is up​ graded when an APIC con​ troller image is up​ graded.

Upgrading the Switch Software Using the CLI .1(1d) 2 Upgrade the firmware on the controllers.0(1. If you see "un​ known" in this field. Ex​ am​ ple: admin@apic1:~> firmware upgrade controllers ver_no.---------- 1 controller 1.202a) inprogress 0 The Up​ grade-Sta​ tus field will show "in​ queue".0(1.202a) inqueue -----------------0 2 controller 1.0(1. When the APIC con​ troller to which you have con​ nected com​ pletes up​ grad​ ing and re​ boots.bin The APIC con​ trollers are up​ graded se​ ri​ ally so that the con​ troller clus​ ter is avail​ able dur​ ing the up​ grade. Ex​ am​ ple: admin@apic1:~> firmware list Name : aci-apic-dk9.1.200) apic-1. or "com​ ple​ teok". Ex​ am​ ple: admin@apic1:~> firmware upgrade status Node-Id Role Current- Target- Upgrade- Progress-Percent Firmware Firmware Status (if inprogress) --------- ----------- ------------ -----------------. The up​ grade oc​ curs in the back​ ground. 3 Check the status of the upgrade.202a) inqueue 0 3 controller 1. you can close the SSH win​ dow.0(1.Upgrading and Downgrading Firmware 64 To up​ grade the APIC con​ troller soft​ ware using the CLI: 1 List the current software in the respository that was previously downloaded.0(1. that con​ troller has prob​ a​ bly up​ graded and is re​ boot​ ing.1d.1.0(1.200) apic-1.200) apic-1. "in​ progress".bin Type : controller Version : 1.

3 Check the upgrade status for the switch.iso" to ".---------- 1017 leaf n9000-11.Upgrading and Downgrading Firmware 65 Upgrading the Switch Software Using the CLI Be​ fore you up​ grade the switches. the APICs must have com​ pleted up​ grad​ ing and have a health state of Fully Fit. Ex​ am​ ple: admin@apic1:~> firmware upgrade switch node 101 ver_no.1.0(1.869S1) n9000-11.11.1(1d) completeok -----------------100 .1d.bin Type : switch Version : 11.1(1d) The name changes from ". To up​ grade the switch soft​ ware using the CLI 1 Check that the output of the following command appears like the output shown below.bin". 2 Upgrade the switches. The output that appears from the following command will appear like the following sample: Ex​ am​ ple: admin@apic1:~> firmware upgrade status node node_id Node-Id Role Current- Target- Upgrade- Progress-Percent Firmware Firmware Status (if inprogress) --------- ----------- ------------------- -----------------.bin Firmware Installation on Switch Scheduled You must up​ grade each switch sep​ ar​ ately. with the correct version number: Ex​ am​ ple: admin@apic1:~> firmware list Name : aci-n9000-dk9.

you will see mes​ sages about the num​ ber of nodes queued and the num​ ber in the process of up​ grad​ ing. . The fol​ low​ ing are the pos​ si​ ble up​ grade states for a node: • NotScheduled: No upgrade is currently scheduled for this node.Upgrading and Downgrading Firmware 66 You can check the sta​ tus of all nodes at once. 4 Repeat Steps 2 and 3 for each additional switch. • CompleteOK: Upgrade completed successfully. Troubleshooting Failures During the Upgrade Process There is one sched​ uler per main​ te​ nance pol​ icy. Verifying Cluster Convergence You can mon​ it​ or the progress of the clus​ ter con​ ver​ gence after a sched​ uled main​ te​ nance. • CompleteNOK: Upgrade failed on this node. • Scheduled: Upgrade is scheduled for this node. This may take a while. Once man​ ual in​ ter​ ven​ tion is com​ plete. by en​ ter​ ing the firmware up​ grade sta​ tus com​ mand. When all the clus​ ters have con​ verged suc​ cess​ fully. The sched​ uler ex​ pects man​ ual in​ ter​ ven​ tion to debug any up​ grade fail​ ures. before declaring failure). the sched​ uler pauses. when an up​ grade fail​ ure is de​ tected. • Inretryqueue: Node is queued again for upgrade retry (5 attempts are made requesting permission to upgrade. which pre​ sents you with a se​ ries of mes​ sages dur​ ing the process of one clus​ ter con​ verg​ ing and then the next clus​ ter. and no more nodes in that group begin to up​ grade. • Queued: There is a currently active window (schedule) and the node is • Inprogress: Upgrade is currently in progress on this node. As the con​ troller and switches move through the up​ grade. These mes​ sages are dis​ played in the Sta​ tus field. you must re​ sume the paused sched​ uler. as well as how many have up​ graded suc​ cess​ fully. you will see "No" in the Wait​ ing for Clus​ ter Con​ ver​ gence field of the Con​ troller Firmware screen. You view the Con​ troller Firmware screen on the GUI. By de​ fault.

​ github. • Is the switch maintenance group paused? The group will be paused if any switch fails its upgrade. then check the fol​ low​ ing: • Is the controller cluster healthy? The controller cluster must be healthy. switches which have not already started their upgrade will be in “queued” state. If you see “waitingForClusterHealth = yes” in the API or "Waiting for Cluster Convergence" showing "Yes" in the GUI.​ io/​ aci-troubleshooting-book/​ . If the sys​ tem takes longer than about 60 min​ utes for a switch to dis​ play “wait​ ing​ For​ Clus​ ter​ Health = no” in the API or "Wait​ ing for Clus​ ter Con​ ver​ gence" show​ ing "No" in the GUI.Upgrading and Downgrading Firmware 67 If you no​ tice that switches are in the “queued” state. you should work through the steps for ver​ if​ y​ ing a pause in the sched​ uler. see the Trou​ bleshoot​ ing Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture doc​ um ​ent at the fol​ low​ ing URL: https://​ datacenter. For ad​ di​ tional trou​ bleshoot​ ing pro​ ce​ dures. that means the controller cluster is not healthy. Until the controller cluster is healthy.

.

69 Fabric Connectivity .

.

Access Policies Domains VLAN Pools AEPs Policy Types Interface Policies Switch Policies Interface Policy Groups Switch Policy Groups Interface Profiles Switch Profiles Best Practices • Adding New Devices to the Fabric Sample Configuration Creating VLAN Pools Create VLAN Pool Create Physical Domain Create an Attachable Access Entity Profile (AEP) Interface Policies Interface Policy Groups Interface Profile​ ​ Switch Profiles Reusability Sample vPC Creation Create VLAN Pool ​ Create a Physical Domain 71 .Fabric Connectivity Section Content • Understanding Fabric Policies • Fabric .

Fabric Connectivity 72 Create Access Entity Profile Interface Policies Switch Profile Create vPC domain Validate Operation of Configured vPC • Server Connectivity Cisco UCS B-Series Servers Standalone Rack Mount Servers or Non-Cisco Servers • Virtual Machine Networking Understanding VM Networking in ACI ACI VM Integration Workflow VMware Integration VMM Policy Model Interaction Publishing EPGS to a VMM Domain Connecting VMs to the EPG Port Groups on vCenter Verifying Virtual Endpoint Learning Verifying VM Endpoint Learning on the APIC from the CLI VMware Integration Use Case • Deploying the Application Virtual Switch Prerequisites Getting Started Install the AVS VIB Manual Installation DHCP Relay Attachable Access Entity Profile (AEP) and AVS VMM Domains for vCenter AVS Switching Modes Create the VMM Domain for AVS Verify AVS Deployment on vCenter Add vSphere Hosts to the AVS .

Fabric Connectivity Verify AVS on ESX VXLAN Load Balancing IGMP Snooping Policy for AVS • External Connectivity ​Extending ACI to External Layer 2 Extending an ACI Bridge Domain Outside of the Fabric Extending Endpoint Groups Outside the ACI Fabric Extending ACI to External Layer 3 Supported Routing Protocols Configure MP-BGP Spine Route Reflectors Layer 3 Integration Through Tenant Network with OSPF NSSA External Layer 3 for Multiple Tenants • Application Migration Use Case Extending the Network to ACI 73 .

.

and stor​ age ar​ rays. It is im​ por​ tant that other teams such as server teams un​ der​ stand these con​ cepts as well. Fabric . For ex​ am​ ple. demon​ strate how to add and pre-pro​ vi​ sion switches. it is also key to un​ der​ stand the im​ pli​ ca​ tions of delet​ ing poli​ cies on the ACI fab​ ric. and basic man​ age​ ment poli​ cies. con​ fig​ ur​ ing do​ mains.Access Policies Domains EPGs are con​ sid​ ered the “who” in ACI. Do​ mains and AEPs are also cre​ ated in the ac​ cess poli​ cies view. AEPs can be con​ sid​ ered the "where" and do​ mains can be thought . The fab​ ric ac​ cess poli​ cies pro​ vide the fab​ ric with the base con​ fig​ ur​ a​ tion of the ac​ cess ports on the leaf switches. de​ vice dis​ cov​ ery and in​ ven​ tory man​ age​ ment. as they will be main​ tain​ ing these poli​ cies for the pur​ poses of in​ ter​ nal con​ nec​ tions be​ tween fab​ ric leaf nodes. The fab​ ric tab in the APIC GUI is used to con​ fig​ ure sys​ tem-level fea​ tures in​ clud​ ing. and walk through the steps and ob​ jects re​ quired when new de​ vices are added to the fab​ ric to ef​ fec​ tively op​ er​ ate an ACI fab​ ric. fab​ ric poli​ cies. di​ ag​ nos​ tic tools. or whether or not to run pro​ to​ cols like LACP on leaf switch in​ ter​ faces is set. The fab​ ric pane is split into three sec​ tions: in​ ven​ tory. but not lim​ ited to. par​ tic​ ul​ arly in the case of their build processes for adding ad​ di​ tional ca​ pac​ ity. While many poli​ cies are reusable. ac​ cess priv​ il​ eges. and ac​ cess poli​ cies. net​ work​ ing equip​ ment. This chap​ ter will re​ view the key ob​ jects in the ac​ cess poli​ cies sub​ sec​ tion of the fab​ ric tab – many of which are reusable. con​ nec​ tions to ex​ ter​ nal en​ ti​ ties such as servers. as they will be in​ ter​ act​ ing with them. The ac​ cess poli​ cies sub​ sec​ tion is split into fold​ ers sep​ ar​ at​ ing out dif​ fer​ ent types of poli​ cies and ob​ jects that af​ fect fab​ ric be​ hav​ ior. like port speed.Fabric Connectivity 75 Understanding Fabric Policies Now that ACME has been pro​ vi​ sioned with ACI fab​ ric and in​ fra​ struc​ ture space has been con​ fig​ ured be​ tween the leaf and spine switches. and switch and port be​ hav​ ior. con​ tracts are con​ sid​ ered the “what/when/why”. it is time to start cre​ at​ ing con​ nec​ tiv​ ity poli​ cies within the ACI fab​ ric. the in​ ter​ face poli​ cies folder is where port be​ hav​ ior is con​ fig​ ured. Un​ der​ stand​ ing how fab​ ric and ac​ cess poli​ cies con​ fig​ ure the fab​ ric is key for the ACME net​ work teams.

an ex​ ter​ nal bridged do​ main could be used to con​ nect an ex​ ist​ ing switch trunked-up to a leaf switch. not sub​ nets and VLANs.​ com/​ watch?​ v=_​ iQvoC9zQ_​ A VLAN Pools VLAN pools con​ tain the VLANs used by the EPGs the do​ main will be tied to. A do​ main is as​ so​ ci​ ated to a sin​ gle VLAN pool. By group​ ing do​ mains into AEPs and as​ so​ ci​ at​ ing them. There are four dif​ fer​ ent do​ main types: phys​ ic ​al do​ mains. and are used to group do​ mains with sim​ il​ ar re​ quire​ ments. AEPs At​ tach​ able Ac​ cess En​ tity Pro​ files (AEPs) can be con​ sid​ ered the "where" of the fab​ ric con​ fig​ ur​ a​ tion. For ex​ am​ ple. ex​ ter​ nal routed do​ mains. The fab​ ric op​ er​ at​ or cre​ ates the do​ mains.76 Fabric Connectivity of as the “how” of the fab​ ric. ex​ ter​ nal bridged do​ mains. Ex​ ter​ nal bridged do​ mains are used for Layer 2 con​ nec​ tions. an ex​ ter​ nal routed do​ main could be used to con​ nect a WAN router to the leaf switch.​ youtube. the fab​ ric knows where the var​ io ​us de​ vices in the do​ main live and the APIC can push the VLANs and pol​ icy where it needs to be. For​ ward​ ing de​ ci​ sions are still based on con​ tracts and the pol​ icy model. Ex​ ter​ nal routed do​ mains are used for Layer 3 con​ nec​ tions. AEPs are con​ fig​ ured under global poli​ cies. One or more do​ mains are added to an AEP. For ex​ am​ ple. VLANs are in​ stan​ ti​ ated on leaf switches based on AEP con​ fig​ ur​ a​ tion. and VMM do​ mains. check out the fol​ low​ ing video ti​ tled "How De​ vices Con​ nect to the Fab​ ric: Un​ der​ stand​ ing Cisco ACI Do​ mains": https://​ www. Do​ mains act as the glue be​ tween the con​ fig​ ur​ a​ tion done in the fab​ ric tab to the pol​ icy model and EPG con​ fig​ ur​ a​ tion found in the ten​ ant pane. VXLAN and mul​ ti​ cast ad​ dress pools are also con​ fig​ urable. Dif​ fer​ ent do​ main types are cre​ ated de​ pend​ ing on how a de​ vice is con​ nected to the leaf switch. . AEPs are tied to in​ ter​ face pol​ icy groups. Phys​ ic ​al do​ mains are gen​ er​ ally used for bare metal servers or servers where hy​ per​ vi​ sor in​ te​ gra​ tion is not an op​ tion. For an in-depth white​ board ex​ pla​ na​ tion on do​ mains. and the ten​ ant ad​ min​ is​ tra​ tors as​ so​ ci​ ate do​ mains to EPGs.

and a 1GE link level pol​ icy must be cre​ ated for de​ vices con​ nected at that speed. cov​ ered in the fol​ low​ ing para​ graphs. under the in​ ter​ face poli​ cies folder there are fold​ ers for con​ fig​ ur​ a​ tion called poli​ cies.in​ ter​ face se​ lec​ tors . there should be a pol​ icy that dic​ tates CDP is dis​ abled. Switch Policy Groups Switch pol​ icy groups allow lever​ ag​ ing of ex​ ist​ ing switch po​ lices like Span​ ning Tree and mon​ it​ or​ ing poli​ cies. Interface Profiles In​ ter​ face pro​ files help tie the pieces to​ gether. Interface Policies First. there should ide​ ally be a set of pol​ icy groups cre​ ated once and reused as new de​ vices are con​ nected to the fab​ ric. which are called ex​ plicit vPC pro​ tec​ tion groups in the APIC GUI. Note that the ports on the leaf switches de​ fault to 10GE. and pro​ files. Pol​ icy groups do not ac​ tu​ ally spec​ ify where the pro​ to​ cols and port be​ hav​ ior should be im​ ple​ mented.for ex​ am​ ple. Note the in​ ter​ face pol​ icy groups sim​ ply dic​ tate pol​ icy. pol​ icy groups. In​ ter​ face pro​ files con​ tain blocks of ports .and are also tied to the in​ ter​ face pol​ icy groups de​ scribed in the . There are three types of in​ ter​ face pol​ icy groups de​ pend​ ing on link type: in​ di​ vid​ ual. con​ fig​ ur​ ing vPC do​ mains.Fabric Connectivity 77 Policy Types Most of the poli​ cies fold​ ers have sub​ fold​ ers. For ex​ am​ ple. For ex​ am​ ple. These are also reusable ob​ jects as many de​ vices are likely to be con​ nected to ports that will re​ quire the same port con​ fig​ ur​ a​ tion. Ide​ ally. in​ ter​ face poli​ cies are cre​ ated to dic​ tate in​ ter​ face be​ hav​ ior. Interface Policy Groups In​ ter​ face pol​ icy groups are tem​ plates to dic​ tate port be​ hav​ ior and are as​ so​ ci​ ated to an AEP. are are later tied to in​ ter​ face pol​ icy groups. and vPC. In​ ter​ face pol​ icy groups use the poli​ cies de​ scribed in the pre​ vi​ ous para​ graph to spec​ ify how links should be​ have. Switch Policies There are also poli​ cies for switches .these can be reused as new de​ vices are con​ nected to the leaf switches. Max​ im ​iz​ ing reusabil​ ity of pol​ icy and ob​ jects makes day-to-day op​ er​ at​ ions ex​ po​ nen​ tially faster and eas​ ier to make large-scale changes. Just like poli​ cies. poli​ cies should be cre​ ated once and reused when con​ nect​ ing new de​ vices to the fab​ ric. CDPen​ abled. Port Chan​ nel. The "where" hap​ pens by as​ so​ ci​ at​ ing one or more in​ ter​ face pro​ files to a switch pro​ file. there may be a pol​ icy that dic​ tates 10GE. and a pol​ icy that dic​ tates CDP is en​ abled . For ex​ am​ ple.

and in​ ter​ face poli​ cies: Relationships to allow a physical interface or interfaces to be attached to an EPG Layer 2 In​ ter​ face Pol​ icy In Cisco ACI ver​ sion 1. This as​ so​ ci​ a​ tion pushes the con​ fig​ u​ ra​ tion to the in​ ter​ face. a new con​ fig​ urable In​ ter​ face Pol​ icy was added to allow a per port-VLAN sig​ nif​ i​ cance. such as e1/1. .VLAN en​ cap​ su​ la​ tion or VXLAN en​ cap​ su​ la​ tion. switch.78 Fabric Connectivity pre​ vi​ ous para​ graphs. The fol​ low​ ing fig​ ure high​ lights the re​ la​ tion​ ship be​ tween the var​ i​ ous global. switch pro​ files allow the se​ lec​ tion of one or more leaf switches and as​ so​ ci​ ate in​ ter​ face pro​ files to con​ fig​ ure the ports on that spe​ cific node. the pro​ file must be as​ so​ ci​ ated to a spe​ cific switch pro​ file (dis​ cussed in the next para​ graph) to con​ fig​ ure the ports. Switch Profiles Lastly. this is just an ar​ bi​ trary port. To con​ nect de​ vices to the ACI fab​ ric we can use un​ tagged traf​ fic. Again.1. and cre​ ates a Port Chan​ nel or vPC if one has been con​ fig​ ured in the in​ ter​ face pol​ icy.

and 10GE port speed. For example. It is expected that the port will flap when the Layer 2 interface policy changes between global and local. For example. When naming policies. • Per Port-VLAN limitations and considerations When per port-VLAN is used.Fabric Connectivity 79 In tra​ di​ tional net​ work​ ing one of the lim​ it​ a​ tions re​ lated to VLAN en​ cap​ su​ la​ tion is scal​ a-​ bil​ ity and re-us​ abil​ ity due to the limit of 4096 VLANs in net​ work​ ing de​ vices. traffic will get affected. 1GE port speed. • Policies Reuse policies whenever possible. it can be hard to remember what all the defaults are.V) is registered internally instead of just a VLAN encapsulation ID. This al​ lows ten​ ants to re-use VLAN en​ cap​ su​ la​ tion IDs thor​ ough the fab​ ric with​ out al​ low​ ing com​ mu​ ni​ ca​ tion be​ tween ten​ ants. How​ ever. a port and VLAN pair (P. These are not re​ quire​ ments and might not work for all en​ vi​ ron​ ments or ap​ pli​ ca​ tions. use names that clearly describe the setting. Best Practices Cisco has es​ tab​ lished sev​ eral best prac​ tices for fab​ ric con​ fig​ ur​ a​ tion. there can be a set of interface policy groups for all VMware ESXi servers connected via 10GE vPCs. a policy that enables LACP in mode active could be called "LACPActive". There are many "default" policies out of the box. with the de​ fault con​ fig​ ur​ a​ tion (global). there should be policies for LACP active/passive/off. global con​ fig​ ur​ a​ tion as​ sumes that ten​ ants do not share leaf switches and there​ fore there is no VLAN over​ lap​ ping within the same leaf. This increases the consumption of hardware resources at a per switch level. For example. but might help sim​ plify day-to-day op​ er​ at​ ion of the ACI fab​ ric. In ACI. and a different set of interface policy groups for all bare metal servers running . A set of interface policy groups should be created for each type of similar devices connected. Two EPGs belonging to a single Bridge Domain cannot share the same encapsulation ID on a given leaf switch. That is. EPGs can use the same VLAN en​ cap​ su​ la​ tion as long as EPGs are bound to sep​ ar​ ate switches. which is why policies should be clearly named to avoid making a mistake when adding new devices to the fabric. However.

or to limit the scope of the presence of VLANs across the fabric. each AEP will have its own set of interface policy groups. Create a switch profile for each leaf switch individually. a single VMM domain can be created and associated with all leaf ports where VMware ESXi servers are connected. such as overlapping VLAN pools. and additionally. • Domains Build one physical domain per tenant for bare metal servers or servers without hypervisor integration requiring similar treatment. • AEPs Multiple domains can be associated to a single AEP for simplicity's sake. There are some cases where multiple AEPs may need to be configured to enable the infrastructure VLAN. Since interface policy groups are tied to a single AEP. . Build one physical domain per tenant for external connectivity. If a VMM domain needs to be leveraged across multiple tenants.Fabric Connectivity 80 1GE with CDP disabled. create a switch profile for each vPC pair (if using vPC).

The re-us​ abil​ ity of these poli​ cies makes it pos​ si​ ble to repli​ cate the con​ fig​ u​ ra​ tion of a switch very eas​ ily. with a focus on how to re-use these pro​ files across the fab​ ric. . The fol​ low​ ing di​ a​ gram de​ picts how this re-us​ abil​ ity sim​ pli​ fies the op​ er​ a​ tion of the fab​ ric over time.Fabric Connectivity 81 Adding New Devices to the Fabric This sec​ tion will demon​ strate how to con​ fig​ ure ACI to re-use the fab​ ric ac​ cess poli​ cies. This sec​ tion will walk through set​ ting up pro​ files from scratch. these var​ i​ ous pro​ files are linked to​ gether and have de​ pen​ den​ cies. As out​ lined in the pre​ vi​ ous sec​ tion. sim​ pli​ fy​ ing day-to-day op​ er​ a​ tion of the fab​ ric. The fol​ low​ ing di​ a​ gram re​ it​ er​ ates the ob​ ject re​ la​ tion​ ships: Object relationships Whereas a tra​ di​ tional com​ mand line in​ ter​ face on a switch gen​ er​ ally re​ quires a port-byport confugu​ ra​ tion. ACI al​ lows de​ f​ i​ n​ i​ tion of ob​ jects and poli​ cies that can be re-used.

or spe​ cial​ ized ap​ pli​ ca​ tions. the con​ fig​ u​ ra​ tion of a cou​ ple of switches does not re​ quire many processes or au​ toma​ tion. These can be net​ work man​ age​ ment tools. This is what is de​ picted as the pol​ icy reuse in​ flec​ tion point in the pre​ vi​ ous di​ a​ gram. and the ver​ i​ fi​ ca​ tion steps to en​ sure proper con​ fig​ u​ ra​ tion. Lever​ ag​ ing the Cisco ACI pol​ icy model. These are the steps to be taken in the APIC GUI when new de​ vices are con​ nected to the leaf switches to en​ sure the ac​ cess ports on the leaf switches have the right switch​ port con​ fig​ u​ ra​ tion. an op​ er​ a​ tor can lever​ age pro​ files to stream​ line the op​ er​ a​ tion of adding de​ vices and man​ ag​ ing those de​ vices. In tra​ di​ tional net​ works. when changes that im​ pact a large set of de​ vices need to be made. and vPC-con​ nected de​ vices from scratch. and will in​ clude a re​ view of the ob​ jects as they are con​ fig​ ured. au​ toma​ tion be​ comes more and more crit​ i​ cal as it has a di​ rect im​ pact on the cost of busi​ ness op​ er​ a​ tions. Sample Configuration The fol​ low​ ing sec​ tions will walk through sam​ ple con​ fig​ u​ ra​ tion of set​ ting up in​ di​ vid​ u​ ally con​ nected de​ vices. . Port Chan​ nel-con​ nected de​ vices. As the data cen​ ter size in​ creases. scripts. The fol​ low​ ing steps rep​ re​ sent the use case of adding a new bare metal server con​ nected to a leaf switch. the op​ er​ a​ tor is faced with the cost of de​ sign​ ing processes to man​ age these de​ vices.82 Fabric Connectivity Policy Re-use In any data cen​ ter.

vPC ex​ tends link ag​ gre​ ga​ tion to two sep​ a​ rate phys​ i​ cal switches. as all links are actively forwarding • Your server configurations are simplified since the configurations simply appears as port-channels without the need for special software. This sec​ tion of the chap​ ter at​ tempts to clar​ ify at a high level what vPC's are. With​ out vPC's. from a driver or kernel-tuning standpoint . the ben​ e​ fits they pro​ vide and how vPC's in the ACI fab​ ric dif​ fer from how they are de​ ployed on Cisco Nexus switches run​ ning NX-OS soft​ ware. At a high level. This al​ lows for sev​ eral key ad​ van​ tages from a server de​ ploy​ ment per​ spec​ tive: • You can create resilient Layer 2 topologies based on link aggregation • You do not need STP • You have increased bandwidth.Fabric Connectivity 83 Be​ fore get​ ting into the con​ fig​ u​ ra​ tion of vPC's. a sin​ gle server is dual homed to two dif​ fer​ ent switches for re​ dun​ dancy. the server will likely use an ac​ tive-standby con​ fig​ u​ ra​ tion. By con​ fig​ ur​ ing ports on two dif​ fer​ ent switches as the same port-chan​ nel and using an in​ ter-switch mes​ sag​ ing chan​ nel (such as the in​ ter-switch port-chan​ nel in the green box on the left hand side) to cover re​ dun​ dancy sce​ nar​ ios. we pro​ vide a log​ i​ cal topol​ ogy that greatly sim​ pli​ fies server pro​ vi​ sion​ ing and man​ age​ ment. vPC Topology In the fig​ ure above. it is im​ por​ tant to un​ der​ stand what vPC's are and how they are dif​ fer​ ent from tra​ di​ tional meth​ ods of server con​ nec​ tiv​ ity. which are a pop​ u​ lar server con​ nec​ tiv​ ity method​ ol​ ogy. or a spe​ cial con​ fig​ u​ ra​ tion on the NIC dri​ ver or the ker​ nel that al​ lows it to in​ tel​ li​ gently loadbal​ ance traf​ fic using an al​ go​ rithm.

The fig​ ure below shows a sin​ gle tra​ di​ tional Layer 2 switch con​ nected to a VPC en​ abled Cisco switch pair. to pro​ vide sim​ i​ lar ben​ e​ fits. Legacy connectivity compared to vPC The com​ po​ nents of a tra​ di​ tional vPC do​ main are il​ lus​ trated below: Traditional vPC topology .84 Fabric Connectivity vPCs can also be used to con​ nect other down​ stream de​ vices. such as Cisco UCS fab​ ricin​ ter​ con​ nects.

Fabric Connectivity 85 As il​ lus​ trated above. The rich in​ ter​ con​ nec​ tiv​ ity be​ tween fab​ ric nodes makes it un​ likely that peers will have an ac​ tive path be​ tween them. The ACI fab​ ric greatly sim​ pli​ fies VPC con​ fig​ u​ ra​ tions. typ​ i​ cally on the out-of-band man​ age​ ment port. . There are also no keepalives being sent on the man​ age​ ment ports. there is no re​ quire​ ment for set​ ting up vPC peer-links. vPC topology in ACI The key dif​ fer​ ences to note here are that rel​ a​ tive to tra​ di​ tional vPC de​ sign. The fab​ ric it​ self serves as the peer-link. Note that at​ tempt​ ing to cable a leaf switch to an​ other leaf switch will lead to a "wiring mis​ match" fault in the GUI and re​ sult in a black​ listed port that will have to be man​ u​ ally re​ cov​ ered. that is used to de​ ter​ mine peer live​ li​ ness to de​ tect a vPC peer-switch fail​ ure. in Cisco switch​ ing prod​ ucts run​ ning NX-OS soft​ ware. There is also a peer-keepalive link. Mak​ ing con​ fig​ u​ ra​ tion changes in such sce​ nar​ ios with​ out the con​ fig-sync fea​ ture en​ abled may lead to sce​ nar​ ios where there are mis​ matched vPC pa​ ra​ me​ ters be​ tween the vPC pri​ mary and the vPC sec​ ondary switches that may cause par​ tial con​ nec​ tiv​ ity loss dur​ ing the change it​ self if a type-1 in​ con​ sis​ tency is de​ tected. vPC con​ fig​ u​ ra​ tions need to be done man​ u​ ally by the op​ er​ a​ tor and re​ quire a pair of ded​ i​ cated "in​ ter-switch" links also called a peer-link.

static vs lacp LACP: Lag ID The fol​ low​ ing di​ ag ​rams il​ lus​ trate how the ACI fab​ ric for​ wards traf​ fic from a vPC do​ main to a non-vPC con​ nected host in the fab​ ric. • Global type-1 parameters: STP • Interface type-1 parameters: STP: Only BPDU Guard is configurable EthPM Port speed Duplex mode Port mode MTU Native VLAN PCM: Channel mode. it is very unlikely that all the redundant paths between vPC peers fail at the same time. . • In the ACI fabric. it is assumed to have crashed. • Role is used in case of vpc type-1 consistency failure. • Role election still happens.Fabric Connectivity 86 The fol​ low​ ing are some other key be​ hav​ ioral changes to vPC as it ap​ plies to the ACI fab​ ric rel​ at​ ive to clas​ sic vPC that are im​ por​ tant for op​ er​ at​ ors to un​ der​ stand: • Configurations are automatically synchronized to avoid an error-free configuration by the APIC which is the central point of control for all configurations in the ACI fabric. and vice-versa. A list of type-1 parameters used for consistency checking for a given vPC domain specific to the ACI fabric are listed below. Slave switch brings down all its vPC ports. The slave switch does not bring down vPC links. the slave switch brings down all its vPC links if the MCT goes down. • In traditional vPC solution. peers assume master-slave roles. Hence if the peer switch becomes unreachable.

4 S2 delivers the pkt to locally attached host H1. 3 Spine switch routes to S3. Creating VLAN Pools In this ex​ am​ ple.Fabric Connectivity 87 vPC forwarding Uni​ cast packet flow H2 -> H1 1 H2 sends a pkt towards H1 on its link to S3. con​ fig​ ur​ ing newly-con​ nected bare metal servers first re​ quires cre​ ation of a phys​ i​ cal do​ main and then as​ so​ ci​ a​ tion of the do​ main to a VLAN pool. 3 Spine switch sees multiple routes for VIP and picks one of them (S2 in this case). . As men​ tioned in the pre​ vi​ ous sec​ tion. 2 S1 does a table lookup and routes with IP of S3. 4 S3 delivers the pkt to locally attached host H2. 2 S3 does a table lookup and routes with vPC Virtual IP (VIP). VLAN pools de​ fine a range of VLAN IDs that will be used by the EPGs. H1 -> H2 1 H1 sends a pkt towards H2 on one of its PC link (S1 in this case).

1Q VLAN tags.Fabric Connectivity 88 The servers are con​ nected to two dif​ fer​ ent leaf nodes in the fab​ ric. phys​ i​ cal port IDs. The leaf switches nor​ mal​ ize the traf​ fic by strip​ ping off tags and reap​ ply​ ing the re​ quired tags on fab​ ric egress. choose Pools > VLAN. the ACI fab​ ric can also act as a gate​ way be​ tween dis​ parate en​ cap​ su​ la​ tion types such as un​ tagged traf​ fic. ACI has to know be​ fore​ hand how to clas​ sify pack​ ets into the dif​ fer​ ent EPGs. VXLAN VNIDs. 4 In the Create VLAN Pool dialog box. The range of VLANs used in the con​ fig​ u​ ra​ tion ex​ am​ ple is 100-199. provide a description for the VLAN pool. choose Actions > Create VLAN Pool. When a packet ar​ rives ingress to a leaf switch in the fab​ ric. There are two allocation modes: dynamic or static. 2 In the Navigation pane. using iden​ ti​ fiers like VLANs. perform the following actions: a. choose Fabric > Access Policies. 802. vir​ tual port IDs. the APIC selects VLANs from the pool dynamically. This is common in VMM integration mode where the actual . 3 In the Work pane. c. it is im​ por​ tant to un​ der​ stand that the de​ f​ i​ n​ i​ tion of VLANs as they per​ tain to the leaf switch ports is uti​ lized only for iden​ ti​ fi​ ca​ tion pur​ poses. NVGRE. Note: When dynamic allocation is selected. and NVGRE tags. Optionally. Encapsulation normalization Create VLAN Pool 1 On the menu bar. In ACI. VXLAN. As de​ picted in the fol​ low​ ing fig​ ure. b. Define a meaningful name for the VLAN pool.1Q or VXLAN en​ cap​ su​ la​ tion. Each server will be tag​ ging using 802.

538-08:00" name="" rn="from-[vlan-100]-to-[vlan-199]" status="" to="vlan199" uid="8131"/> </fvnsVlanInstP> Create Physical Domain A phys​ ic ​al do​ main acts as the link be​ tween the VLAN pool and the Ac​ cess En​ tity Pro​ file (AEP). choose Fabric > Access Policies. b. The cre​ ation of the AEP and its as​ so​ ci​ at​ ion will be cov​ ered later in this sec​ tion. XML Ob​ ject <fvnsVlanInstP allocMode="static" childAction="" configIssues="" descr="" dn="uni/infra/vlanns-[bsprint-vlan-pool]-static" lcOwn="local" modTs="2015-02- 23T15:58:33. . In the Create Physical Domain dialog box. Static allocation is typically used when the pool will be referenced from a static source like a static path binding for an EPG for use with bare metal servers. The do​ main also ties the fab​ ric con​ fig​ ur​ a​ tion to the ten​ ant con​ fig​ ur​ a​ tion. choose Actions > Create Physical Domain. The encap blocks are used to define the range of VLANs in the VLAN pool. Define a meaningful name for the profile. choose Physical and External Domains > Physical Domains. perform the following actions: a.538-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-vlan-pool" ownerKey="" ownerTag="" status="" uid="8131"> <fvnsRtVlanNs childAction="" lcOwn="local" modTs="2015-02-25T11:35:33. Remember it's the EPG itself which policies are applied to.365-08:00" rn="rtinfraVlanNs-[uni/l2dom-JC-L2-Domain]" status="" tCl="l2extDomP" tDn="uni/l2domJC-L2-Domain"/> <fvnsRtVlanNs childAction="" lcOwn="local" modTs="2015-02-23T16:13:22. Select the VLAN pool you just created. 2 In the Navigation pane. When con​ fig​ ur​ ing in this order. while the do​ mains are cre​ ated under the fab​ ric tab. d.Fabric Connectivity 89 VLAN ID is not important. 3 4 In the Work pane. Note multiple ranges can be added to a single pool. 1 On the menu bar. as the ten​ ant ad​ min​ is​ tra​ tor is the one who as​ so​ ci​ ates do​ mains to EPGs. only the pro​ file name and the VLAN pool are con​ fig​ ured.007-08:00" rn="rtinfraVlanNs-[uni/phys-bsprint-PHY]" status="" tCl="physDomP" tDn="uni/physbsprint-PHY"/> <fvnsEncapBlk childAction="" descr="" from="vlan-100" lcOwn="local" modTs="2015-02- 23T15:58:33.

b.945-08:00" rn="rtdomP-[uni/infra/attentp-bsprint-AEP]" status="" tCl="infraAttEntityP" tDn="uni/infra/attentp-bsprint-AEP"/> </physDomP> Create an Attachable Access Entity Profile (AEP) The AEP links the phys​ ic ​al do​ main and its VLAN Pool to the in​ ter​ face poli​ cies.065-08:00" rType="mo" rn="rsvlanNsDef" state="formed" stateQual="none" status="" tCl="fvnsAInstP" tDn="uni/infra/vlanns-[bsprint-vlan-pool]-static" tType="mo"/> <infraRtDomP childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.Fabric Connectivity 90 XML Ob​ ject <physDomP childAction="" configIssues="" dn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:21. The con​ fig​ ur​ a​ tion for an AEP is straight​ for​ ward. . Define the a meaningful name for the profile. 1 On the menu bar. d. Click + to associate the domain to the AEP. 5 Click Next. choose Fabric > Access Policies. 6 Click Submit. c.065-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsvlanNs" state="formed" stateQual="none" status="" tCl="fvnsVlanInstP" tDn="uni/infra/vlanns[bsprint-vlan-pool]-static" tType="mo" uid="8131"/> <infraRsVlanNsDef childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:22. Optionally. 3 In the Work pane. choose Actions > Create Attached Entity Profile. enter a description. 2 In the Navigation pane. perform the following actions: a. 4 In the Create Attached Entity Profile dialog box. choose Global Policies > Attached Acess Entity Profle.906-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-PHY" ownerKey="" ownerTag="" status="" uid="8131"> <infraRsVlanNs childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:22. Select the physical domain that was previously configured.

980-08:00" rn="rtattEntP-[uni/infra/funcprof/accportgrp-bsprint-AccessPort]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-bsprint-AccessPort"/> <infraRsDomP childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 25T11:35:33.961-08:00" monPolDn="uni/fabric/monfab-default" profileDn="" rn="source-[uni/phys-bsprint-PHY]" status=""/> </infraRsToEncapInstDef> </infraContNS> <infraRtAttEntP childAction="" lcOwn="local" modTs="2015-02-24T11:59:37. de​ fine the in​ ter​ face pro​ files and show​ case the re-us​ abil​ ity of the fab​ ric poli​ - .570-08:00" monPolDn="uni/fabric/monfab-default" profileDn="" rn="source[uni/l2dom-JC-L2-Domain]" status=""/> <fabricCreatedBy childAction="" creatorDn="uni/phys-bsprint-PHY" deplSt="" domainDn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:52.961-08:00" rn="assocdomp-[uni/phys-bsprint-PHY]" status=""/> <infraAssocDomP childAction="" dompDn="uni/l2dom-JC-L2-Domain" lcOwn="local" modTs="2015-02-25T11:35:33.961-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsdomP[uni/phys-bsprint-PHY]" state="formed" stateQual="none" status="" tCl="physDomP" tDn="uni/phys-bsprint-PHY" tType="mo" uid="8131"/> </infraAttEntityP> Create Interface Policies Next.874-08:00" monPolDn="uni/fabric/monfab-default" name="bsprint-AEP" ownerKey="" ownerTag="" status="" uid="8131"> <infraContDomP childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.570-08:00" rn="assocdomp-[uni/l2dom-JC-L2-Domain]" status=""/> </infraContDomP> <infraContNS childAction="" lcOwn="local" modTs="2015-02-23T16:13:52.874-08:00" monPolDn="uni/fabric/monfab-default" rn="nscont" status=""> <infraRsToEncapInstDef childAction="" deplSt="" forceResolve="no" lcOwn="local" modTs="2015-02-23T16:13:52.874-08:00" rn="dompcont" status=""> <infraAssocDomP childAction="" dompDn="uni/phys-bsprint-PHY" lcOwn="local" modTs="2015-02-23T16:13:52.961-08:00" monPolDn="uni/fabric/monfabdefault" rType="mo" rn="rstoEncapInstDef-[allocencap-[uni/infra]/encapnsdef- [uni/infra/vlanns-[bsprint-vlan-pool]-static]]" state="formed" stateQual="none" status="" tCl="stpEncapInstDef" tDn="allocencap-[uni/infra]/encapnsdef[uni/infra/vlanns-[bsprint-vlan-pool]-static]" tType="mo"> <fabricCreatedBy childAction="" creatorDn="uni/l2dom-JC-L2-Domain" deplSt="" domainDn="uni/l2dom-JC-L2-Domain" lcOwn="local" modTs="2015-02- 25T11:35:33.570-08:00" monPolDn="uni/fabric/monfab-default" rType="mo" rn="rsdomP- [uni/l2dom-JC-L2-Domain]" state="formed" stateQual="none" status="" tCl="l2extDomP" tDn="uni/l2dom-JC-L2-Domain" tType="mo" uid="8754"/> <infraRsDomP childAction="" forceResolve="no" lcOwn="local" modTs="2015-02- 23T16:13:52.Fabric Connectivity 91 XML Ob​ ject <infraAttEntityP childAction="" configIssues="" descr="" dn="uni/infra/attentp- bsprint-AEP" lcOwn="local" modTs="2015-02-23T16:13:52.

XML Ob​ ject <fabricHIfPol autoNeg="on" childAction="" descr="" dn="uni/infra/hintfpol-1G-Auto" lcOwn="local" linkDebounce="100" modTs="2015-01-14T06:47:15. 2 In the Navigation pane. Optionally. d. 3 In the Work pane. c. 4 In the Create Link Level Policy dialog box. choose Actions > Create Link Level Policy. Define the meaningful name for the policy. Leaf switch ports default to 10GE. Change the de-bounce interval if required. provide a description for the policy. 2 In the Navigation pane. 3 In the Work pane. Select the auto negotiation mode for the interface.331- 08:00" rn="rtinfraHIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example"/> </fabricHIfPol> Create a CDP Interface Policy 1 On the menu bar. choose Interface Policies > Policies > CDP Interface. choose Actions > Create CDP Interface Policy.Fabric Connectivity 92 cies. choose Interface Policies > Policies > Link Level.693-08:00" name="1G-Auto" ownerKey="" ownerTag="" speed="1G" status="" uid="15374"> <fabricRtHIfPol childAction="" lcOwn="local" modTs="2015-01-14T06:48:48. Define a meaningful name for the policy such as 'CDP-Enable'. b. 5 Click Submit. perform the following actions: a. perform the following actions: a. . This sec​ tion will il​ lus​ trate cre​ ation of new pro​ files. e. choose Fabric > Access Policies. 1 On the menu bar. In​ ter​ face poli​ cies can be re-used as needed by dif​ fer​ ent in​ ter​ face pro​ file de​ f​ in ​i​ t​ ion re​ quire​ ments. Select the interface speed.081- 08:00" rn="rtinfraHIfPol-[uni/infra/funcprof/accportgrp-UCS-1G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-1G-PG"/> <fabricRtHIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. Create Link Level Policies Link level poli​ cies dic​ tate con​ fig​ ur​ a​ tion like the speed of ports. but ide​ ally there are al​ ready poli​ cies in place that can sim​ ply be se​ lected. 4 In the Create CDP Interface Policy dialog box. choose Fabric > Access Policies.

choose Actions > Create LLDP Interface Policy.331- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example"/> </cdpIfPol> Create an LLDP Interface Policy 1 On the menu bar.081- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-UCS-1G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-1G-PG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-24T11:59:37. 2 In the Navigation pane.980- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-bsprint-AccessPort]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-bsprintAccessPort"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. 4 In the Create LLDP Interface Policy dialog box. c. ​ . c. b. Optionally. choose Fabric > Access Policies.Fabric Connectivity 93 b. perform the following actions: a.957- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accportgrp-UCS-10G-PG]" status="" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-UCS-10G-PG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. Choose the receive state. Choose either admin state enabled or disabled. provide a description for the policy. choose Interface Policies > Policies > LLDP Interface.470-08:00" name="CDP-Enable" ownerKey="" ownerTag="" status="" uid="15374"> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-01-14T07:23:54. XML Ob​ ject <cdpIfPol adminSt="enabled" childAction="" descr="" dn="uni/infra/cdpIfP-CDP-Enable" lcOwn="local" modTs="2015-01-14T06:47:25. 5 Click Submit. provide a description for the policy. Optionally. Define a meaningful name for the policy. Choose the transmit state. 3 In the Work pane. 5 Click Submit. d.154- 08:00" rn="rtinfraCdpIfPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" status="" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG"/> <cdpRtCdpIfPol childAction="" lcOwn="local" modTs="2015-01-14T06:48:48.

perform the following actions: a.331- 08:00" rn="rtinfraLldpIfPol-[uni/infra/funcprof/accportgrp-L3-Example]" tCl="infraAccPortGrp" tDn="uni/infra/funcprof/accportgrp-L3-Example" </lldpIfPol> l​ ows Create an LACP Interface Policy ​ faces.Fabric Connectivity 94 XML Object descr="" <lldpIfPol adminRxSt="enabled" adminTxSt="enabled" childAction="" dn="uni/infra/lldpIfP-LLDP-Enable" lcOwn="local" modTs="2015-02-11T07:40:35. b. Optionally. 2 In the Navigation pane. choose Actions > Create LACP Policy. choose Fabric > Access Policies.664-08:00" status="" name="LLDP-Enable" ownerKey="" ownerTag="" status="" uid="15374"> /> <lldpRtLldpIfPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. . 1 On the menu bar. Note if LACP is enabled on the leaf switch. provide a description for the policy. d. c. Define a meaningful name for the policy.3ad) that al r​ a​ tion re​ quire​ ments an op​ tional ne​ go​ ti​ at​ ion pro​ to​ col to be run on Port Chan​ nel in​ ter pre-emp​ tively de​ fined to be used as needed for the var​ io ​us con​ fig​ u in the data cen​ ter. 5 Click Submit.154- 08:00" rn="rtinfraLldpIfPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG" status="" /> <lldpRtLldpIfPol childAction="" lcOwn="local" modTs="2015-02-25T11:48:11. choose Interface Policies > Policies > LACP 3 In the Work pane. LACP must also be enabled on the server or other connected device. These can be Link Ag​ gre​ ga​ tion Con​ trol Pro​ to​ col is part of an IEEE spec​ if​ i​ ca​ tion (802. specify the minimum and maximum number of links in the Port Channel. . Optionally. Select the LACP mode required for the server. 4 In the Create LACP Policy dialog box.

547-08:00" mode="active" name="LACP-Active" ownerKey="" ownerTag="" status="" uid="8131"> <lacpRtLacpPol childAction="" lcOwn="local" modTs="2015-02-24T14:59:11. provide a description for the policy. choose Actions > Create Spanning Tree Interface Policy. It is a com​ mon best prac​ tice to en​ able BPDU guard on in​ ter​ faces con​ nected to servers. 2 In the Navigation pane. Define a meaningful name for the policy. d. . choose Fabric > Access Policies. choose Fabric > Access Policies. 5 Click Submit. choose Interface Policies > Policies > Spanning Tree Interface. If required.154- 08:00" rn="rtinfraLacpPol-[uni/infra/funcprof/accbundle-ACI-VPC-IPG]" status="" tCl="infraAccBndlGrp" tDn="uni/infra/funcprof/accbundle-ACI-VPC-IPG"/> </lacpLagPol> Create an LACP Member Profile (optional) Op​ tion​ ally.Fabric Connectivity 95 XML Ob​ ject <lacpLagPol childAction="" ctrl="fast-sel-hot-stdby. The Span​ ning Tree in​ ter​ face pol​ icy sim​ ply de​ fines the port be​ hav​ ior.graceful-conv. 4 In the Create LACP Member Policy dialog box.susp-individual" descr="" dn="uni/infra/lacplagp-LACP-Active" lcOwn="local" maxLinks="16" minLinks="1" modTs="2015-02-24T11:58:36. choose Actions > Create LACP Member Policy. change the transmit rate. b. 1 On the menu bar. choose Interface Policies > Policies > LACP Member. 3 In the Work pane. 2 In the Navigation pane. c. 3 In the Work pane. change the priority. If required. Note: ACI does not run Span​ ning Tree on the fab​ ric be​ tween the leaves and spines. 1 On the menu bar. Optionally. Create a Spanning Tree Interface Policy (optional) The Span​ ning Tree pol​ icy dic​ tates the be​ hav​ ior of south​ bound leaf port Span​ ning Tree fea​ tures. perform the following actions: a. the LACP mem​ ber pro​ file pro​ vides the abil​ ity to pro​ vide pri​ or​ ity spec​ if​ i​ ca​ tions to mem​ bers of an LACP group.

Fabric Connectivity

96

4

In the Create Spanning Tree Interface Policy dialog box, perform the following
actions:
a. Define a meaningful name for the policy.
b. Optionally, provide a description for the policy.
c. Enable BPDU filter and/or BPDU guard.

5

Click Submit.

Create a Storm Control Policy (optional)

A traf​
fic storm oc​
curs when pack​
ets flood the LAN, cre​
at​
ing ex​
ces​
sive traf​
fic and de​
grad​
ing net​
work per​
for​
mance. The traf​
fic storm con​
trol fea​
ture can be used to pre​
vent dis​
rup​
tions on ports by a broad​
cast, mul​
ti​
cast, or uni​
cast traf​
fic storm on phys​
i​
cal in​
ter​
faces.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policies > Storm Control.

3

In the Work pane, choose Actions > Create Storm Control Policy.

4

In the Create Storm Control Policy dialog box, perform the following actions:
a. Define a meaningful name for the policy.
b. Optionally, provide a description for the policy.
c. Specify how the control policy is to be applied, either through percentage of
the total bandwidth or as a packet per second definition that matches the
requirement for the data center

5

Click Submit.

Creating a Layer 2 Interface Policy to enable per port-VLAN

1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policies > L2 Interface.

3

In the Work pane, choose Actions > Create L2 Interface Policy.

4

In the Create L2 Interface Policy dialog box, perform the following actions:
a. Give the L2 Interface name and an optional description.
b. Select VLAN scope to Port Local scope to enable per port-VLAN.

Create Interface Policy Groups
The in​
ter​
face pol​
icy groups com​
prise the in​
ter​
face poli​
cies as a func​
tional group that is
as​
so​
ci​
ated to an in​
ter​
face. The fol​
low​
ing di​
ag
​ram shows how pre​
vi​
ously cre​
ated items
are grouped under the pol​
icy group.

Fabric Connectivity

97

Policies contained in a policy group
Once all the in​
ter​
face poli​
cies have been de​
fined, the in​
di​
vid​
ual poli​
cies can be brought
to​
gether to form a pol​
icy group that will be linked to the in​
ter​
face pro​
file. The pol​
icy
group is de​
fined from a mas​
ter de​
f​
i​
n​
i​
tion that en​
com​
passes being one of the fol​
low​
ing:

Access Policy Group

Port Channel Policy Group

VPC Policy Group

Create Access Port Policy Group

The ac​
cess port pol​
icy is de​
fined for an in​
di​
vid​
ual link (non-Port Chan​
nel or vPC).
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create Access Policy Group.

4

In the Create Access Policy Group dialog box, perform the following actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Use the profiles created previously that are relevant for this policy group.

5

Click Submit.

Fabric Connectivity

98

Create Port Channel Interface Policy Group

Port Chan​
nel​
ing also load-bal​
ances traf​
fic across the phys​
ic
​al in​
ter​
faces that are mem​
bers of the chan​
nel group. For every group of in​
ter​
faces that needs to be con​
fig​
ured
into a port chan​
nel, a dif​
fer​
ent pol​
icy group has to be cre​
ated. This pol​
icy group de​
fines
the be​
hav​
iour. For ex​
am​
ple, if ports 1/1-4 are to be con​
fig​
ured into one port chan​
nel, and ports 1/5-8 into a sep​
ar​
ate port chan​
nel, each of those groups would re​
quire
the cre​
ation of a sep​
ar​
ate pol​
icy group.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create PC Interface Policy Group.

4

In the Create PC Interface Policy Group dialog box, perform the following
actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Select the policies created previously that are relevant for this PC policy
group.

5

Click Submit.

Create VPC Interface Policy Group

Note: This ob​
ject must be unique for each VPC cre​
ated.
A vir​
tual PortChan​
nel (vPC) al​
lows links that are phys​
ic
​ally con​
nected to two dif​
fer​
ent
de​
vices to ap​
pear as a sin​
gle Port Chan​
nel to a third de​
vice. In the world of ACI, pairs of
leaf switches may be con​
fig​
ured in a vPC do​
main so that down​
stream de​
vices can be
ac​
tive-ac​
tive dual-homed.
For every group of in​
ter​
faces that are to be con​
fig​
ured into a vPC, a dif​
fer​
ent in​
ter​
face
pol​
icy group needs to be cre​
ated. The vPC pol​
icy group con​
tains both the de​
f​
in
​i​
t​
ion for
the be​
hav​
iour of the port chan​
nel, and the iden​
ti​
fier. For ex​
am​
ple, if ports 1/1-4 are to
be con​
fig​
ured into one vPC across two switches, and ports 1/5-8 into a sep​
ar​
ate vPC
across two switches, each of those groups would re​
quire the de​
f​
in
​i​
t​
ion of a sep​
ar​
ate
pol​
icy group.
Note: For vPC you will also re​
quire a unique vPC do​
main de​
f​
in
​i​
t​
ion be​
tween the two
paired switches. More de​
tails to fol​
low.
1

On the menu bar, choose Fabric > Access Policies.

Fabric Connectivity

2

In the Navigation pane, choose Interface Policies > Policy Groups.

3

In the Work pane, choose Actions > Create VPC Interface Policy Group.

4

In the Create VPC Interface Policy Group dialog box, perform the following

99

actions:
a. Define a meaningful name for the policy group.
b. Optionally, provide a description for the policy group.
c. Choose the policies created previously that are relevant for this vPC policy
group.
5

Click Submit.

Interface Profile
The in​
ter​
face pro​
file in ACI links the pol​
icy groups that de​
fine how the in​
ter​
face is
going to be​
have, and as​
signs them to spe​
cific ports via the con​
cept of in​
ter​
face se​
lec​
tor. In turn, the in​
ter​
face pro​
file is even​
tu​
ally tied to a switch pro​
file to spec​
ify which
leaf switch the ref​
er​
enced ports should be con​
fig​
ured. As we con​
tinue the process of
defin​
ing the port pro​
files, you can ob​
serve how we have started at the bot​
tom of this
ob​
ject tree con​
fig​
ur​
ing the dif​
fer​
ent pro​
files. The pur​
poses for all these in​
di​
vid​
ual poli​
cies that tie to​
gether is to max​
i​
mize pol​
icy re-use.

Interface Profile links to Interface Selector and Interface Policy Group
The di​
a​
gram in the pre​
vi​
ous sec​
tion pro​
vides a vi​
sual de​
scrip​
tion of what can be ac​
com​
plished by group​
ing the poli​
cies that have been de​
fined under the in​
ter​
face pro​
file,
and then as​
signed to ports with in​
ter​
face se​
lec​
tors and the ac​
cess port pol​
icy groups.

100 Fabric Connectivity

Create Interface Profile

The in​
ter​
face pro​
file is com​
posed of two pri​
mary ob​
jects. The in​
ter​
face se​
lec​
tor and
the ac​
cess port pol​
icy group. The in​
ter​
face se​
lec​
tor de​
fines what in​
ter​
faces will apply
the ac​
cess port pol​
icy. The ports that share the same at​
trib​
utes can then be grouped
under the same in​
ter​
face pro​
file.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

Next, add the in​
ter​
face se​
lec​
tors that are as​
so​
ci​
ated to this in​
ter​
face pro​
file.
Create Interface Selector

1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

3
4

In the Work pane, choose Actions > Create Access Port Selector.
In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter the interface IDs.
d. Choose the interface policy group that should be associated to these ports.

5

Click Submit.

Create Interface Profile for Port Channel

If a server has two or more up​
links to a leaf switch, the links can be bun​
dled into a Port
Chan​
nel for re​
siliency and load dis​
tri​
bu
​t​
ion. In order to con​
fig​
ure this in ACI, cre​
ate an
in​
ter​
face pol​
icy group of type Port Chan​
nel to bun​
dle the in​
ter​
faces. Dif​
fer​
ent Port
Chan​
nels re​
quire dif​
fer​
ent pol​
icy groups.

Fabric Connectivity 101

Port Channel Policy Group
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

Next, cre​
ate an in​
ter​
face port se​
lec​
tor. Since you will be con​
fig​
ur​
ing a Port Chan​
nel,
the op​
er​
a​
tor will add all of the in​
ter​
faces re​
quired in the Port Chan​
nel in​
ter​
face. In this
ex​
am​
ple in​
ter​
faces Eth​
er​
net 1/1-2 will be con​
fig​
ured in one Port Chan​
nel and in​
ter​
faces Eth​
er​
net 1/3-4 will be con​
fig​
ured in an​
other Port Chan​
nel.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

3

In the Work pane, choose Actions > Create Access Port Selector.

102 Fabric Connectivity

4

In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter interface IDs for the first port channel.
d. Choose the interface policy group.

5

Click Submit.

6

Repeat this process for the second Port Channel (if you have another Port
Channel to add).

Create an Interface Profile for Virtual Port Channel

A vPC do​
main is al​
ways made up of two leaf switches, and a leaf switch can only be a
mem​
ber of one vPC do​
main. In ACI, that means that the de​
f​
i​
n​
i​
tion of the poli​
cies is sig​
nif​
i​
cant be​
tween the two switches. The same pol​
icy can be reused be​
tweeen the two
switches, and through the vPC do​
main the pair as​
so​
ci​
a​
tion can be de​
fined. vPC Switch
do​
main mem​
bers should be taken into con​
sid​
er​
a​
tion when con​
fig​
ur​
ing firmware main​
te​
nance groups. By keep​
ing this in mind, firmware up​
grades should never im​
pact both
vPC switch peers at the same time. More de​
tails on this can be found in the Up​
grad​
ing
and Down​
grad​
ing Firmware sec​
tion.
For this rea​
son, a switch pro​
file that would rep​
re​
sent two sep​
a​
rate switch IDs needs to
be cre​
ated. The re​
la​
tion​
ship of these switches to the two ports in the same pol​
icy
group is de​
fined through the in​
ter​
face pro​
file.

vPC Policy Group

Fabric Connectivity 103

The same process would have to be re​
peated for every grouped in​
ter​
face on each side
that will be a mem​
ber of the vPC.
1

On the menu bar, choose Fabric > Access Policies.

2

In the Navigation pane, choose Interface Policies > Profiles.

3

In the Work pane, choose Actions > Create Interface Profile.

4

In the Create Interface Profile dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.

5

Click Submit.

6

In the Navigation pane, choose Interface Policies > Profiles
> Name_of_Interface_Profile_Created.

7
8

In the Work pane, choose Actions > Create Access Port Selector.
In the Create Access Port Selector dialog box, perform the following actions:
a. Define a meaningful name for the profile.
b. Optionally, provide a description for the profile.
c. Enter interface IDs.
d. Select of the interface policy group to be used for the vPC port behavior.

9

Click Submit.

Create a vPC Domain for Virtual Port Channel

When con​
fig​
ur​
ing a vPC, there is one ad​
di​
tional step to be con​
fig​
ured once to put two
leaf switches into the same vPC do​
main.

Creating a vPC Domain
1

On the menu bar, choose Fabric > Access Policies.

104 Fabric Connectivity

2

In the Navigation pane, choose Switch Policies > VPC Domain > Virual Port
Channel default.

3

In the Work pane, choose Actions > Explicit VPC Protection Group.

4

In the Explicit VPC Protection Group dialog box, perform the following actions:
a. Define a meaningful name for the vPC domain.
b. Provide a unique ID to represent the vPC domain.
c. Choose the first switch you want to be part of the vPC domain.
d. Choose the second switch you want to be part of the vPC domain.

5

Click Submit.

Switch Profiles
A switch pro​
file groups all the in​
ter​
face pro​
files that de​
fine the be​
hav​
ior of its re​
spec​
tive switch ports. A switch pro​
file could be the de​
f​
in
​i​
t​
ion of a sin​
gle switch or it could
be the de​
f​
in
​i​
t​
ion of mul​
ti​
ple switches. As a best prac​
tice, there should be a switch pro​
file for each leaf switch, and an ad​
di​
tional switch pro​
file for each vPC do​
main pair of
leaf switches.
The in​
ter​
face pro​
files that you have cre​
ated can be as​
so​
ci​
ated to the switch through a
sin​
gle switch pro​
file or they can be as​
so​
ci​
ated through dif​
fer​
ent switch pro​
files. If you
have var​
io
​us racks that are iden​
ti​
cal in the way the in​
ter​
face ports are con​
fig​
ured, it
could be ben​
e​
fi​
cial to uti​
lize the same switch pro​
file. This would make it pos​
si​
ble to
mod​
ify the con​
fig​
ur​
a​
tion of many switches dur​
ing op​
er​
at​
ions with​
out hav​
ing to con​
fig​
ure each switch in​
di​
vid​
ua
​lly.

Reusability
The ca​
pa​
bil​
ity of pol​
icy reusabil​
ity is cru​
cial to re-em​
pha​
size from an op​
er​
at​
ional per​
spec​
tive. If a pro​
file has been de​
fined to con​
fig​
ure a port as 1GB speed for ex​
am​
ple, that
pro​
file can be reused for many in​
ter​
face pol​
icy groups. When look​
ing at whole switch
con​
fig​
ur​
a​
tions, the re-us​
abil​
ity of the pro​
file can be ex​
tended to sim​
plify data cen​
ter
op​
er​
at​
ions and en​
sure com​
pli​
ance. The fol​
low​
ing fig​
ure il​
lus​
trates the reusabil​
ity of
pro​
files across racks of switches.

If all these racks are con​ fig​ ured in the same fash​ ion (mean​ ing they are wired in the same way) the same poli​ cies could be reused by sim​ ply as​ sign​ ing the switches to the same switch pro​ file. It would then in​ herit the pro​ file tree and be con​ fig​ ured the exact same way as the other racks. each of the top of rack switches is based on the same switch pro​ file. If a pro​ file has been reused across many de​ vices. make sure to check where it is being used be​ fore you delete the pro​ file or pol​ icy. The fol​ low​ ing topol​ ogy will be con​ fig​ ured: .Fabric Connectivity 105 Policy re-use at scale In the pre​ vi​ ous di​ a​ gram. It is also im​ por​ tant to be aware of the im​ pli​ ca​ tion of delet​ ing pro​ files. Sample vPC Creation The fol​ low​ ing pro​ ce​ dure demon​ strates what a vPC bringup looks like and also API POST con​ fig​ u​ ra​ tion as​ s​ es​ ment of the vPC.

choose Actions > Create VLAN Pool. Note: For this example the pool will be from VLAN 100 to VLAN 199. choose Fabric > Access Policies. 3 In the Work pane. Click Static Allocation for the allocation mode. In the Create Physical Domain dialog box. REST :: /api/node/class/fvnsVlanInstP. choose Pools > VLAN. 2 In the Navigation pane. 2 In the Navigation pane. Optionally.106 Fabric Connectivity Sample Topology Create VLAN Pools 1 On the menu bar. perform the following actions: a. 4 In the Create VLAN Pool dialog box. provide a description for the pool. b. b. perform the following actions: a. c. Define a meaningful name for the pool. choose Fabric > Access Policies. . choose Physical and External Domains > Physical Domains. choose Actions > Create Physical Domain. Define a meaningful name for the domain. 3 4 In the Work pane. Choose the VLAN pool that you previously created.xml CLI :: moquery -c fvnsVlanInstP Create a Physical Domain 1 On the menu bar.

c. Choose the auto negotiation for the interface. d. 2 In the Navigation pane. Define a meaningful name for the AEP. 4 In the Create Link Level Policy dialog box. choose Actions > Create Link Level Policy. In the Create Attached Entity Profile dialog box. provide a description for the policy.Fabric Connectivity 107 REST :: /api/node/class/physDomP. c. Optionally.xml CLI :: moquery -c infraAttEntityP Interface Policies Create Link Level Policy 1 On the menu bar. b. 5 Click Next. 3 4 In the Work pane. Provide a unique ID to represent the AEP. Click + to add a domain that will be associated to the interfaces. . choose Fabric > Access Policies. 2 In the Navigation pane. perform the following actions: a. d. perform the following actions: a. Choose the physical domain that you previously created. Define a meaningful name for the pool. 6 Click Submit. choose Fabric > Access Policies. 3 In the Work pane. b. choose Actions > Create Attached Entity Profile. REST :: /api/node/class/infraAttEntityP. 5 Click Submit. choose Interface Policies > Policies > Link Level. choose Global Policies > Attachable Access Entity Profles.xml CLI :: moquery -c physDomP Create Access Entity Profile 1 On the menu bar. Choose the speed to match the interface requirement.

4 In the Create LACP Policy dialog box. perform the following actions: a.xml CLI :: moquery -c infraAccBndlGrp . d. b. In mode click on Active.xml CLI :: moquery -c lacpLagPol Create vPC Interface Policy Group 1 On the menu bar. 2 In the Navigation pane. Note: LACP is recommended for vPCs. Define a meaningful name for the policy. Optionally. choose Actions > Create vPC Interface Policy Group. 5 Click Submit. 3 In the Work pane. b. provide a description for the pool. Define a meaningful name for the policy group. Choose an AEP to associate the policy group to. However.xml CLI :: moquery -c fabricHIfPol Create LACP Policy 1 On the menu bar. perform the following actions: a. REST :: /api/node/class/infraAccBndlGrp. provide a description for the policy group. REST :: /api/node/class/lacpLagPol. c. 2 In the Navigation pane. choose Interface Policies > Policies > LACP. choose Actions > Create LACP Policy. 5 Click Submit. c. choose Fabric > Access Policies. 4 In the Create vPC Interface Policy Group dialog box. choose Interface Policies > Policy Groups.108 Fabric Connectivity REST :: /api/node/class/fabricHIfPol. Choose a Link Level Policy. Optionally. ensure LACP is configured on the device connected to the leaf switch. Choose an LACP Policy. e. choose Fabric > Access Policies. 3 In the Work pane.

choose Actions > Create Interface Profile. b. Optionally.Fabric Connectivity 109 Create an Interface Profile 1 On the menu bar. choose Actions > Create Access Port Selector. Optionally. 3 In the Work pane. 7 8 In the Work pane. b. 5 Click Submit. perform the following actions: a. choose Fabric > Access Policies. In the Create Access Port Selector dialog box. choose Switch Policies > Profiles. Define a meaningful name for the profile. c. provide a description for the profile. i. . provide a description. Optionally. choose Interface Policies > Profiles. ii.xml CLI :: moquery -c infraAccPortP Create a Switch Profile 1 On the menu bar. provide a description for the policy group. Name: 103-104 (example node numbers). Select an interface policy group. In switch selectors click on the + symbol. 4 In the Create Switch Profile dialog box. Select the proper interfaces. Define a meaningful name for the policy group. 9 Click Submit. Blocks: Select switch 103 and switch 104. choose Fabric > Access Policies. perform the following actions: a. 2 In the Navigation pane. choose Actions > Create Switch Profile. c. d. b. REST :: /api/node/class/infraAccPortP. 2 In the Navigation pane. Define a meaningful name for the profile. choose Interface Policies > Profiles > Profiles > ACIVPC-int-profile. 4 In the Create Interface Profile dialog box. 3 In the Work pane. 6 In the Navigation pane. perform the following actions: a.

6 Select the previously created interface selector. 7 Click Finish. choose Switch Policies > VPC Domain > Virual Port Channel default. choose Fabric > Inventory. a. 3 In the Work pane. 2 In the Navigation pane. perform the following actions: b. 4 Click Submit. . choose Actions > Explicit VPC Protection Group. In the Explicit VPC Protection Groups section. f. You can also val​ id ​ate the op​ er​ at​ ion of the vPC di​ rectly from the CLI of the switch it​ self. choose Fabric > Access Policies.xml CLI :: moquery -c fabricExplicitGEp Validate Operation of Configured vPC 1 On the menu bar.110 Fabric Connectivity 5 Click Next. REST :: /api/node/class/fabricExplicitGEp. 2 In the Navigation pane. the status should be displayed and you should see successful establishment of the vPC domain. e. click + to create a vPC protection group. there will be a table that shows the status of the vPC interface. Provide a unique ID for the vPC domain. REST :: /api/node/class/infraNodeP. c. choose POD 1 > Interfaces > vPC Interfaces. In the Explicit VPC Protection Group dialog box. 3 In the Work pane. Select the first switch ID that is part of this vPC pair: 103. d. Define a meaningful name for the vPC domain.xml CLI :: moquery -c infraNodeP Create vPC domain 1 On the menu bar. Select the second switch ID that is part of this vPC pair: 104. If configured correctly.

local vPC is down.-----success success ------------ . Leaf-3# show vpc Legend: (*) .-------------------------------------------------up - vPC status ---------------------------------------------------------------------id Port Status Consistency Reason Active vlans 1 Po1 up - -- ---- -----. forwarding via vPC peer-link vPC domain id : 100 vPC keep-alive status : Disabled Peer status : peer adjacency formed ok Configuration consistency status Per-vlan consistency status Type-2 consistency status : success : success : success vPC role : primary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------id -1 Port ---- Status Active vlans -----.----------.Fabric Connectivity 111 If you con​ nect to the con​ sole or the out of band man​ age​ ment in​ ter​ face of the leaf node you should be able to see the sta​ tus with the com​ mand show vpc.

xml <polUni> <infraInfra> <!-.Switch Selector --> <infraNodeP name="switchProfileforVPC_201"> <infraLeafS name="switchProfileforVPC_201" type="range"> <infraNodeBlk name="nodeBlk" from_="201" to_="201"/> </infraLeafS> <infraRsAccPortP tDn="uni/infra/accportprof-intProfileforVPC_201"/> </infraNodeP> <infraNodeP name="switchProfileforVPC_202"> <infraLeafS name="switchProfileforVPC_202" type="range"> <infraNodeBlk name="nodeBlk" from_="202" to_="202"/> </infraLeafS> <infraRsAccPortP tDn="uni/infra/accportprof-intProfileforVPC_202"/> </infraNodeP> <!-.xml <polUni> <fvTenant descr="" dn="uni/tn-Cisco" name="Cisco" ownerKey="" ownerTag=""> <fvAp descr="" name="CCO" ownerKey="" ownerTag="" prio="unspecified"> .112 Fabric Connectivity The fol​ low​ ing REST API call can be used to build vPCs and at​ tach vPCs to sta​ tic port bind​ ings. URL: https://{{apic-ip}}/api/policymgr/mo/.Interface Profile --> <infraAccPortP name="intProfileforVPC_201"> <infraHPortS name="vpc201-202" type="range"> <infraPortBlk name="vpcPort1-15" fromCard="1" toCard="1" fromPort="15" toPort="15"/> <infraRsAccBaseGrp tDn="uni/infra/funcprof/accbundle-intPolicyGroupforVPC"/> </infraHPortS> </infraAccPortP> <infraAccPortP name="intProfileforVPC_202"> <infraHPortS name="vpc201-202" type="range"> <infraPortBlk name="vpcPort1-1" fromCard="1" toCard="1" fromPort="1" toPort="1"/> <infraRsAccBaseGrp tDn="uni/infra/funcprof/accbundle-intPolicyGroupforVPC"/> </infraHPortS> </infraAccPortP> <!-.Interface Policy Group --> <infraFuncP> <infraAccBndlGrp name="intPolicyGroupforVPC" lagT="node"> <infraRsAttEntP tDn="uni/infra/attentp-AttEntityProfileforCisco"/> <infraRsCdpIfPol tnCdpIfPolName="CDP_ON" /> <infraRsLacpPol tnLacpLagPolName="LACP_ACTIVE" /> <infraRsHIfPol tnFabricHIfPolName="10GigAuto" /> </infraAccBndlGrp> </infraFuncP> </infraInfra> </polUni> https://{{hostName}}/api/node/mo/uni.

Fabric Connectivity 113 <fvAEPg descr="" matchT="AtleastOne" name="Web" prio="unspecified"> <fvRsPathAtt encap="vlan-1201" instrImedcy="immediate" mode="native" tDn="topology/pod-1/protpaths-201-202/pathep-[vpc201-202]” /> </fvAEPg> <fvAEPg descr="" matchT="AtleastOne" name="App" prio="unspecified"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/protpaths-201-202/pathep-[vpc201-202]” /> </fvAEPg> </fvAp> </fvTenant> </polUni> .

.

Cisco Dis​ cov​ ery Pro​ to​ col (CDP) must be en​ abled on any ACI fab​ ric in​ ter​ faces that are con​ nect​ ing to UCS Fab​ ric In​ ter​ con​ nects. If the work​ load is a bare metal server. the type of Layer 2 con​ nec​ tion needed on the Fab​ ric In​ ter​ con​ nect fac​ ing ports must be de​ ter​ mined first. When con​ nect​ ing UCS to the ACI fab​ ric. Cisco UCS B-Series Servers If UCS B-se​ ries Fab​ ric In​ ter​ con​ nects are being con​ nected to your ACI fab​ ric and Cisco UCS. These poli​ cies are all gov​ erned by in​ ter​ face pol​ icy groups. For more in​ for​ ma​ tion on the process needed to con​ fig​ ure links to UCS as ei​ ther a vPC or a tra​ di​ tional port chan​ nel. Standalone Rack Mount Servers or Non-Cisco Servers Any non-UCS server ar​ chi​ tec​ ture can also be con​ nected di​ rectly to the ACI fab​ ric or to a Cisco Nexus 2000 Fab​ ric Ex​ ten​ der (FEX). The fab​ ric con​ nec​ tiv​ ity re​ quire​ ments that are dic​ tated by the server in​ fra​ struc​ ture must be care​ fully con​ sid​ ered. You can pre-pro​ vi​ sion these poli​ cies as part of the ini​ tial man​ age​ ment tasks.Fabric Connectivity 115 Server Connectivity Server con​ nec​ tiv​ ity is nec​ es​ sary for all ap​ pli​ ca​ tion work​ loads to func​ tion prop​ erly on the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric. A best prac​ tice is to lever​ age a vir​ tual pri​ vate cloud (vPC) to con​ nect the UCS en​ vi​ ron​ ment so as to cre​ ate a multichas​ sis ether​ chan​ nel. the kind of traf​ fic ex​ pected out of the server links needs to be de​ ter​ mined. as well as some third party servers that all need to be con​ nected to the ACI fab​ ric. When being con​ nected to the ACI fab​ ric. ACME Inc. see the Adding New De​ vices to the Fab​ ric sec​ tion. traf​ fic can be clas​ si​ fied on a per port basis and as​ so​ ci​ ated AEPs and EPGs can be mapped ap​ pro​ pri​ ately to match the en​ cap​ su​ lated traf​ fic. such as Cisco UCS B and C se​ ries. In this sce​ nario. in​ di​ vid​ ual link and fab​ ric switch fail​ ures are mit​ i-​ gated to main​ tain a higher ex​ pected up time. the Link Level Dis​ cov​ ery Pro​ to​ col (LLDP) must be dis​ abled. fab​ ric ac​ cess poli​ cies must be pro​ vi​ soned to match these re​ quire​ ments. See the Con​ fig​ ur​ ing Man​ age​ ment Pro​ to​ cols chap​ ter for more in​ for​ ma​ tion. has sev​ eral dif​ fer​ ent mod​ els of servers in their data cen​ ters. If a sup​ ported hy​ - . In the case of Cisco Uni​ fied Com​ put​ ing Sys​ tem (UCS).

a Vir​ tual Ma​ chine Man​ ager (VMM) do​ main must be prop​ erly con​ fig​ ured. all host-fac​ ing ports are treated the same way as if they were di​ rectly at​ tached to the ACI fab​ ric. are still true. and con​ nec​ tiv​ ity to any other net​ work de​ vice will not func​ tion prop​ erly. . Re​ stric​ tions that are pre​ sent in NX-OS mode such that non-host-fac​ ing ports are not sup​ ported. When uti​ liz​ ing a FEX. Ports must only be con​ nected to hosts. The key is to map the ex​ pected traf​ fic clas​ si​ fi​ ca​ tion to the ports that are con​ nected to the server in​ fra​ struc​ ture. Uti​ liz​ ing a FEX is an al​ ter​ na​ tive way to con​ nect host de​ vices into the ACI fab​ ric. and then as​ so​ ci​ ated with the cor​ re​ spond​ ing ports on the fab​ ric as a hy​ per​ vi​ sor that is fac​ ing through EPG and AEP map​ ping.116 Fabric Connectivity per​ vi​ sor is to be used.

• Credentials represent the authentication credentials to communicate with virtual machine controllers.Fabric Connectivity 117 Virtual Machine Networking Understanding VM Networking in ACI One of the most com​ mon uses of the Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) will be to help man​ age and de​ ploy ap​ pli​ ca​ tions in vir​ tual en​ vi​ ron​ ments. This is the pool type that is required for VMM integration. the user defines the interface and the encapsulation. For static EPG deployment. The Application Policy Infrastructure Controller (APIC) communicates with the VMM to publish network policies that are applied to virtual workloads. . Static pools . and multicast addresses. A VMware vCenter domain can associate only to a dynamic pool. The ACI pro​ vides the abil​ ity to man​ age both vir​ tual and phys​ ic ​al end​ points with the same set of poli​ cies. The encapsulation must be within the range of a pool that is associated with a domain with which the EPG is associated. VMware vShield. A pool is a shared resource and can be consumed by multiple domains.The EPG has a relation to the domain. The pool contains a range of encapsulated VLANs and VXLANs. Multiple controllers can use the same credentials. and Microsoft Systems Center Virtual Machine Manager (SCVMM). and the domain has a relation to the pool. A virtual machine controller administrator provides an APIC administrator with a virtual machine controller authentication credential. A leaf switch does not support overlapping VLAN pools.Managed internally by the APIC to allocate VLANs for endpoint groups (EPGs). such as VMM and Layer 4 to Layer 7 services. such as VMware vCenter. The fol​ low​ ing list de​ scribes some vir​ tual ma​ chine man​ ager (VMM) sys​ tem terms: • A virtual machine controller is an external VMM entity. Different overlapping VLAN pools must not be associated with the same attachable entity profile (AEP). This chap​ ter will look at var​ io ​us op​ er​ at​ ional tasks that will be per​ formed through​ out the daily op​ er​ at​ ions. such as VLAN and VXLAN IDs. multiple controllers of the same type can use the same credential. • The two types of VLAN-based pools are as follows: Dynamic pools . • A pool represents a range of traffic encapsulation identifiers.

Both pro​ vide sim​ i​ lar basic vir​ tual net​ work​ ing func​ tion​ al​ ity. the VLAN range must also be cre​ ated on any in​ ter​ me​ di​ ate de​ vices. how​ ever. the AVS pro​ vides . lever​ ag​ ing the VMware vSphere Dis​ trib​ uted Vir​ tual Switch (DVS) or the Cisco Ap​ pli​ ca​ tion Vir​ tual Switch (AVS). see the fol​ low​ ing doc​ u​ ments: • Cisco APIC Getting Started Guide VMware Integration When in​ te​ grat​ ing ACI into your VMware in​ fra​ struc​ ture you have two op​ tions for de​ ploy​ ing net​ work​ ing. such as tra​ di​ tional switches or blade switches. This in​ cludes cre​ at​ ing the VLANs on Uni​ fied Com​ put​ ing Sys​ tem (UCS). ACI VM Integration Workflow ACI VM Integration Workflow For de​ tailed in​ for​ ma​ tion on how to de​ ploy the VMware vSphere Dis​ trib​ uted Switch with the Cisco APIC.118 Fabric Connectivity When cre​ at​ ing dy​ namic VLAN pools for VMM in​ te​ gra​ tion. VMware do​ mains can be de​ ployed.

For or​ ga​ ni​ za​ tions in​ ter​ ested in using the stan​ dard DVS pro​ vided by VMware.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ getting-started/​ video/​ cisco_​ apic_​ create_​ vcenter_​ domain_​ profile_​ using_​ gui. has cho​ sen to lever​ age the ad​ di​ tional fea​ tures pro​ vided by AVS.Fabric Connectivity 119 ad​ di​ tional ca​ pa​ bil​ i​ ties. please refer to the fol​ low​ ing doc​ u​ ments: http://​ www. such as VXLAN and mi​ croseg​ men​ ta​ tion sup​ port. ACME Inc. VMM Policy Model Interaction .​ html VMM Policy Model Interaction Shown below are some of the var​ i​ ous ACI poli​ cies which are in​ volved with set​ ting up VM In​ te​ gra​ tion. This serves as a ref​ er​ ence for the ways the var​ i​ ous poli​ cies are re​ lated to each other.​ cisco.

5 In the Add VM Domain Association dialog box. There is no communication delay or traffic loss by keeping the default selections. choose Actions > Add VM Domain Association. choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > Application_EPG_Name > Domains (VMs and Bare-Metals).120 Fabric Connectivity Publishing EPGs to a VMM Domain This sec​ tion will de​ tail how to pub​ lish an ex​ ist​ ing end​ point group (EPG) to a Vir​ tual Ma​ chine Man​ ager (VMM) do​ main. Cisco recommends keeping the default option of On Demand. 3 Click on the Network Adapter. 4 In the Work pane. choose Tenants > ALL TENANTS 2 In the Work pane. This provides the best resource usage in the fabric by only deploying policies to Leaf nodes when endpoints assigned to this EPG are connected. a. 1 On the menu bar. right click on your Virtual Machine and select "Edit Settings". It should display in the format of [TENANT|APPLICATION_PROFILE|EPG|VMM_DOMAIN_PROFILE] . see the Ten​ ants sec​ tion. For an EPG to be pushed to a VMM do​ main. and in the Network Connection dropdown box select the Port Group which corresponds to your EPG. a do​ main bind​ ing within the ten​ ant EPG must be cre​ ated. 6 Click Submit. For the Deployment & Resolution Immediacy. Note: The EPG will now be available as a Port Group to your VMM. 3 In the Navigation pane. 2 From the Host and Clusters view. choose the Tenant_Name. choose the VMM Domain Profile that you previously created. For de​ tails on how to cre​ ate EPGs. Connecting VMs to the EPG Port Groups on vCenter 1 Connect to your vCenter using the VI Client.

4 In the Work pane. you should ver​ ify the APIC has learned your vir​ tual end point. it means one of the fol​ low​ ing: • The VM is running on a host which is not attached to the Distributed Switch • There may be a communication between your APIC and vCenter either through managed by the APIC. choose Tenants > ALL TENANTS. and one for the Class Name of 'fvCEp' (Fab​ ric Vec​ tor Client End​ point) moquery -c fvCEp --dn uni/tn-<TENANT_NAME>/ap-<APP_PROFILE_NAME>/epg-<EPG_NAME> You can de​ ter​ mine the DN of your EPG by right click​ ing on the EPG in the GUI. Verifying Virtual Endpoint Learning Once the VMs are con​ nected to the ap​ pro​ pri​ ate port group/EPG. 3 In the Navigation pane. All endpoints either virtual or physical will be displayed. choose the Tenant_Name. choose the Operational tab. se​ lect​ ing "Save As" and look​ ing at the XML ob​ ject. the OOB or INB management network. Note: The current tab should display CLIENT ENDPOINTS. Verifying VM Endpoint Learning on the APIC from the CLI You can ver​ ify the same info as above from the CLI by using the 'mo​ query' (Man​ aged Ob​ ject Query) com​ mand and adding two fil​ ters. 2 In the Work pane.Fabric Connectivity 121 If you do not see your ACI EPG in the Net​ work Con​ nec​ tion list. 1 On the menu bar. One for the Dis​ tin​ guished Name (DN) name of your EPG. From here you should be able to find your Virtual Machine by filtering the "Learning Source" column for rows with values of "Learned VMM". From this file you will see the DN entry for the par​ tic​ ul​ ar EPG: . choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > Application_EPG_Name.

729+11:00" matchT="AtleastOne" lcOwn="local" dn="uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od" descr="" configSt="applied" configIssues="" childAction=""/></imdata> Next.229+11:00 rn : cep-00:50:56:BB:8C:6A status : uid : 0 uuid : VMware Integration Use Case A VMWare ad​ min​ is​ tra​ tor in ACME re​ quests the net​ work team to trunk a set of VLANs down to the ESX hosts to pro​ vide con​ nec​ tiv​ ity to their DVS switches. as well as pro​ vid​ ing un​ lim​ ited Layer 2 mo​ bil​ ity for all the VM hosts within the ACI fab​ ric.CEp name : 00:50:56:BB:8C:6A childAction : dn : uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od/cep-00:50:56:BB:8C:6A encap : vlan-211 id : 0 idepdn : ip : 10. the net​ work team de​ cides to lever​ age a new method​ ol​ ogy to be more agile and lever​ age the on-de​ mand pro​ vi​ sion​ ing of re​ sources where and when they are needed.122 Fabric Connectivity <imdata totalCount="1"><fvAEPg uid="15374" triggerSt="triggerable" status="" scope="2588672" prio="unspecified" pcTag="49159" name="epg-od" monPolDn="uni/tn-common/monepg-default" modTs="2015-02-06T06:46:24. Rather than trunk​ ing VLANs on a per server basis.vmm lcOwn : local mac : 00:50:56:BB:8C:6A mcastAddr : not-applicable modTs : 2015-02-06T06:48:52.10 lcC : learned.10. use this DN with the mo​ query to re​ turn the list of client En​ points for this EPG: admin@apic1:~> moquery -c fvCEp --dn uni/tn-mb-tennant1/ap-mb-app-pro/epg-epg-od Total Objects shown: 1 # fv.10. .

How​ ever. Once this is de​ cided. This al​ lows VMware ad​ mins to main​ tain con​ trol and move vir​ tual NICs into these port-groups on de​ mand. They de​ cide on an un​ used VLAN range of (600 . the APIC ad​ min​ is​ tra​ tor pro​ ceeds to con​ fig​ ure VMM in​ te​ gra​ tion in the APIC GUI by pro​ vid​ ing the vCen​ ter cre​ den​ tials to APIC. doing on-de​ mand re​ source al​ lo​ ca​ tion. As the VMware admin pro​ vi​ sions ESX hosts and se​ lects the ap​ pro​ pri​ ate port-groups for VMs. It is im​ por​ tant to note that ACME can still choose to de​ ploy tra​ di​ tional VLAN trunk​ ing down to VMware DVS switches by sta​ t​ i​ cally pro​ vi​ sion​ ing EPGs on a per-port basis. VMs are al​ lowed to move any​ where within the ACI fab​ ric with no re​ stric​ tions other than those im​ posed by vCen​ ter. . and still reap the ad​ van​ tages of the Layer 2-any​ where ACI fab​ ric. ACME chose VMM in​ te​ gra​ tion as the pre​ ferred de​ ploy​ ment model as it is the most ef​ fec​ tive method of break​ ing down or​ ga​ ni​ za​ tional chal​ lenges. APIC dy​ nam​ ic ​ally pro​ vi​ sions all EPGs and makes them avail​ able to the ESX hosts as a port-group. the APIC dy​ nam​ ic ​ally com​ mu​ ni​ cates with vCen​ ter to make EPGs avail​ able through port groups. Note: The APIC does not au​ to​ mat​ ic ​ally move VM​ NICs into the port-group. Dur​ ing a vMo​ tion event. the net​ work ad​ mins work with the VMware ad​ mins to de​ cide on a range of VLANs that will be pro​ vided dy​ nam​ ic ​ally by APIC to the ESX hosts that need them.Fabric Connectivity 123 To do so. The APIC also con​ fig​ ures VLAN IDs on the leaf-switches as needed. APIC is au​ to​ mat​ ic ​ally in​ formed of the VM move and then up​ dates the end​ point track​ ing table to allow seam​ less com​ mu​ ni​ ca​ tion. and get​ ting en​ hanced vis​ i​ bil​ ity and teleme​ try into both the vir​ tual and phys​ i​ cal en​ vi​ ron​ ments.800). This is their dy​ namic VLAN pool.

.

fol​ lowed by tem Com​ pat​ i​ bil​ ity List doc​ um ​ent to en​ sure your ver​ sion 5.2. configured. Each ver​ sion of AVS soft​ ware in​ cludes the VIB files for all sup​ ported vSphere ver​ sions. • VMware vCenter is installed. Just like any soft​ ware.1 and 5.1. This al​ lows ei​ ther de​ vice to be up​ graded in​ de​ pen​ dently.5 (vSphere 5. As of this pub​ li​ ca​ tion there are two VIBs to sup​ port vSphere ver​ sions 5. • One or more vSphere hosts are available for deployment to the AVS.1. • A DNS server policy has been configured to enable connection to a VMM using a hostname. Getting Started The AVS soft​ ware was de​ signed to op​ er​ ate in​ de​ pen​ dently of the APIC soft​ ware ver​ sion. Refer to the ACI Ecosys​ de​ sired ver​ sion of AVS is com​ pat​ ib ​le with the APIC and vSphere ver​ sions being run.0 is not sup​ ported). new ver​ sions of the AVS will be re​ leased to in​ clude new fea​ tures and im​ prove​ ments. These can be down​ loaded from CCO at the fol​ low​ ing lo​ ca​ tion: Down​ loads Home > Prod​ ucts > Switches > Vir​ tual Net​ work​ ing > Ap​ pli​ ca​ tion Vir​ tual Switch . • A dynamic VLAN pool has been created with enough VLAN IDs to accommodate one VLAN per EPG you plan on deploying to each VMM domain. The AVS pack​ age for ei​ ther ver​ sion will in​ clude vSphere In​ stal​ la​ tion Bun​ dles (VIBs). The ini​ tial AVS soft​ ware re​ leased was ver​ sion 4.Fabric Connectivity 125 Deploying the Application Virtual Switch Prerequisites • All switch nodes have been discovered by the fabric • INB or OOB management connectivity is configured. Al​ ways refer to the AVS re​ lease notes to con​ firm if any spe​ cial con​ sid​ er​ at​ ions may exist.2. and available.

2.1.2.1. Right click on the desired datastore and select Browse. This can be achieved in a va​ ri​ ety of ways. navigate to the Host > Configuration > Storage > Datastore_X. The easiest way to copy the VIB to the host is to leverage the VMware VI Client.2.0-3. this can eas​ ily be done man​ ua ​lly. all of which are dis​ cussed in Cisco Ap​ pli​ ca​ tion Vir​ tual stal​ la​ tion Guide.1 Bundle cross_cisco-vem-v172-5.0-3.vib AVS 5. For a few hosts.3.2.2.126 Fabric Connectivity The VIBs can be iden​ ti​ fied as fol​ lows: AVS 4. but for 10+ Switch In​ hosts it may be eas​ ier to lever​ age the Vir​ tual Switch Up​ date Man​ ager (VSUM) to help au​ to​ mate the in​ stal​ la​ tion process. 2 SSH into the vSphere host on which the AVS VIB is to be installed.1 Bundle cross_cisco-vem-v165-4.1.1.1.3.3.2.0-3.3.1.2.2.0-3.1.vib Install the AVS VIB Be​ fore set​ ting up the AVS con​ fig​ ur​ a​ tion on the APIC.vib cross_cisco-vem-v165-4.vib cross_cisco-vem-v172-5. re​ ferred to as the Vir​ tual Eth​ er​ net Mod​ ule (VEM).1. From here.1.VIB can be uploaded directly to the host's datastore.2. the AVS soft​ ware must be in​ stalled in vSphere.3.2.3.1.2.1.2.1. 3 Install or upgrade the VIB using the esxcli command: To in​ stall the AVS VIB: esxcli software vib install -v /<path>/<vibname> --maintenance-mode --no-sig-check . Manual Installation 1 Copy the VIB file to a host.

3.1.3. # vem status VEM modules are loaded Switch Name Num Ports Used Ports Configured Ports MTU Uplinks vSwitch0 3072 6 128 1500 VMNIC0 VEM Agent (vemdpa) is running DHCP Relay The first task is to cre​ ate a DHCP relay pol​ icy in the infra ten​ ant on the APIC. . Reboot Required: false VIBs Installed: Cisco_bootbank_cisco-vem-v172-5. This will allow the AVS to cre​ ate a vmk Vir​ tual Tun​ nel End​ point (VTEP) in​ ter​ face on each host.0 Releasebuild-1623387 4 Confirm the VEM is loaded and running.1.1 VSM Version: System Version: VMware ESXi 5. This VTEP in​ ter​ face will be used for the Open​ Flex con​ trol chan​ nel and/or the VXLAN tun​ nel source be​ tween the VTEP and ACI fab​ ric.1.5.3.3.2.2.2.0-3.3.1.Fabric Connectivity 127 To Up​ grade an ex​ ist​ ing AVS VIB: esxcli software vib update -v /<path>/<vibname> --maintenance-mode --no-sig-check A sam​ ple out​ put is shown below: # esxcli software vib install -v /vmfs/volumes/datastore1/cross_cisco-vem-v1725.1.1.3.0-3.1 VIBs Removed: VIBs Skipped: /vmfs/volumes/53cab6da-55209af3-0ef2-24e9b391de3e # vem version Running esx version -1623387 x86_64 VEM Version: 5.vib --maintenance-mode --no-sig-check Installation Result Message: Operation finished successfully.2.2.0-3.1.2.

choose Tenants > ALL TENANTS. Change the Scope to TENANT.​ 0. Provide a name for the Relay policy. 6 Click Submit. Create the Infra DHCP Relay Policy Cre​ ate a DHCP relay pol​ icy that will setup the DHCP server (the APIC in this case) and a label using the Infra Ten​ ant and de​ fault EPG. 4 5 In the Work pane. In the Create DHCP Relay Policy dialog box. Leave other fields blank and click OK. 4 5 In the Work pane. The way APIC would do it is through over the infra vlan. such as "avs-dhcp-relay-pol". 6 Click Submit. choose Actions > Create DHCP Relay Policy. choose the infra. perform the following actions: a. choose infra > Networking > Protocol Policies > DHCP > Relay Policies. choose Actions > Create DHCP Relay Policy. . 2 In the Work pane.​ 0/​ 16).128 Fabric Connectivity If the de​ fault TEP Ad​ dress pool was used dur​ ing ini​ tial setup (10. choose Tenants > ALL TENANTS. 3 In the Navigation pane. STEPS 1 On the menu bar. b. In the Create DHCP Relay Policy dialog box. For this rea​ son it is crit​ ic ​al to ex​ tend the Infra VLAN from a leaf through to each vSphere host that will host the AVS.​ 0. choose infra > Networking > Bridge Domains > default > DHCP Relay Labels. perform the following actions: a. b. choose the infra. 2 In the Work pane. From the Name drop-down list. choose the DHCP Relay Policy that you created previously. this will allow the AVS VTEP in​ ter​ faces to pull an ad​ dress from this pool. Create the DHCP Relay Label 1 On the menu bar. The APIC IP ad​ dresses au​ to​ mat​ ic ​ally func​ tion as the DHCP providers and DO NOT need to be ex​ plic​ itly added. 3 In the Navigation pane.

3 In the Work pane. b. perform the following actions: a. Name: Provide any name to identify the AEP. Re​ gard​ less of using an ex​ ist​ ing AEP or cre​ at​ ing a new one. choose Fabric > Access Policies. This can be com​ pared to a "switch​ port trunk al​ lowed VLAN xxx" com​ mand in tra​ di​ tional net​ work​ ing. 4 In the Create Attachable Access Entity Profile dialog box. Re​ fer​ ring back to the "VMM Pol​ icy Model In​ ter​ ac​ tion" di​ ag ​ram from the "VM Net​ work​ ing Overview" chap​ ter. and CDP. such as "AVS-AEP". This will be covered later in the Publishing EPGs to VMM Domains chapter. ii. choose Global Policies > Attachable Access Entity Profile. . Domains (VMs or Baremetal): Leave blank. LLDP. From the next page of the wizard. This is to en​ sure that the traf​ fic of in​ ter​ est (DHCP re​ quest/offer can flow through the in​ fra​ struc​ ture VLAN to the AVS). i. such as speed/ne​ go​ ti​ at​ ion. 2 In the Navigation pane. the AEP is what ties the VMM do​ main to the phys​ ic ​al in​ ter​ faces where the vSphere hosts are con​ nected. Create a New AEP 1 On the menu bar. the En​ able In​ fra​ struc​ ture VLAN check box must be checked for the AEP pol​ icy. The AEP can be cre​ ated on-the-fly dur​ ing the cre​ ation of the VMM do​ main it​ self. See the "Adding New De​ vices to the Fab​ ric" chap​ ter for more de​ tail on cre​ at​ ing the in​ ter​ face pol​ icy group and in​ ter​ face pro​ files. Note: In​ ter​ face Pol​ icy Group cre​ ation is cov​ ered else​ where in this guide. Enable Infrastructure VLAN: Check this box.Fabric Connectivity 129 Attachable Access Entity Profile (AEP) and AVS An im​ por​ tant com​ po​ nent used by the AVS is the At​ tach​ able En​ tity Pro​ file (AEP). select the Interface Policy Group your AEP will be associated to. The AEP de​ fines which VLANs will be per​ mit​ ted on a host fac​ ing in​ ter​ face. iii. but this guide will de​ tail cre​ at​ ing the AEP sep​ ar​ ately first. This procedure assumes your Interface Policy Group has already been created. choose Actions > Create Attachable Access Entity Profile. Click the All Interfaces radio button for the desired Interface Policy Group. Es​ sen​ tially the In​ ter​ face Pol​ icy Group is a col​ lec​ tion of In​ ter​ face poli​ cies which de​ fine In​ ter​ faces Se​ lec​ tors and prop​ er​ ties. Fill in the AEP wizard information then click Next.

AVS Switching Modes The AVS can op​ er​ ate in the fol​ low​ ing switch​ ing modes: • Local Switching: Supports VXLAN encapsulation or VLAN encapsulation. This switching mode allows Inter-EPG traffic to be switched locally on the AVS. to also be ap​ plied to vir​ tual end​ points. A new VMM Do​ main will be cre​ ated from scratch to sup​ port AVS de​ ploy​ ment. This switching mode sends all traffic (Inter-EPG included) to the Leaf switch. In the Navigation pane. VMM Domains for vCenter A Vir​ tual Ma​ chine Man​ ager (VMM) do​ main de​ fines a vir​ tual in​ fra​ struc​ ture that will be in​ te​ grated into ACI. You can​ not change from one to the other.130 Fabric Connectivity Modify an Exisiting AEP 1 On the menu bar. check the Enable Infrastructure VLAN check box. vCen​ ter VMM Do​ mains are cre​ ated using ei​ ther the VMware DVS or Cisco AVS. choose Global Policies > Attachable Access Entity Profile. Note: As men​ tioned early in this chap​ ter. the In​ fra​ struc​ ture VLAN is re​ quired for AVS com​ mu​ ni​ ca​ tion to the fab​ ric using the Open​ Flex con​ trol chan​ nel. In the Work pane. This al​ lows the same poli​ cies ap​ plied to phys​ ic ​al end​ points. choose the existing AEP b. 2 In the Navigation pane. • No Local Switch only supports VLAN encapsulation. choose Fabric > Access Policies. . a.

All traf​ fic be​ tween the AVS up​ links and ACI fab​ ric will be en​ cap​ su​ lated by VXLAN and trans​ fered using the in​ fra​ struc​ ture VLAN. 2 In the Navigation pane. Create the VMM Domain for AVS Now that the DHCP server pol​ icy has been cre​ ated and AEP cre​ ated/mod​ i​ fied. This in​ cludes cre​ at​ ing the VLANs on in​ ter​ me​ di​ ate de​ vices such as UCS and the vNICs for any AVS vSphere hosts. 1 On the menu bar. choose the Policies tab. choose VM NETWORKING. If VLAN en​ cap​ su​ la​ tion is pre​ ferred.Fabric Connectivity 131 ​ AVS Switching Modes: Non-Local and Local switching mode The de​ ci​ sion be​ tween using VLAN or VXLAN en​ cap​ su​ la​ tion will man​ date dif​ fer​ ent VLAN ex​ ten​ sion re​ quire​ ments out​ side of the fab​ ric. 3 In the Work pane. you will need to en​ sure every VLAN in the VM Do​ main VLAN pool has been ex​ tended be​ tween the fab​ ric and AVS host. . choose Actions > Create VCenter Domain. When using VXLAN en​ cap​ su​ la​ tion. you can cre​ ate the VMM do​ main for the AVS. only the infra VLAN is re​ quired to be ex​ tended to the AVS hosts.

Switching Preference: <Choose Local or No Local Switching> • For No Local Switching mode: Multicast Address: <Assign a multicast address to represent your AVS> Multicast Address Pool: <Create a unique Multicast Address Pool large enough to include each AVS vSphere host. Click Submit. Virtual Switch: Cisco AVS c. b.132 Fabric Connectivity 4 In the Create vCenter Domain dialog box. perform the following actions: a. Name: This value will be used as the AVS "Switchname" displayed in vCenter.> d.> • For Local Switching mode: Encapsulation: <Choose VLAN or VXLAN based on preference> For VLAN Encapsulation VLAN Pool: <Choose/Create a VLAN pool> For VXLAN Encapsulation Multicast Address: <Assign a multicast address to represent your AVS> Multicast Address Pool: <Create a unique Multicast Address Pool large enough to include each AVS vSphere host. vCenter: Add the vCenter details. . vCenter Credentials: Create a credential set with administrator/root acces to vCenter f. Attachable Access Entity Profile: <Choose the AEP previously created/modified> e. 5 • Name: Friendly name for this vCenter • Hostname/IP Address: <DNS or IP Address of vCenter> • DVS Version: vCenter Default • Datacenter: <Enter the exact Datacenter name displayed in vCenter> • Management EPG: <Set to oob or inb Management EPG> • Associated Credentials: <Choose the Credential set previously created> • Click OK to complete the creation of the vCenter.

. choose any vSphere hosts to add to the AVS.. 2 Right click on the newly created AVS switch (not the folder) and choose Add Host. AVS up​ links can not be shared with any other ex​ ist​ ing vSwitch or vDS. navigate to HOME > INVENTORY > NETWORKING and confirm a new Distributed Virtual Switch folder has been created. Add vSphere Hosts to the AVS After the AVS has been cre​ ated in vCen​ ter. Note: For blade switch sys​ tems such as UCS. 3 In the Add Host dialog box.Fabric Connectivity 133 Verify AVS Deployment on vCenter 1 In the vCenter client. 4 Click Next until the wizard completes. this re​ quires the vNIC within the ser​ vice pro​ file to have all rel​ e​ vant VLANs ac​ tive on the vNICs. 2 Expand this folder to find your AVS. and a few default Port Groups including "uplink" and "vtep". and select an unassigned VMNIC uplink.. the VMNIC in​ ter​ face used must have all nec​ es​ sary VLANs al​ lowed on the in​ ter​ face. In UCS terms. skipping the migration of any virtual adapters or virtual machine networking at this time. To do this you will need at least one un​ used phys​ ic ​al in​ ter​ face (VMNIC) to act as the up​ link on each host. . you then need to at​ tach hosts to it. navigate to HOME > INVENTORY > NETWORKING. 1 From the vCenter client.

95 00:50:56:65:3d:b3 1500 65535 . ~ # esxcfg-VMKNIC -l Interface Port Group/DVPort Netmask Broadcast vmk0 Management Network 255.54 00:50:56:61:1c:92 1500 IPv4 65535 10.255 vmotion 192.99.176. we see that the VMkernel port has received the IP address from the APIC. The APIC uses the same 10.0.176.0.0.255.255. you should see a new vmk interface created on your distributed switch within vCenter and assigned to the 'vtep' port group.0 vmk2 255.255 9 10.16.255.16.54 00:25:b5:00:00:29 1500 IPv4 192.255.255.0.0 vmk1 255. As can be seen from the screenshot below.255. This vmk is your Virtual Tunnel Endpoint (vtep) interface and should have pulled a DHCP address from the APIC from the TEP subnet.168.134 Fabric Connectivity Adding Virtual host physical NICs to participate in Virtual Switch 5 Assuming the ACI fabric can reach the vSphere host over the infra VLAN.255 IP Family IP Address MAC Address IPv4 MTU TSO MSS Enabled Type 65535 true STATIC true STATIC true DHCP 172.16.0.99. This implies that we are ready for Opflex communication in between the AVS and the APIC.168.0 172.0/16 pool that is created during the APIC setup to provision the IP address.

0. there are three dif​ fer​ ent Vir​ tual Ma​ chines con​ nected to dif​ fer​ ent EPGs.1.0. Ver​ ify that the ‘sta​ tus: 12 (Ac​ tive)’ is seen as well as the switch​ ing mode.30 Port: 8000 Infra vlan: 4093 FTEP IP: 10.1 Ver​ ify on the AVS host . VXLAN Load Balancing VXLAN load bal​ anc​ ing is au​ to​ mat​ ic ​ally en​ abled as soon as more than one VMKNIC is con​ nected to the Cisco AVS. you should have an equal num​ ber of VMKNICs and up​ links.32 Switching Mode: LS NS GIPO: 225.127.there should be one mul​ ti​ cast group per de​ ployed EPG on the host. A max​ im ​um of eight VMKNICs can be .0. ~ # vemcmd show epp multicast Number of Group Additions 3 Number of Group Deletions 0 Multicast Address EPP Ref Count 225.76 1 225. issue the ‘vem​ cmd show open​ flex’ com​ mand.58 1 225.0.0. Also ver​ ify that the GIPO ad​ dress is the same as the mul​ ti​ cast ad​ dress that was used while cre​ at​ ing the VMM do​ main. ~ # vemcmd show openflex Status: 12 (Active) Dvs name: comp/prov-VMware/ctrlr-[AVS-TEST]-VC/sw-dvs-87 Remote IP: 10.0.0. In the out​ put below.92 1 These mul​ ti​ cast ad​ dresses will cor​ re​ spond to the EPG de​ tails found in the APIC GUI antX > Ap​ pli​ ca​ tion Pro​ files > Ap​ pli​ ca​ tion​ Pro​ fileX > End Point under Ten​ ants > Ten​ Groups > End​ Point​ GroupX and click the Op​ er​ a​ tional Tab > Client End Points. Each VMKNIC can use only one up​ link port.Fabric Connectivity 135 Verify AVS on ESX On the ESX com​ mand line.0.0.0.

You need to have as many VMKNICs as the host has VM​ NICs. In VXLAN load bal​ anc​ ing. the VMKNICs pro​ vide a unique MAC ad​ dress to pack​ ets of data that can then be di​ rected to use cer​ tain phys​ i​ cal NICs (VM​ NICs). Each vmk in​ ter​ face cre​ ated for the AVS should be at​ tached to the vtep port group and con​ fig​ ured for DHCP. if the host has five VM​ NICs. Distributed Switch configured with APIC port groups .136 Fabric Connectivity at​ tached to a Cisco AVS switch. Each of the VMKNICs that you cre​ ate has its own soft​ ware-based MAC ad​ dress. In VMware vSphere Client. you will need to cre​ ate an ad​ di​ tional vir​ tual adapter (VMK) for each AVS up​ link. Note that each VMK in​ ter​ face added is as​ signed a unique DHCP ad​ dress from the fab​ ric TEP pool. up to a max​ i​ mum of eight. For ex​ am​ ple. In the screen​ shot below you can see the four VMNIC up​ links to the AVS and the four vmk vir​ tual in​ ter​ faces to pro​ vide equal load bal​ anc​ ing traf​ fic across all up​ links. you need to add four VMKNICs to en​ able VXLAN load-bal​ anc​ ing. the Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) al​ ready cre​ ated one VMKNIC when the host was added to the dis​ trib​ uted vir​ tual switch (DVS).

we have to con​ fig​ ure an IGMP querier pol​ icy on the APIC. d. This will cause flood​ ing of the mul​ ti​ cast traf​ fic to all end​ points. Click the Fast Leave check box. choose Tenants > ALL TENANTS. Provide a name for the policy. 3 In the Navigation pane. then IGMP pol​ icy is not needed since the blade switch will flood the mul​ ti​ cast traf​ fic on all the rel​ e​ vant ports. Create an IGMP Snooping Policy for AVS 1 On the menu bar. such as "IGMP_Infra". By de​ fault. a. Due to this. choose Create IGMP Snooping Policy. c. USC-B FI has IGMP snoop​ ing en​ abled. The al​ ter​ nate method would be to cre​ ate an IGMP pol​ icy on UCS to dis​ able IGMP snoop​ ing. Click the Enable Querier check box. If we dis​ able IGMP snoop​ ing on UCS or other in​ ter​ me​ di​ ate blade switches. In the IGMP Snoop Policy drop-down list. . 4 Click Submit. b. 2 In the Work pane. choose infr > Networking > Bridge Domain > default. The IGMP snoop​ ing pol​ icy needs to be en​ abled on the infra ten​ ant.Fabric Connectivity 137 IGMP Snooping Policy for AVS Cisco UCS-B Series Considerations with AVS Deployments This sec​ tion of the ar​ ti​ cle will focus the nec​ es​ sary steps to en​ able AVS through the Cisco UCS-B se​ ries. Note: Verify if IGMP snooping is working properly on the vSphere host CLI using 'vemcmd show epp multicast' as shown above. choose infra.

.

4 In the Work pane. choose Tenants > ALL TENANTS. ACME Inc. as the traf​ fic clas​ si​ fi​ ca​ tion will be based on the en​ cap​ su​ la​ tion re​ ceived on a port. and as such. choose Action > Deploy Static EPG on PC. the con​ fig​ ured pol​ icy for this EPG will be en​ forced. perform the following actions: a.Fabric Connectivity 139 External Connectivity Extending ACI to External Layer 2 As men​ tioned in the in​ tro​ duc​ tion of this book. ACME Inc. such as a VLAN resource and policy content-addressable memory . Click one of the Deployment Immediacy radio buttons. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). In the Path field. vPC or Interface. all of the traf​ fic re​ ceived on this leaf port with the con​ fig​ ured VLAN ID will be mapped to the EPG. To as​ sign a Layer 2 con​ nec​ tion sta​ ti​ c ​ally on an ACI leaf port to an EPG: 1 On the menu bar. vPC or Interface dialog box. There​ fore. 5 In the Deploy Static EPG on PC. Deployment immediacy determines when the actual configuration will be applied on the leaf switch hardware. This is nec​ es​ sary for ex​ tend​ ing Layer 2 con​ nec​ tiv​ ity to a Data Cen​ ter In​ ter​ con​ nect (DCI) plat​ form. or sim​ ply to ex​ tend a Layer 2 do​ main out​ side of the fab​ ric to con​ nect in an ex​ ist​ ing Layer 2 net​ work in a non-ACI fab​ ric. 3 In the Navigation pane. 2 In the Work pane. After doing so. The immediacy also determines when the hardware resource. b. choose the Tenant_Name. is a multi​ na​ tional com​ pany with mul​ ti​ ple data cen​ ters. specify a port as well as a VLAN ID. Extending Endpoint Groups Outside the ACI Fabric The sim​ plest way to ex​ tend an end​ point group (EPG) out​ side of the ACI fab​ ric is to sta​ t-​ i​ cally as​ sign a leaf port and VLAN ID to an ex​ ist​ ing end​ point group. The end​ points do not need to be di​ rectly con​ nected to the ACI leaf. to fur​ ther ex​ tend con​ nec​ tiv​ ity to a re​ mote data cen​ ter. must con​ fig​ ure some Layer 2 con​ nec​ tiv​ ity.

Choose this mode if the traffic from the host is tagged with a VLAN ID. e. This option can be used to connect a leaf port to a bare metal server whose network interface cards (NICs) typically generate untagged traffic. This is the default deployment mode. ii. Click one of the Mode radio buttons. but (unlike the untagged mode) 802. .The 802. The option Immediate means that the EPG configuration and its related policy configuration will be programmed in the hardware right away. Associate the physical domain to the EPG in question.140 Fabric Connectivity (CAM) to support the related contract for this EPG.The untagged option means that the leaf expects untagged traffic without a VLAN ID. Any traffic received on links with this mode classification will have the following conditions applied to them: d. 802.1P . The mode option specifies whether the ACI leaf expects incoming traffic to be tagged with a VLAN ID or not. The option On Demand instructs the leaf switch to program the EPG and its related policy in the hardware only when traffic matching this policy is received for this EPG.1P option refers to traffic tagged with 802. 802.The tagged option means that the leaf node expects incoming traffic to be tagged with the specified VLAN ID previously established. with this option you can only assign the interface to one EPG. i.1P mode is useful when its necessary to handle the traffic on one EPG as untagged to the interface (similar to the switchport trunk native vlan vlan_ID command). A port can have only one EPG statically bound to a port as untagged.1P headers. Similar to the switchport access vlan vlan_ID command. Untagged . Multiple EPGs can be statically bound to the same interface as long as the encap VLAN/VXLAN ID is unique. Tagged . iii. Create a physical domain and VLAN pool that are associated to this physical domain.1P will allow other 'tagged' EPGs to be statically bound to the same interface. c. will be consumed on the leaf switch.

Extending an ACI Bridge Domain Outside of the Fabric A Layer 2 out​ side con​ nec​ tion is as​ so​ ci​ ated with a bridge do​ main and is de​ signed to ex​ tend the whole bridge do​ main. perform the following actions: a. Clas​ sify any out​ side con​ nec​ tions or end​ points into this new ex​ ter​ nal EPG. This VLAN ID must be in the range of the VLAN pool that is used for the Layer 2 outside connection. . 3 In the Navigation pane. Associate the Layer 2 outside connection with the bridge domain and a VLAN ID. Dur​ ing this process. This Layer 2 outside connection will put this VLAN and the bridge domain of the ACI fabric under the same Layer 2 domain. choose Tenants > ALL TENANTS. i. With two sep​ ar​ ate EPGs. this method will also allow the end​ points to share the same sub​ net and de​ fault gate​ way. This new EPG will be part of the ex​ ist​ ing bridge do​ main.Fabric Connectivity 141 f. For the External Bridged Domain drop-down list. In the Create Bridged Outside dialog box. a Layer 2 out​ side con​ nec​ tion must be cre​ ated for the bridge do​ main. To cre​ ate an ex​ ter​ nal Layer 2 do​ main: 1 On the menu bar. not just an in​ di​ vid​ ual EPG under the bridge do​ main. Create an attachable access entity profile (AEP) to map the interfaces and policies together. See the Adding New De​ vices to the Fab​ ric sec​ tion for more in​ for​ ma​ tion on how to con​ fig​ ure an AEP and a phys​ ic ​al do​ main. To ac​ com​ plish an ex​ ten​ sion of the bridge do​ main to the out​ side. cre​ ate a new ex​ ter​ nal EPG to clas​ sify this ex​ ter​ nal traf​ fic. to the out​ side net​ work. create a Layer 2 domain if one does not already exist. Sim​ il​ ar to the pre​ vi​ ous ex​ am​ ple of adding an end​ point to a pre-ex​ ist​ ing EPG. you will also need to se​ lect which traf​ fic you would like to tra​ verse be​ tween the two EPGs. choose Action > Create Bridged Outside. 2 In the Work pane. choose the Tenant_Name. This VLAN must be configured on the external Layer 2 network. 4 5 In the Work pane. choose Tenant_Name > Networking > External Bridged Network.

b. Simply provide a name for the Layer 2 EPG. For more information on how to create a Access Attachable Entity Profile. choose External Bridged Networks > Networks and specify a contract to govern this policy as the consumed contract. This is a policy object that tells the APIC to allow certain encap (VLANs) on selected ports. e. After adding a Layer 2 border leaf and Layer 2 interface. and pro​ vides IP con​ nec​ tiv​ ity be​ tween a ten​ ant pri​ vate net​ work and an ex​ ter​ nal IP net​ work. and there​ fore there must be con​ nected to the ex​ ter​ nal net​ work. Extending ACI to External Layer 3 The most im​ por​ tant com​ po​ nent of any ap​ pli​ ca​ tion is the user or cus​ tomer. Configure a contract to allow communication between your existing endpoints in the existing EPG and your new external Layer 2 EPG. Add a Layer 2 border leaf node and Layer 2 interface for a Layer 2 outside connection. as well as to the In​ ter​ net to pro​ vide ac​ cess to the mo​ bile ap​ pli​ ca​ tion. the communication between this external Layer 2 EPG and your existing internal EPG will be allowed. In the Navigation pane. which gen​ er​ ally does not di​ rectly at​ tach to the fab​ ric. create a VLAN pool to associate to the VLAN on the Layer 2 outside connection. Each Layer 3 ex​ ter​ nal con​ nec​ tion is as​ so​ ci​ ated with one ten​ ant . After specifying this contract as the provided contract for your internal EPG. click Next to start creating a Layer 2 EPG. ACME must be able to con​ nect to both their in​ ter​ nal cor​ po​ rate back​ bone. All of the traffic entering the ACI fabric with the designated VLAN (the VLAN ID provided in step 1) will be classified into this Layer 2 EPG. This is a means to specify the range of the VLAN IDs that will be used for creating a Layer 2 outside connection. This in​ te​ gra​ tion is pos​ si​ ble with Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) at the ten​ ant pol​ icy level. You should now have the de​ sired reach​ ab ​il​ ity be​ tween the in​ side and out​ side Layer 2 seg​ ments.142 Fabric Connectivity ii. While creating the Layer 2 domain. c. d. Create an AEP. This helps avoid any overlapping in the VLAN range between VLANs used for an EPG and those in use for a Layer 2 outside connection. Layer 3 con​ nec​ tiv​ ity to a de​ vice like a router is known as an Ex​ ter​ nal Routed Net​ work. if it does not already exist. see the Adding New Devices to the Fabric section.

A bor​ der leaf is sim​ ply ter​ mi​ nol​ ogy to refer to a leaf that hap​ pens to be con​ nected to a Layer 3 de​ vice.Fabric Connectivity 143 pri​ vate net​ work. you can pro​ vide the ten​ ant ad​ min​ is​ tra​ tor with the abil​ ity to in​ ter​ face to an ex​ ter​ nal Layer 3 con​ nec​ tion in var​ i​ ous ways by using a uniquely-de​ fined Layer 3 con​ struct for the ten​ ant ap​ pli​ ca​ tion pro​ file or a shared com​ mon in​ fra​ struc​ ture. Ex​ ter​ nal Layer 3 con​ nec​ tions are usu​ ally es​ tab​ lished on the bor​ der leaf con​ struct of the ACI. In the ACI fab​ ric. In large scale ACI de​ signs it might be pro​ duc​ tive to have ded​ i​ cated ACI leafs as bor​ der leafs. As the op​ er​ a​ tor of the fab​ ric. Any ACI leaf can be​ come a bor​ der leaf. or tiers. of an ap​ pli​ ca​ tion into end​ point groups (EPGs). Other de​ vices. the ex​ ter​ nal Layer 3 con​ nec​ tion can be one of the fol​ low​ ing types: 1 Physical Layer 3 interface 2 Subinterface with 8021. These ap​ pli​ ca​ tion com​ po​ nents might have re​ quire​ ments for ex​ ter​ nal con​ nec​ tiv​ ity into them.Q tagging 3 Switched Virtual Interface (SVI) . The re​ quire​ ment of the Layer 3 ex​ ter​ nal net​ work is only needed when a group of de​ vices in the ap​ pli​ ca​ tion pro​ file re​ quire Layer 3 con​ nec​ tiv​ ity to a net​ work out​ side of the ACI fab​ ric. The fol​ low​ ing fig​ ure shows part of a sim​ pli​ fied fab​ ric: A sample application profile for a three-tiered application with contracts between the tiers For ex​ am​ ple. An ap​ pli​ ca​ tion pro​ file en​ ables an op​ er​ a​ tor to group the dif​ fer​ ent com​ po​ nents. the con​ nec​ tiv​ ity is de​ fined by a con​ tract to a de​ fined ex​ ter​ nal Layer 3 end​ point group. web servers need a con​ nec​ tion to the out​ side world for users to reach them. can still con​ nect to the bor​ der leaves. It is im​ por​ tant to note that the spine nodes can​ not have con​ nec​ tions to ex​ ter​ nal routers. like servers. With ACI.

• Shared . and the net​ works are not reach​ able to de​ vices ex​ ter​ nal to the fab​ ric. These sub​ nets can be clas​ si​ fied as pri​ vate. In re​ leases prior to ver​ sion 1. as it is ad​ ver​ tised to the ad​ ja​ cent router through the Layer 3 ex​ ter​ nal con​ nec​ tion.1 of Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC).​ 0/​ 24. public and private networks annotated With de​ vices con​ nect​ ing through the ex​ ter​ nal Layer 3 con​ nec​ tion. . In re​ lease ver​ sion 1. not just fab​ ric routes. ACI is ca​ pa​ ble of act​ ing as a tran​ sit net​ work.1 and later. and routes learned from one ex​ ter​ nal Layer 3 con​ nec​ tion can be ad​ ver​ tised out to a dif​ fer​ ent ex​ ter​ nal Layer 3 con​ nec​ tion. the fab​ ric only ad​ ver​ tises sub​ nets that are marked pub​ lic in the as​ so​ ci​ ated bridge do​ main. ACI does not ad​ ver​ tise the net​ works through the rout​ ing pro​ to​ col to the ad​ ja​ cent Layer 3 router. This be​ hav​ ior is known as a non-tran​ sit fab​ ric. The fol​ low​ ing fig​ ure de​ picts the logic of pub​ lic and pri​ vate net​ works: Application profile with external consumers. the ex​ ter​ nal net​ work has learned of the in​ ter​ nal ACI net​ work 10.Indicates that this subnet will be contained only within the ACI fabric private network. The net​ work team will pro​ vide the ex​ ter​ nal Layer 3 con​ nec​ tiv​ ity for their ten​ ants. or shared: • Public .​ 1.Indicates that this subnet will be leaked to one or more private networks inside of the ACI fabric.144 Fabric Connectivity Bridge do​ mains con​ tain one or more sub​ nets. For the pri​ vate net​ works. pub​ lic. • Private . One com​ mon mech​ a​ nism is to use sub-in​ ter​ faces on a router to cre​ ate dif​ fer​ ent Layer 3 do​ mains since each ten​ ant will likely not have their own ex​ ter​ nal router.Indicates that this subnet will be advertised to the external router.​ 1. Routes that are learned ex​ ter​ nally from the fab​ ric are not ad​ ver​ tised through other ports.

Without MP-BGP. To en​ able route re​ flec​ tors in the ACI fab​ ric. the fab​ ric in​ fra​ struc​ ture op​ er​ at​ or must con​ fig​ ure a Route Re​ flec​ tor pol​ icy to des​ ig​ nate which spines act as the route re​ flec​ tor(s). the fab​ ric ad​ min​ is​ tra​ tor must se​ lect at least one spine switch that will be a route re​ flec​ tor. but increases administrator overhead. ACI supports only one autonomous system (AS) number that has to match the one that is used for the internal Multiprotocol Border Gateway Protocol (MP-BGP) route reflector. For re​ dun​ dancy pur​ poses. including a default path out of the fabric. With OSPF NSSA.Fabric Connectivity 145 Supported Routing Protocols The fol​ low​ ing rout​ ing pro​ to​ cols are sup​ ported at time of writ​ ing: • Static routes—You can define static routes to the outside world. With static routes. Given that the same AS number is used for both cases. con​ fig​ ure more than one spine as a router re​ flec​ tor node. Using static routes reduces the sizing and complexity of the routing tables in the leaf nodes. and pro​ vide the au​ tonomous sys​ tem (AS) num​ ber for the fab​ ric. the router learns only a summarization of routes. and use that AS number as the BGP AS number for the ACI fabric. • OSPF NSSA—Using not-so-stubby area (NSSA) reduces the size of the Open Shortest Path First (OSPF) database and the need to maintain the overhead of the routing protocols with large tables of routes. Configure MP-BGP Spine Route Reflectors The ACI fab​ ric route re​ flec​ tors use mul​ ti​ pro​ to​ col bor​ der gate​ way pro​ to​ col (MP-BGP) to dis​ trib​ ute ex​ ter​ nal routes within the fab​ ric so a full mesh BGP topol​ ogy is not re​ quired. To con​ nect ex​ ter​ nal Layer 3 de​ vices to the ACI fab​ ric. you must also configure the static path back to the interanal network that you wish to be reachable from the outside world. Once route re​ flec​ tors are con​ fig​ ured. or BGP) for the Layer 3 outside connections are not propagated within the ACI fabric. • iBGP peering leaf and external router—With internal Border Gateway Protocol (iBGP). ad​ min​ is​ tra​ tors can setup con​ nec​ tiv​ ity to ex​ ter​ nal net​ works. and the ACI leaves that are not part of the border leaf does not have IP connectivity to any outside networks. . OSPF NSSA advertises to the adjacent router the internal public subnets part of the Layer 3 external. the user must find out the AS number on the router to which the ACI border leaf will connect. OSPF. the external routes (static.

which pairs the bor​ der leaf node with one of the route re​ flec​ tor nodes as a BGP peer. d. To con​ fig​ ure the Route Re​ flec​ tor pol​ icy: 1 On the menu bar. c. In the BGP Route Reflector Policy drop-down list. The in​ fra​ struc​ ture op​ er​ at​ or con​ fig​ ures each of the paired leaf nodes with the routes (or route pre​ fixes) that the nodes can ad​ ver​ tise. 6 In the Create Pod Policy Group dialog box. In the Navigation pane. choose default. in the Fabric Policy Group drop-down list. choose Actions > Create Pod Policy Group. b. in the Date Time Policy dropdown list. In the BGP Route Reflector Policy drop-down list. 7 Click Submit. 5 In the Work pane. Change the Autonomous System Number to match the required number for your network. b. 3 In the Work pane. choose Create Pod Policy Group. perform the following actions: a. they can ad​ ver​ tise routes in the fab​ ric. choose default. choose Pod Policies > Policies > BGP Route Deflector default. . In the Create Pod Policy Group dialog box. After the route re​ flec​ tors are con​ fig​ ured. choose Pod Policies. Complete the remainder of the dialog box as appropriate to your setup. 2 In the Navigation pane. choose default. f. Each leaf node can store up to 4000 routes at time of writ​ ing.146 Fabric Connectivity When a ten​ ant re​ quires a Layer 3 con​ nec​ tion. the router should peer with mul​ ti​ ple leaf nodes. In the Work pane. the in​ fra​ struc​ ture op​ er​ at​ or con​ fig​ ures the leaf node to which the WAN router is being con​ nected as bor​ der leaf node. Add the two spine nodes that will be members of this reflector policy and c. choose Fabric > Fabric Policies. perform the following actions: a. 4 In the Navigation pane. e. If a WAN router must ad​ ver​ tise more than 4000 routes. Click Submit. choose Pod Policies > Profiles > default.

Fabric Connectivity 147 The fol​ low​ ing fig​ ure il​ lus​ trates the ob​ jects and their re​ la​ tion​ ships for ex​ ter​ nal Layer 3 con​ nec​ tions: Object relationships for Layer 3 outside objects Layer 3 Integration Through Tenant Network with OSPF NSSA The fol​ low​ ing fig​ ure shows a sim​ ple in​ te​ gra​ tion of a Layer 3 ex​ ter​ nal into ACI using OSPF: Logical topology for an external OSPF router communicating with two border leafs .

choose Tenant_Name > Networking > External Routed Networks. choose the tenant. 4. 2. In the Nodes and Interfaces Protocol Profiles section. Note: See the "Adding New De​ vices to The Fab​ ric" sec​ tion to setup the ac​ cess poli​ cies for the in​ ter​ faces of the leaves that are con​ nected to the router. perform the following actions: i. click + to add a node. In the Private Network drop-down list. Click OK. such as "10. iii. 2 In the Work pane. click + to add a loopback address. In the OSPF Area Control section. enter the OSPF area ID.0. and click Update. choose Tenants > ALL TENANTS. b. choose Action > Create Routed Outside. In the Node ID drop-down list. In the OSPF Area Type section. In the Select Node dialog box. Uncheck the Router ID as Loopback Address check box. such as "10. In the Create Node Profile dialog box. c. ii. 5.254. enter a name for the profile. perform the following actions: a. click the Send redistributed LSAs into NSSA area check box. choose a node.1.148 Fabric Connectivity The setup in​ cludes a sin​ gle router with two in​ ter​ faces con​ nected to leaf switches. In the Loopback Addresses section. 6. choose the private network for this tenant. To in​ te​ grate Layer 3 through a ten​ ant net​ work with OSPF/NSSA: 1 On the menu bar. click the NSSA Area radio button. such as Leaf-1. . In the OSPF Area ID field. Enter the loopback address.​ g. h.1". e. In the Create Routed Outside dialog box. enter a name for the profile.1". In the Name field. such as "1". In the Name field. click + to add a profile. In the Router ID field. Click the OSPF check box. enter the router's IP address as the ID. 3.254. d. In the Nodes section. perform the following actions: 1. 3 In the Navigation pane. f. 4 5 In the Work pane.

In the Name field. choose Create OSPF Interface Policy. enter the IP address of the path that is attached to the layer 3 outside profile. perform the following actions: a. In the Path drop-down list. 4. Click OK. 6. such as e1/9 on Leaf-1. perform the following actions: a. click + to create an OSPF interface profile. Complete the remainder of the dialog box as appropriate to your setup. d.Fabric Connectivity 149 iv. In the OSPF Policy drop-down list. In the OSPF Interface Profiles section. enter the maximum MTU of the external network. Complete the remainder of the dialog box as appropriate to your setup and click OK. This document does not explain the different OSPF parameters. 5. c. perform the following actions: 1. In the Network Type section. In the Create External Network dialog box. In the MTU (bytes) field. perform the following actions: . you must specify the policy interaction. c. Click the + sign to select a routed interface.0. click on the Routed Intefaces tab. i. Click OK. In the Interfaces section. Click Next. such as Point to Point.1/24". b. click the radio button that matches the adjacent router. In the External EPG Networks section. In the Create OSPF Interface Policy dialog box. b.1. choose the interface on the leaf. In the Name field. 3. When defining the interaction with another OSPF router. such as "OSPFPoint2Point". In the Select Routed Interface dialog box. Click Submit. j. such as "10. enter a name for the OSPF policy. 2. k. vi. 7. enter a name for the profile. such as "1500" to match the example peering router. In the Create Interface Profile dialog box. v. d. In the IP Address field. click + create an external network.

you can use var​ io ​us mech​ an ​isms to reuse the same ex​ ter​ nal Layer 3 router for mul​ ti​ ple ten​ ants. If the ad​ ja​ cent router is a Cisco Nexus Se​ ries Switch with a Layer 2 trunk in​ ter​ face. enter 0. The fab​ ric op​ er​ at​ or can con​ fig​ ure mul​ ti​ ple ex​ ter​ nal Layer 3 con​ nec​ tions using ei​ ther sub-in​ ter​ face or SVI and pro​ vide that to each ten​ ant. you must configure the external network EPG.0. .0/0 to permit the learning of any subnet and click OK. those can be used to pro​ vide mul​ ti​ ple ex​ ter​ nal Layer 3 con​ nec​ tion for mul​ ti​ ple VRFs and/or ten​ ants. the ex​ ter​ nal Layer 3 con​ nec​ tion can be con​ fig​ ured to route via SVI. l.0. Click Finish. Next. For routers ca​ pa​ ble of using sub-in​ ter​ faces.150 Fabric Connectivity i. In the IP Address field. External Layer 3 for Multiple Tenants In ACI.

As​ sum​ ing that the SVIs re​ side on the ex​ ter​ nal Layer 2 net​ work. ACME must man​ age the mi​ gra​ tion of SVI in​ ter​ faces as well as the pol​ icy al​ low​ ing traf​ fic to tra​ verse a Layer 2 out​ side net​ work and then on to the ACI fab​ ric. servers or vir​ tu​ al​ iza​ tion hosts from out​ side the ACI fab​ ric. At a high level. onto the fab​ ric. They would like to mi​ grate with min​ im ​al ser​ vice in​ ter​ rup​ tion while tak​ ing ad​ van​ tage of ACI in​ no​ va​ tions where ap​ plic​ ab ​le. Once they have suc​ cess​ fuly es​ tab​ lished con​ nec​ tiv​ ity be​ tween the out​ side Layer 2 net​ work (where the work​ load or host is com​ ing from) and the ex​ ist​ ing in​ ter​ nal fab​ ric EPG. can be found in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter of this book. In this ex​ am​ ple. Fur​ ther in​ for​ ma​ tion and steps on how to cre​ ate this Layer 2 and Layer 3 con​ nec​ tiv​ ity in​ clud​ ing the pol​ icy. you can then start the mi​ gra​ tion process of mov​ ing ap​ pli​ ca​ tion work​ loads onto the fab​ ric. One com​ mon ex​ am​ ple is when mi​ grat​ ing from a tra​ di​ tional data cen​ ter con​ fig​ ur​ a​ tion over to a pol​ icy-dri​ ven data cen​ ter using ACI. . it will be come nec​ es​ sary to per​ form these mi​ gra​ tions. You will also need to con​ fig​ ure Layer 3 con​ nec​ tiv​ ity from the fab​ ric out to the ex​ ist​ ing Layer 3 net​ works in prepa​ ra​ tion for full con​ nec​ tiv​ ity after SVI mi​ gra​ tion. you must start by con​ fig​ ur​ ing the Layer 2 out​ side net​ work to allow traf​ fic from the source VLAN to com​ mu​ ni​ cate with the same VLAN re​ sid​ ing on the ACI fab​ ric. ACME would like to per​ form the mi​ gra​ tion in mul​ ti​ ple stages. there can be oc​ ca​ sions when you will have to mi​ grate work​ loads. Cisco rec​ om​ mends that you move the SVIs over to the ACI fab​ ric once a ma​ jor​ ity of the hosts have been mi​ grated over. As ACME starts to use ACI in more of their data cen​ ters.Fabric Connectivity 151 Application Migration Use Case When op​ er​ at​ ing the ACI fab​ ric. One key con​ sid​ er​ at​ ion should be when to switch over the SVI in​ ter​ faces from the ex​ ist​ ing en​ vi​ ron​ ment into the ACI fab​ ric and when to start ad​ ver​ tis​ ing routes to this SVI net​ work. Extending the Network to ACI One of the ACME sites would like to mi​ grate from the legacy data cen​ ter ar​ chi​ tec​ ture to the next gen​ er​ at​ ion ACI Fab​ ric.

152 Fabric Connectivity The Legacy data cen​ ter prior to mi​ gra​ tion: Traditional pre-migration data center The ACI data cen​ ter fol​ low​ ing mi​ gra​ tion: Post-migration ACI based data center topology .

In the Create Tenant dialog box perform the following actions: a. choose Tenants > Add Tenant. 1 On the menu bar. or a sin​ gle in​ ter​ face. . you log​ ic ​ally map a VLAN=EPG. The cre​ ated ten​ ant will rep​ re​ sent the legacy data cen​ ter into the ACI fab​ ric. 1 Provide a physical connection from aggregation switch #1 to the ACI border leaf #1. 1 2 On the menu bar. Note: Be​ fore con​ nect​ ing ex​ ter​ nal phys​ ic ​al con​ nec​ tions into the fab​ ric. the Fab​ ric Ac​ cess Poli​ cies for the ac​ cess ports that you will be used for the DCI must be con​ fig​ ured. choose the Tenant_Name. The in​ ter​ con​ nect from the legacy net​ work to the ACI fab​ ric will be ac​ com​ plished through stan​ dard Layer 2 ex​ ten​ sions (VLAN/VXLAN). 3 Click Finish. In the APIC con​ troller you will now con​ fig​ ure a sin​ gle ten​ ant. In the Name field enter a name for the tenant. This allows for the host VLAN to be extended into the fabric. For de​ tails on con​ fig​ ur​ ing the ac​ cess poli​ cies please ref​ er​ ence the Fab​ ric Con​ nec​ tivty sec​ tion of this book.Fabric Connectivity 153 The first stage will pro​ vide con​ nec​ tiv​ ity from the legacy data cen​ ter to the ACI fab​ ric. 2 In the Work pane. Con​ fig​ ure the ag​ gre​ ga​ tion links as a Layer 2 trunk. b. This con​ nec​ tiv​ ity can be ac​ com​ plished in ei​ ther the form of a Vir​ tual Port Chan​ nel. Pro​ vide phys​ ic ​al con​ nec​ tiv​ ity from the ex​ ist​ ing ag​ gre​ ga​ tion layer to the ACI bor​ der leafs. 2 Provide a physical connection from aggregation switch #2 to the ACI border leaf #1. Con​ fig​ ure a sin​ gle pri​ vate net​ work. choose Tenants > ALL TENANTS. 1 Trunk the VLAN representing the host connectivity. In this state. Click Next. Port Chan​ nel.

perform the following actions: a. choose Actions > Create Private Network. choose Tenants > ALL TENANTS. choose the Tenant_Name. choose Tenants > ALL TENANTS 2 In the Work pane. 3 In the Navigation pane choose Application Profiles. For the Layer 2 Unknown Unicast radio buttons. In the Name field enter a name for the bridge domain. Con​ fig​ ure a sin​ gle ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. In the Name field enter a name for the Application Profile. 4 In the Work pane. Click Next. f. d. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name.154 Fabric Connectivity 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. Con​ fig​ ure a sin​ gle end​ point group: 1 On the menu bar. 5 In the Create Application Profile dialog box. By de​ fault BPDU frames are flooded within EPG. 2 In the Work pane. perform the following actions: a. 4 In the Work pane. For the Multi Destination Flooding radio buttons. click Flood. Click the ARP Flooing check box. g. choose the Tenant_Name. choose Actions > Create Application Profile. 5 In the Create Private Network dialog box. the bridge do​ main will then mimic the be​ hav​ ior in the ACI Fab​ ric. e. 6 Click Finish. b. click Flood in BD. b. . c. In the Forwarding drop-down list. Note: The con​ cept of flood​ ing the un​ known uni​ cast and arp within the Fab​ ric is to allow for the Layer 2 se​ man​ tics from the legacy data cen​ ter to be ex​ tended into the ACI Fab​ ric. Click Submit. choose Custom. In the Name field enter a name for the private network. When a host in the legacy data cen​ ter sends an ARP re​ quest and/or floods an un​ known uni​ cast frame.

Note: The EPG within the fab​ ric will map to a sin​ gle VLAN in the legacy data cen​ ter. Click Finish. Click Submit. the Legacy host VLAN is now ex​ tended to the ACI fab​ ric and all hosts from the Fab​ ric point of view are in the EPG. enter a name for the endpoint group. The use of a sin​ gle VLAN per EPG will pro​ vide a path for a net​ work-cen​ tric mi​ gra​ tion— min​ im ​iz​ ing im​ pact while in​ tro​ duc​ ing fab​ ric in​ no​ va​ tions. 2 In the Work pane. choose Actions > Deploy Static EPG on PC. In the Bridge Domain field. VPC or Interface. From the ACI Fab​ ric per​ spec​ tive. This is what is known as nor​ mal​ iza​ tion where ACI uses the tagged vlan to map ex​ ter​ nal Layer 2 con​ nec​ tions into ACI End Point Groups. b. . Choose the Path Type. choose the appropiate bridge domain. 1 On the menu bar. c. Fol​ low​ ing the Stage 1 mi​ gra​ tion. Acme​ Out​ Side. con​ fig​ ure a sta​ tic trunk bind​ ing using the VPC under the EPG. See the "Fab​ ric Con​ nec​ tiv​ ity" sec​ tion. the local VLAN num​ ber is not sig​ nif​ ic ​ant as it will be mapped to the tagged VLAN on the VPC to​ ward the Legacy data cen​ ter. Con​ fig​ ure a VPC for the con​ nec​ tiv​ ity to the legacy data cen​ ter. 4 In the Work pane.Fabric Connectivity 155 4 5 In the Work pane. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). In the Name field. perform the following actions: a. The en​ cap​ su​ la​ tion VLAN should match the de​ fined legacy data cen​ ter VLAN de​ f​ in ​i​ t​ ion. choose Tenants > ALL TENANTS. VPC or Interface dialog box. Then. Acme​ Out​ Side. Enter the encapsulation VLAN. In the Create Application EPG dialog box. 5 In the Deploy Static EPG on PC. perform the following actions: a. c. Choose the Path. b. choose Actions > Create Application EPG. d.

Create a static binding for each physical host to be migrated. 2 Begin migrating the host to the fabric EPG. i. Dur​ ing this stage. . Create a VMM domain to deploy the host within the ACI Fabric (optional). The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter or pro​ vided by the Legacy data cen​ ter. For the Allow drop-down list. VLAN pools. c. Configure contracts for inter-EPG communications. ii. Configure additional Private Networks (optional). Configure additional End Point Groups. b. Create the endpoint group "AcmeInSide". For the Scope drop-down list. d. AEP. see the "Fab​ ric Con​ nec​ tivty" sec​ tion. Note: En​ sure the ap​ pro​ pri​ ate re​ sources have been cre​ ated to sup​ port the con​ nec​ tiv​ ity. Configure additional Application Profiles (optional).156 Fabric Connectivity The con​ nec​ tiv​ ity noted below rep​ re​ sents a BD=EPG=VLAN. and switch or in​ ter​ face pro​ files. Configure additional Tenants (optional). phys​ i​ cal do​ main or VMM Do​ main. Configure additional Bridge Domains (optional). a. b. An EPG as a VLAN datacenter deployment The sec​ ond stage of the mi​ gra​ tion takes ad​ van​ tage of the fab​ ric pol​ icy model to de​ fine fur​ ther the ap​ pli​ ca​ tion mod​ el​ ing. choose Tenant. "AcmeInSide". f. i. The fol​ low​ ing list is an overview of the steps that you must per​ form: 1 In the APIC: a. mul​ ti​ ple EPGs will be cre​ ated with APIC con​ tracts that de​ fine reach​ a​ bil​ ity. e. choose Any-Any. For more in​ for​ ma​ tion.

the host con​ nec​ tiv​ ity across both the legacy data cen​ ter and ACI fab​ ric are now gov​ erned by the APIC pol​ icy (con​ tracts). Note: The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter are pro​ vided by the legacy data cen​ ter.Fabric Connectivity 157 Fol​ low​ ing the stage 2 mi​ gra​ tion. choose Unknown Unicast. Datacenter migration with Layer 3 provided by existing DC and Layer 2 extended to new ACI datacenter In the third stage of the mi​ gra​ tion the Layer 3 gate​ way will move from the legacy data cen​ ter to the ACI Fab​ ric. The fol​ low​ ing list is an overview of the steps that you must per​ form: 1 In the APIC: a. iv. iii. choose Routing. ii. In the Flood Layer 2 drop-down list. In the Unicast drop-down list. choose AcmeBD. In the Bridge Domain drop-down list. The Layer 3 gate​ way for the ACI fab​ ric and the legacy data cen​ ter are pro​ vided by the ACI fab​ ric. In the ARP drop-down list. i. Configure a Layer 3 out. Note: The con​ cept of uni​ cast rout​ ing within the bridge do​ main al​ lows for the con​ fig​ u​ ra​ tion of the per​ va​ sive gate​ way across the fab​ ric. b. . Migrate the gateway from the legacy data center to the fabric. choose Flooding.

.

159 Tenants .

.

Tenants 161 Section Content • • ACI Tenancy Models Application Profile Application Profile Configuration Create a New Application Profile Modify Application Profile Remove Application Profile Verify Application Profile • Endpoint Group Endpoint Group Configuration Create a New Endpoint Group Modify Endpoint Group Remove Endpoint Group Verify Endpoint Group • Endpoint Verify Endpoint Group • Private Network Private Network Configuration Parameters Creating a New Private Network Modify Private Network Remove Private Network Verify Private Network .

162 Tenants • Bridge Domain Bridge Domain Configuration Parameters Create a new Bridge Domain Modify a Bridge Domain Remove a Bridge Domain Verify Bridge Domain • Tenant Networking Use Cases Common Private Network for All Tenants Multiple Private Networks with Intra-Tenant Communication Multiple Private Networks with Inter-Tenant Communication .

The ten​ ant ob​ ject gives us the abil​ ity to draw a box around the log​ ic ​al and con​ crete ob​ jects that we use to pro​ vide a uni​ fied view of the con​ fig​ ur​ a​ tion de​ pen​ den​ cies for un​ der​ lay and over​ lay net​ works. and Production. The in​ for​ ma​ tion se​ cu​ rity team will be able to in​ te​ grate this into the cor​ po​ rate LDAP sys​ tem. 3 Separation of duties by domain owner. app. such as web. This means dif​ fer​ ent things to dif​ fer​ ent peo​ ple (much like the term Cloud) based on their per​ spec​ tive. In the case of a clas​ sic ser​ vice provider. In tra​ di​ tional net​ work​ ing en​ vi​ ron​ ments. while in a typ​ ic ​al endcus​ tomer en​ vi​ ron​ ment a ten​ ant could be an op​ er​ at​ ing group.Tenants 163 ACI Tenancy Models ACME Inc. Ten​ ants will be used to draw vir​ tual bound​ aries for dif​ fer​ ent lines of busi​ ness. and pre​ vent changes which would im​ pact other groups. such as different business units. Ad​ di​ tion​ ally. They will be using ten​ ant con​ structs for the ap​ pli​ ca​ tion life​ cy​ cle of their cur​ rent de​ ploy​ ment. ap​ pli​ ca​ tion owner. etc. 2 Separation of environments from a software development lifecycle perspective: Development. . ACI has been de​ signed from the be​ gin​ ning to be “multi-ten​ ant”. and fi​ nally a pro​ duc​ tion ten​ ant. a ten​ ant that will be used for the au​ to​ mated test​ ing. security. as men​ tioned in the in​ tro​ duc​ tion. servers. 4 Fault domain size and scope to limit the impact of failures. and so on. Lever​ ag​ ing the ACI pol​ icy model. they are also look​ ing to build an in​ fra​ struc​ ture which can be lever​ aged for sim​ il​ ar ini​ tia​ tives in the fu​ ture. busi​ ness unit. This in​ tro​ duces a war​ ranted level of cau​ tion around change con​ trol and ap​ pli​ ca​ tion im​ pact. the phys​ ic ​al hard​ ware is ab​ stracted from the log​ ic ​al con​ structs. mak​ ing a rout​ ing pro​ to​ col change on a router or Layer 3 switch could po​ ten​ tially af​ fect hun​ dreds of unique VLANs/sub​ nets. and database owners. The de​ ci​ sion on how to lever​ age ten​ ancy mod​ els is dri​ ven by a num​ ber of fac​ tors: 1 Overall IT operations and support models in your organization to manage application. Quality Assurance. networking. main​ tain​ ing a sep​ a​ rate ten​ ant for the re​ sources that de​ vel​ op​ ers will be using to build the ap​ pli​ ca​ tion. a ten​ ant is a unique cus​ tomer. will be using ten​ ancy for a cou​ ple of use cases.

the ten​ ant poli​ cies are where you de​ fine ap​ pli​ ca​ tions. a web​ site could use a 3-tier ap​ pli​ ca​ tion model. an ap​ pli​ ca​ tion can be formed by sev​ eral end​ point groups and they are grouped in an Ap​ pli​ ca​ tion Pro​ file. In the next sec​ tions of this chap​ ter these con​ cepts will be cov​ ered in de​ tail. Fi​ nally. There​ fore. In​ side. Al​ though the ten​ ant net​ work​ ing and the ten​ ant poli​ cies are de​ fined sep​ ar​ ately. End​ points are clas​ si​ fied in ACI to apply poli​ cies. . In the fol​ low​ ing sec​ tions each di​ ag ​ram shows the progress of how ACME Inc. The web servers in turn com​ mu​ ni​ cate with core ap​ pli​ ca​ tions that can be di​ vided amongst sev​ eral ap​ pli​ ca​ tions servers for load bal​ anc​ ing or high avail​ abil​ ity pur​ poses. adds each com​ po​ nent. The fol​ low​ ing image shows all of the com​ po​ nents that can be con​ fig​ ured within a ten​ ant. bridge do​ mains and sub​ nets. An ap​ pli​ ca​ tion could con​ sist of a com​ bi​ na​ tion of phys​ ic ​al servers or VMs that we will call servers from now on. they might ac​ tu​ ally be com​ mu​ ni​ cat​ ing with a vir​ tual IP ad​ dress on a load bal​ ancer that in turn can dis​ trib​ ute the web re​ quest to a num​ ber of dif​ fer​ ent web servers. The ten​ ant net​ work​ ing is used to de​ fine net​ work​ ing poli​ cies and will be ap​ plied to the un​ der​ ly​ ing hard​ ware in a trans​ par​ ent way thanks to the layer of ab​ strac​ tion pro​ vided by ACI using pri​ vate net​ works. You cre​ ate end​ point groups with end​ points that share the same type of poli​ cies. When a user browses the web site. com​ prised of web servers. the net​ work​ ing poli​ cies used by an ap​ pli​ ca​ tion are de​ fined with a re​ la​ tion​ ship be​ tween the End​ point Groups and the Bridge Do​ main. and the ob​ jects that de​ fine the ten​ ant poli​ cies such as Ap​ pli​ ca​ tion Pro​ files and End​ point Groups. such as with whom are they going to com​ mu​ ni​ cate and what type of com​ mu​ ni​ ca​ tion or re​ stric​ tions are re​ quired. In ACI. Below you can find an il​ lus​ tra​ tion with the dif​ fer​ ent ob​ jects that com​ pound a ten​ ant and how they are re​ lated. such as Pri​ vate Net​ works. you can dif​ fer​ en​ ti​ ate be​ tween the ob​ jects that de​ fine the ten​ ant net​ work​ ing. Bridge Do​ mains and sub​ nets. Each server is re​ ferred to as an End​ point in ACI. ap​ pli​ ca​ tion servers and data​ base servers.164 Tenants A Ten​ ant in the ACI Ob​ ject model rep​ re​ sents the high​ est-level ob​ ject. the ap​ pli​ ca​ tion servers com​ mu​ ni​ cate with the data​ base which could also be a clus​ ter of servers. For ex​ am​ ple.

Tenants 165

Tenant Logical Model
There are 3 Ten​
ants pre​
con​
fig​
ured in the sys​
tem by de​
faut:
1

Common – a special tenant with the purpose of providing “common” services to
other tenants in the ACI fabric. Global reuse is a core principle in the common
tenant. Some examples of common services are:
a. Shared Private Networks
b. Shared Bridge Domains
c. DNS
d. DHCP
e. Active Directory

2

Infra – The Infrastructure tenant that is used for all internal fabric
communications, such as tunnels and policy deployment. This includes switch
to switch (Leaf, Spine, Application Virtual Switch (AVS)) and switch to APIC.
The infra tenant does not get exposed to the user space (tenants) and it has its
own private network space and bridge domains. Fabric discovery, image
management, and DHCP for fabric functions are all handled within this tenant.

3

Mgmt - The management tenant provides convenient means to configure
access policies for fabric nodes. While fabric nodes are accessible and
configurable through the APIC, they can also be accessed directly using in-band
and out-of band connections. In-band and out-of-band policies are configured
under the mgmt tenant:

In-Band Management Access

Out-of-Band Management Access

Tenants 167

Application Profile
An ap​
pli​
ca​
tion pro​
file is a con​
ve​
nient log​
i​
cal con​
tainer for mul​
ti​
ple hosts (phys​
i​
cal or
vir​
tual). You can cre​
ate Ap​
pli​
ca​
tion Pro​
file con​
tain​
ers based on a va​
ri​
ety of cri​
te​
ria,
such as what func​
tion the ap​
pli​
ca​
tion pro​
vides, how the ap​
pli​
ca​
tion looks from the
end-user per​
spec​
tive, where they are lo​
cated within the con​
text of the data cen​
ter, or
any other log​
i​
cal group​
ing rel​
a​
tive to the im​
ple​
men​
ta​
tion. Ap​
pli​
ca​
tion Pro​
file servers
are grouped in EPGs de​
pend​
ing on the use of com​
mon poli​
cies.
Ap​
pli​
ca​
tion Pro​
files pro​
vide a mech​
a​
nism to un​
der​
stand groups of servers as a sin​
gle
ap​
pli​
ca​
tion. This ap​
proach makes an ACI ap​
pli​
ca​
tion aware and al​
lows us to check the
op​
er​
a​
tional state for an ap​
pli​
ca​
tion mon​
i​
tor​
ing all the servers that are part of an ap​
pli​
ca​
tion as a whole and be​
come in​
formed about rel​
e​
vant faults and health sta​
tus for that
par​
tic​
u​
lar ap​
pli​
ca​
tion. Each Ap​
pli​
ca​
tion Pro​
file cre​
ated can have a unique mon​
i​
tor​
ing
pol​
icy and QOS pol​
icy.
An Ap​
pli​
ca​
tion Pro​
file is a child ob​
ject of the Ten​
ant and a sin​
gle Ten​
ant can con​
tain
mul​
ti​
ple Ap​
pli​
ca​
tion Pro​
files.

Adding components to a Tenant - 1. Application Profile

168 Tenants

Application Profile Configuration
Name - The name of the ap​
pli​
ca​
tion pro​
file.
Tags - A tag or meta​
data is a non-hi​
er​
ar​
chi​
cal key​
word or term as​
signed to the fab​
ric
mod​
ule.
Mon​
i​
tor​
ing Pol​
icy - The mon​
it​
or​
ing pol​
icy name for the EPG se​
man​
tic scope (op​
tional).

Create a New Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles.

4

In the Work pane, choose Actions > Create Application Profile.

5

In the Create Application Profile dialog box, perform the following actions:
a. Enter an Application Profile Name.
b. Enter a TAG (optional).
c. Choose a Monitoring Policy (optional).

6

Click Submit.

Modify Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles >
Application_Profile.

4
5

In the Work pane, choose policy.
In the Create Application Profile dialog box, perform the following actions:
a. Enter an Application Profile Name.
b. Enter an appropriate TAG (optional).
c. Choose the Monitoring Policy (optional).

6

Click Submit.

Tenants 169

Remove Application Profile
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles
> Application_Profile.

4

In the Work pane, choose Actions > Delete.

Verify Application Profile
REST :: /api/node/class/fvAp.xml
CLI :: moquery -c fvAp

Tenants 171

Endpoint Group
End​
point Groups are used to cre​
ate log​
i​
cal group​
ings of hosts or servers that per​
form
sim​
i​
lar func​
tions within the fab​
ric. Each End​
point Group cre​
ated can have a unique
mon​
i​
tor​
ing pol​
icy and QoS pol​
icy and must be as​
so​
ci​
ated with a Bridge Do​
main.
An End​
point group is a child ob​
ject of the Ap​
pli​
ca​
tion Pro​
file and an Ap​
pli​
ca​
tion Pro​
file can con​
tain mul​
ti​
ple End​
point Groups. Each end​
point within an End​
point Group is
sus​
cep​
ti​
ble to the same pol​
icy in the Fab​
ric.

Adding components to a tenant - 2. End Point Group in the Application Profile
All the End​
points in​
side an EPG can com​
mu​
ni​
cate with each other. How com​
mu​
ni​
ca​
tions be​
tween EPGs con​
tracts will be re​
quired is gov​
erned by con​
tracts and not tra​
di​
tional Layer 2/Layer 3 for​
ward​
ing con​
structs. For ex​
am​
ple, Host-A in EPG-A can have
the IP ad​
dress/mask of 10.​
1.​
1.​
10/​
24 and Host B in EPG B can have the IP ad​
dress/mask 10.​
1.​
1.​
20/​
24 (note that both hosts be​
lieve they are "in the same sub​
net"). In
this case they would not be al​
lowed to com​
mu​
ni​
cate un​
less a con​
tract that per​
mit​
ted
con​
nec​
tiv​
ity ex​
isted be​
tween EPG-A and EPG-B. Con​
tracts will be ex​
plained in greater
de​
tail in a fol​
low​
ing sec​
tion.
Note that there are some types of End​
point Groups within the fab​
ric that are not con​
tained under Ap​
pli​
ca​
tion Pro​
files such as, Ap​
pli​
ca​
tion End​
point Group, Ex​
ter​
nal Bridge
Net​
works (aka Lay​
er2 Ex​
ter​
nal), Ex​
ter​
nal Routed Net​
works (aka as Lay​
er3 Ex​
ter​
nal) and
Man​
age​
ment End​
point Groups. These End​
point Groups might have spe​
cial re​
quire​
-

172 Tenants

ments, for ex​
am​
ple, in Ex​
ter​
nal Bridge Net​
works, MAC ad​
dresses of the end​
points are
not learnt by the leaf switches.
End​
point Groups are linked to Bridge Do​
mains but they will re​
ceive a VLAN ID dif​
fer​
ent
from the bridge do​
main, un​
less Bridge Do​
main legacy mode is used.
It is im​
por​
tant to un​
der​
stand that a sin​
gle sub​
net can be ex​
tended across sev​
eral EPGs.
Each EPG is iden​
ti​
fied by an en​
cap​
su​
la​
tion VLAN or VXLAN so that the same sub​
net will
be using dif​
fer​
ent en​
cap​
su​
la​
tion IDs across the fab​
ric. This con​
cept is dif​
fer​
ent from
tra​
di​
tional net​
work​
ing.

Endpoint Group Configuration
Name - The name for the end​
point group.
Tag - A tag or meta​
data is a non-hi​
er​
ar​
chi​
cal key​
word or term as​
signed to the fab​
ric
mod​
ule.
Qos Class - The QoS pri​
or​
ity class iden​
ti​
fier.
The class can be:

Unspecified

Level1-Class 1 Differentiated Services Code Point (DSCP) value.

Level2-Class 2 DSCP value.

Level3-Class 3 DSCP value.

Cus​
tom Qos - The QoS traf​
fic pri​
or​
ity class iden​
ti​
fier. The Cus​
tom class is a user-con​
fig​
urable DSCP value.
Bridge Do​
main - The name of the bridge do​
main as​
so​
ci​
ated with this ob​
ject.
it​
or​
ing pol​
icy name for the EPG se​
man​
tic scope (op​
tional).
Mon​
it​
or​
ing Pol​
icy – The mon​
As​
so​
ci​
ated Do​
main Pro​
file – A source re​
la​
tion to an in​
fra​
struc​
ture do​
main pro​
file as​
so​
ci​
ated with ap​
pli​
ca​
tion end​
point groups.

Tenants 173

Sub​
net - An End​
point Group sub​
net is rel​
e​
vant when and only when con​
fig​
ur​
ing route
leak​
ing be​
tween VRF's/Pri​
vate Net​
work within a Ten​
ant (op​
tional).
Sta​
tic End​
point - The sta​
tic client end​
point rep​
re​
sents a silent client end​
point at​
tached
to the net​
work (op​
tional).

Create a New Endpoint Group
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles >
Application_Profile > Application EPGs.

4
5

In the Work pane, choose Actions > Create Application EPG.
In the Create Application EPG dialog box, perform the following.
a. Enter an Application EPG Name.
b. Enter an Tag (optional).
c. Enter an Qos Class (optional).
d. Enter an Custom Qos (optional).
e. Enter a Bridge Domain Name.
f. Choose a Monitoring Policy (optional).
g. Enter an Associated Domain Profile Name.

6

Click Finish.

Modify Endpoint Group
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the Tenant_Name.

3

In the Navigation pane choose Tenant_Name > Application Profiles
> Application_Profile > Application EPGs > Application_EPG.

4

In the Work pane, select policy.
a. Enter an Application EPG Name.
b. Enter an Tag (optional).
c. Enter an Qos Class (optional).
d. Enter an Custom Qos (optional).

3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile > Application EPGs > Application_EPG.. 2 In the Work pane.174 Tenants e. Choose the appropriate Monitoring Policy if applicable (optional).xml CLI :: moquery -c fvAEPg . 5 Click Finish. choose Tenants > ALL TENANTS. Verify Endpoint Group REST :: /api/node/class/fvAEPg. Enter an Associated Domain Profile Name. Remove Endpoint Group 1 On the menu bar. Enter a Bridge Domain Name. choose Actions > Delete. 4 In the Work pane. choose the Tenant_Name. f. g.

. 2 In the Work pane. an en​ cap​ su​ la​ tion. Choose the Path Type. Each end​ point has a path. VPC or Interface dialog box. An End​ point is a child ob​ ject of the End​ point Group and an End​ point Group con​ struct can con​ tain mul​ ti​ ple End​ points. choose the Tenant_Name. perform the following actions: a. a lo​ ca​ tion. See the VVM sec​ tion for an ex​ am​ ple of a dy​ namic bind​ ing. Click Submit. c. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Static Bindings (Paths). choose Actions > Deploy Static EPG on PC. 5 In the Deploy Static EPG on PC. Below is an ex​ am​ ple of a sta​ tic bind​ ing. 4 In the Work pane. and at​ trib​ utes. 2 In the Work pane. choose Tenants > ALL TENANTS. b. d. In order to show the end​ points that are con​ nected to the fab​ ric under cer​ tain EPGs: 1 On the menu bar. You can add Sta​ tic End​ points by cre​ at​ ing Sta​ tic Bind​ ings within the End​ point Group. Enter the encapsulation VLAN. and a de​ ploy​ ment Im​ me​ di​ acy mode as​ so​ ci​ ated with it. VPC or Interface. Choose the Path.Tenants 175 Endpoint End​ points are de​ vices that are con​ nected to the net​ work ei​ ther di​ rectly or in​ di​ rectly. and can be ei​ ther vir​ tual or phys​ ic ​al. 1 On the menu bar. The End​ points ref​ er​ enced within the fab​ ric can be ei​ ther sta​ tic (de​ fined within the APIC) or dy​ namic (au​ to​ mated by vCen​ ter/Open​ stack). End​ points have an ad​ dress (iden​ tity). choose Tenants > ALL TENANTS.

4 In the Work pane.xml CLI :: moquery -c fvCEp .176 Tenants 3 In the Navigation Pane choose Tenant_Name > Application Profiles > Application_Profile > Application EPGs > Application_EPG. choose Operational. Verify Endpoint REST :: /api/node/class/fvCEp.

Pri​ vate net​ works are a child of the Ten​ ant ob​ ject. . a Pri​ vate Net​ work that is in​ tended to be used by mul​ ti​ ple ten​ ants and is not cre​ ated in the com​ mon ten​ ant re​ quires ex​ plicit con​ fig​ u​ ra​ tion to be shared. see the overview sec​ tion of this chap​ ter. pri​ vate Layer 3 net​ work. spe​ cial con​ sid​ er​ a​ tion for sub​ net con​ fig​ u​ ra​ tion is needed. One or more bridge do​ mains are as​ so​ ci​ ated with a pri​ vate net​ work. How​ ever. This will be dis​ cussed in de​ tail in the Bridge Do​ main and EPG con​ fig​ u​ ra​ tion sec​ tions.3.Tenants 177 Private Network A Pri​ vate Net​ work is also re​ ferred to as a VRF. Pri​ vate Net​ works cre​ ated in the com​ mon ten​ ant are shared glob​ ally within the fab​ ric. or con​ text. For more in​ for​ ma​ tion about com​ mon ten​ ants. It is a unique Layer 3 for​ ward​ ing and ap​ pli​ ca​ tion pol​ icy do​ main. Adding components to a Tenant . When there is a re​ quire​ ment to route traf​ fic be​ tween sep​ a​ rate Pri​ vate Net​ work in​ stances. Private Network as part of the Tenant Logical Model The most com​ mon method to share Pri​ vate Net​ works be​ tween ten​ ants is through the com​ mon ten​ ant. All of the end​ points within the Pri​ vate Net​ work must have unique IP ad​ dresses be​ cause it is pos​ si​ ble to for​ ward pack​ ets di​ rectly be​ tween these de​ vices if the pol​ icy al​ lows it.

Choose the appropriate DNS Label if applicable (optional). perform the following actions: a. Pol​ icy En​ force​ ment .The net​ work do​ main name label. choose the Tenant_Name.Name of the BGP timers pol​ OSPF Timers . e. Choose an End Point Retention Policy Name (optional). 4 5 In the Work pane. Creating a New Private Network 1 On the menu bar. The de​ fault is en​ forced. con​ tracts be​ tween EPGs are re​ quired to allow en​ forced. Choose a BGP Policy Name (optional). Un​ en​ forced al​ lows all traf​ fic within the Pri​ vate Net​ work. g. When en​ traf​ fic. Choose the appropriate Monitoring Policy if applicable (optional).Name of the OSPF timers pol​ icy as​ so​ ci​ ated with this ob​ ject.The end point re​ ten​ tion pol​ icy name (op​ tional). The val​ ues can be en​ forced or un​ forced is cho​ sen. b. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks.The pre​ ferred pol​ icy con​ trol. choose Tenants > ALL TENANTS.The mon​ DNS Label . icy as​ so​ ci​ ated with this ob​ ject. i​ tor​ ing pol​ icy name for the Ten​ ant se​ man​ tic scope (op​ tional). choose Actions > Create Private Network. 2 In the Work pane. End Point Re​ ten​ tion Pol​ icy . Mon​ i​ tor​ ing Pol​ icy . 6 Click Finish.The name of the Pri​ vate Net​ work. Choose an OSPF Policy Name (optional). Enter an Private Network Name. In the Create Private Network dialog box. Choose a Policy Enforcement (optional). d. BGP Timers . . f.178 Tenants Private Network Configuration Parameters Name . La​ bels en​ able clas​ si​ fy​ ing which ob​ jects can and can​ not com​ mu​ ni​ cate with one an​ other (op​ tional). c.

Verify Private Network REST :: /api/node/class/fvCtx. 4 In the Work pane. Choose a BGP Policy Name (optional).Tenants 179 Modify Private Network 1 On the menu bar. g. Choose the appropriate DNS Label if applicable (optional). choose the Tenant_Name. 5 Click Finish. b.xml CLI :: moquery -c fvCtx . 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network. Choose an OSPF Policy Name (optional). 4 In the Work pane. Choose the appropriate Monitoring Policy if applicable (optional). choose Tenants > ALL TENANTS. Choose an EIGRP Policy Name (optional). 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network. Remove Private Network 1 On the menu bar. choose Tenants > ALL TENANTS. d. c. a. choose the Tenant_Name. e. select policy. 2 In the Work pane. choose Actions > Delete. Choose an End Point Retention Policy Name (optional). Choosesa Policy Enforcement (optional). f.

.

While a Pri​ vate Net​ work de​ fines a unique IP ad​ dress space. A bridge do​ main can have mul​ ti​ ple sub​ nets. Those sub​ nets will be spread across one or more bridge do​ mains con​ tained in the Pri​ vate Net​ work. Adding components to a Tenant . How​ ever. Bridge Domain as part of the Tenant Application Profile The fol​ low​ ing image pro​ vides an ex​ am​ ple of a ten​ ant to show how bridge do​ mains are con​ tained in​ side of Pri​ vate Net​ works and how they are linked to EPGs and the other el​ e​ ments.Tenants 181 Bridge Domain A Bridge Do​ main is the ab​ stract rep​ re​ sen​ ta​ tion of a Layer 2 for​ ward​ ing do​ main within the fab​ ric. A Bridge Do​ main is a child of the Ten​ ant ob​ ject and must be linked to a Pri​ vate Net​ work.4. . that ad​ dress space can con​ sist of mul​ ti​ ple sub​ nets. a sub​ net is con​ tained within a sin​ gle bridge do​ main. The bridge do​ main de​ fines the unique Layer 2 MAC ad​ dress space and a Layer 2 flood do​ main if flood​ ing is en​ abled. Bridge do​ mains will span all switches in which as​ so​ ci​ ated EPG are con​ fig​ ured.

From a prac​ ti​ cal per​ spec​ tive. . can be trans​ lated lo​ cally as a VLAN with local sig​ nif​ i​ cance. the bridge do​ main Acme-Ap​ pli​ ca​ tions-BD was as​ signed the PI VLAN ID 42 in the Leaf-1. under the Ten​ ant Acme. each Bridge Do​ main will exist in a par​ tic​ u​ lar leaf if there is a con​ nected end​ point that be​ longs to that EPG. You in​ stead should think of it as a dis​ trib​ uted switch. which. Each Bridge Do​ main re​ ceives a VLAN ID in the leaf switches. The VLAN ID used is also called Plat​ form In​ de​ pen​ dent VLAN or PI VLAN. on a leaf. In the fol​ low​ ing ex​ am​ ple. al​ though it can act sim​ i​ lar to a VLAN.182 Tenants End Point Group as part of the Tenant Application Profile It is im​ por​ tant to un​ der​ stand that a bridge do​ main is NOT a VLAN. Each PI VLAN is then linked to a VXLAN ID that will be used for for​ ward​ ing pur​ poses in​ side of the fab​ ric. This VLAN ID is sig​ nif​ i​ cant lo​ cally in the leaf switches and there​ fore it might be dif​ fer from one to other leaf switch. This VLAN con​ cept is dif​ fer​ ent from tra​ di​ tional net​ work​ ing and it is not used to for​ ward traf​ fic but as an iden​ ti​ fier.

it might be re​ quired to have only one en​ cap​ su​ la​ tion VLAN per Bridge do​ main across all the leaf switches and EPGs that ref​ er​ ence that Bridge Do​ main. When a Sub​ net is de​ fined in a Bridge Do​ main. There​ fore in ACI. This VLAN ID is dif​ fer​ ent from the Bridge Do​ main. For this sit​ u​ a​ tion. each leaf will con​ fig​ ure the de​ fault gate​ way. Bridge do​ main in legacy mode is not rec​ om​ mended under ACI best prac​ tices.Tenants 183 VLAN output from Leaf node EPGs are also as​ signed with a PI VLAN ID that is lo​ cally sig​ nif​ i​ cant in each leaf. In some sit​ u​ a​ tions where tra​ di​ tional net​ work​ ing is re​ quired. For more de​ tails refer to the EP sec​ tion in this chap​ ter. If the EPGs have end​ points on mul​ ti​ ple leaves. EPGs con​ tained in bridge do​ mains in legacy mode can​ not com​ mu​ ni​ cate with EPGs in ACI bridge do​ mains. It is thought to use tra​ di​ tional net​ work​ ing within ACI but it lim​ its the ACI fea​ tures that can be used. the de​ fault gate​ way for the end​ points will al​ ways be the first switch of the fab​ ric that is reached. sev​ eral VLANs will be used for EPs in​ side on one bridge do​ main. In that way. the leaf switches will be the de​ fault gate​ way for the EPGs using that sub​ net. for ex​ am​ ple dur​ ing some mi​ gra​ tion processes. This means that an SVI will be con​ fig​ ured under the VRF that rep​ re​ sents the pri​ vate . also know as a per​ va​ sive gate​ way. there is a fea​ ture called Bridge do​ main legacy mode.

mul​ ti​ cast traf​ fic is lim​ ited to the sub​ set of VLAN in​ ter​ faces on which the hosts re​ side. L3 Out for Route Pro​ file .The as​ so​ ci​ ated layer 3 con​ text.The for​ te​ ria (IP or MAC ad​ dress).Op​ ti​ mize/Cus​ tom L2 Un​ known Uni​ cast .184 Tenants net​ work that the Bridge Do​ main is linked to. erty to spec​ ify whether ARP flood​ ing is en​ abled. Every BD by de​ fault takes the fab​ ric wide de​ fault mac ad​ dress. ward​ ing method based on pre​ de​ fined for​ ward​ ing cri​ Uni​ cast Rout​ ing .The name of the Bridge Do​ main. there will be only one SVI per Bridge Do​ main but it will use sec​ ondary IP ad​ dresses. As​ so​ ci​ ated L3 Outs .A prop​ ing is dis​ abled. De​ fault is Proxy. Uni​ cast rout​ ing is en​ abled by de​ fault but can be dis​ abled by click​ ing the check box.The for​ ward​ ing method for un​ known layer 2 des​ ti​ na​ tions. Con​ fig BD MAC Ad​ dress . If flood​ ARP Flood​ ing . ing pol​ icy name. uni​ cast rout​ ing will be per​ formed on the tar​ get IP ad​ dress. For​ ward​ ing . .The MAC ad​ dress of the bridge do​ main (BD) or switched vir​ tual in​ ter​ face (SVI).The IGMP Snoop​ mem​ ber​ ship re​ port mes​ sages from in​ ter​ ested hosts.The node for​ ward​ ing pa​ ra​ me​ ter for un​ known Mul​ ti​ cast des​ ti​ na​ tions.The Layer 3 out​ side in​ ter​ face iden​ ti​ fier con​ trol​ ling con​ nec​ tiv​ ity to out​ side net​ works.The name of the Layer 3 out​ side in​ ter​ face as​ so​ ci​ ated with this ob​ ject. If a Bridge Do​ main has sev​ eral sub​ nets. By ex​ am​ in​ ing (snoop​ ing) IGMP IGMP Snoop Pol​ icy . Bridge Domain Configuration Parameters Name . Net​ work . Flood​ ing is dis​ abled by de​ fault but can be en​ abled by click​ ing the check box. Un​ known Mul​ ti​ cast Flood​ ing .

Choose the Network. It will be con​ fig​ ured in the leaf nodes which have EPGs in that bridge do​ main. • Querier IP . configured.The con​ trol can be spe​ cific pro​ to​ cols ap​ plied to the sub​ net such as IGMP Snoop​ ing. The default is Private.The as​ so​ ci​ ated route pro​ file name. Mon​ i​ tor​ ing Pol​ icy . b.The mon​ it​ or​ ing pol​ icy name for the Ten​ ant se​ man​ tic scope (op​ tional). Choose the Forwarding Semantics (optional). . with the Public option configured. choose Actions > Create Bridge Domain. The net​ work vis​ ib ​il​ ity of the sub​ net. Sub​ net Con​ trol . The scope can be: • Shared .The default is Unspecified. 5 In the Create Bridge Domain dialog box.Defines subnets under a bridge domain. The sub​ net is a por​ tion of a net​ work shar​ ing a par​ tic​ ul​ ar sub​ net ad​ dress.Tenants 185 Route Pro​ file . c. DHCP La​ bels . perform the following actions: a. with the Private option configured. • Private . choose the Tenant_Name. with the Shared option • Public .The net​ work do​ main name label. to share with Layer 3 outbound.Defines subnets under an endpoint group.Defines subnets under a bridge domain. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains. Sub​ nets . Enter an Bridge Domain Name. 2 In the Work pane.The IP ad​ dress and mask of the de​ fault gate​ way. 4 In the Work pane. The con​ trol can be: • Unspecified .Enables IGMP Snooping on the subnet. to only be used in that Tenant (will not be leaked). choose Tenants > ALL TENANTS. Create a new Bridge Domain 1 On the menu bar. to route leak to other Tenants within the Fabric.

Choose the Associated L3 Outs (optional). h. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains > Bridge_Domain_Name. Modify a Bridge Domain 1 On the menu bar. Choose the L3 Out for Route Profile (optional). c. choose the Tenant_Name. e. choose Tenants > ALL TENANTS. i. choose the Policy tab and perform the following actions: a. j. Choose the Forwarding Semantics (optional). Choose the Monitoring Policy if applicable (optional). 4 In the Work pane. Choose the IGMP Snoop Policy (optional). Choose the Associated L3 Outs (optional). 2 In the Work pane. Choose the DNS Label if applicable (optional). choose the Tenant_Name. g. b. Choose the Route Profile (optional). f. f. 3 In the Navigation Pane choose Tenant_Name > Networking > Bridge Domain > Bridge Domain_Name. Remove a Bridge Domain 1 On the menu bar. 6 Click Submit. 5 Click Finish. Choose the Subnets (optional). h. i. Choose the Monitoring Policy if applicable (optional). Choose the Subnets (optional). choose Actions > Delete. Choose the IGMP Snoop Policy (optional). 2 In the Work pane.186 Tenants d. d. g. Choose the DNS Label if applicable (optional). Choose the Route Profile (optional). Choose the L3 Out for Route Profile (optional). choose Tenants > ALL TENANTS. . Choose the Network. e. 4 In the Work pane.

xml CLI :: moquery -c fvBD .Tenants 187 Verify Bridge Domain REST :: /api/node/class/fvBD.

.

This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages: • Ability to use a single private network for all internal and external fabric • No route leaking needed between EPGs in different VRFs • Single Layer 3 Outside can be used by all tenants connectivity Dis​ ad​ van​ tages: • Changes to routing will impact all tenants From a con​ tain​ ment and re​ la​ tion​ ship per​ spec​ tive. 2 In the Work pane.Tenants 189 Tenant Networking Use Cases Common Private Network for All Tenants This use case may be typ​ i​ cal for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate mul​ ti​ ple ten​ ants. choose the common. but place all within a sin​ gle pri​ vate net​ work in the fab​ ric. . this topol​ ogy looks as fol​ lows: Common Private Network for all Tenants To Con​ fig​ ure the com​ mon Ten​ ant pri​ vate net​ work: 1 On the menu bar. choose Tenants > ALL TENANTS.

In the Subnets field. 2 In the Work pane. Choose the appropriate Monitoring Policy if applicable (optional). . click +. e. Choosean End Point Retention Policy Name (optional). Choose network. Now the net​ work ad​ min​ is​ tra​ tor will have to as​ so​ ci​ ate the com​ mon pri​ vate net​ work to the Ten​ ant by first cre​ at​ ing a bridge do​ main. 5 In the Create Bridge Domain dialog box. Enter a Bridge Domain Name. d. For more information on what to select please reference the External Layer 3 section. c. In the Gateway IP field enter the IP address for this subnet. Public and Shared. d. 7 Click Finish. a. Choosea BGP Policy Name (optional). 1 On the menu bar. Note: By default the Private option is selected. perform the following actions: a. choose Tenants > ALL TENANTS. choose Actions > Create Bridge Domain. 4 In the Work pane. e. b. b. Choose an OSPF Policy Name (optional). 4 In the Work pane. f. Choosethe appropriate DNS Label if applicable (optional). choose policy. In the Scope field you have the option to select Private.190 Tenants 3 In the Navigation pane choose common > Networking > Private Networks > default. The Ten​ ant has been cre​ ated. 5 Click Finish. 6 Click OK. choose the Tenant_Name. Choose a Policy Enforcement (optional). c. 3 In the Navigation pane choose Tenant_Name > Networking > Bridge Domains.

1/24' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit # criterion cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs/EPG1/vm-attributescriteria' mocreate 'default' moconfig commit This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API .16.Tenants 191 The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'default' moconfig commit # subnet cd '/aci/tenants/Cisco/networking/bridge-domains/Cisco/subnets' mocreate '172.0.

This par​ tic​ ul​ ar use case demon​ strates how a pri​ vate net​ work can be as​ so​ ci​ ated with each ten​ ant.192 Tenants XML : Ten​ ant Cisco <fvTenant name="Cisco"> <fvBD arpFlood="no" multiDstPktAct="bd-flood" name="Cisco" unicastRoute="yes" unkMacUcastAct="proxy" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="default"/> <fvSubnet ctrl="nd" descr="" ip="172.1/24" preferred="no" scope="private"/> </fvBD> <fvAp name="App1"> <fvAEPg matchT="AtleastOne" name="EPG1"> <fvRsBd tnFvBDName="Cisco"/> </fvAEPg> </fvAp> <fvRsTenantMonPol tnMonEPGPolName=""/> </fvTenant> For many multi-ten​ ant en​ vi​ ron​ ments it is de​ sir​ able to allow each ten​ ant to man​ age and own their own ad​ dress space and not be con​ cerned with over​ laps be​ tween other ten​ ants.0. One Pri​ vate Net​ work per Ten​ ant with In​ tra-EPG com​ mu​ ni​ ca​ tions Ad​ van​ tages: • Allow for maximum isolation between tenants • Ability to address hosts in tenants with overlapping IP addresses Dis​ ad​ van​ tages: • Increased complexity when needing to allow EPG communication between different tenants with dedicated VRF .16.

1 On the menu bar. choose Actions > Create Private Network. perform the following actions: a. 2 In the Create Tenant dialog box. Click Next. b. . In the Create Private Network dialog box. perform the following actions: a. Enter Private Network Name. 4 5 In the Work pane. b. Enter a Tenant Name. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work.Tenants 193 The ob​ ject con​ tain​ ment for this par​ tic​ u​ lar setup can be de​ picted as shown below: Private Network per Tenant To cre​ ate the ten​ ant: 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. 3 Click Finish. choose Tenants > ALL TENANTS. choose the Tenant_Name. 2 In the Work pane. Click Next. choose Tenants > ADD TENANT. The Ten​ ant has been cre​ ated.

0. d.194 Tenants c.16. In the Gateway IP field enter the IP address for this subnet. click +. 6 Click OK. Enter Associated Bridge Domain Name. In the Subnets field. e. Note: By default the Private option is selected.1/24' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit . f. For more information on what to select please reference the External Layer 3 section. Public and Shared. In the Scope field you have the option to select Private. 7 Click Finish. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # subnet cd '/aci/tenants/Cisco/networking/bridge-domains/Cisco/subnets' mocreate '172.

but not at a man​ age​ ment level.0.Tenants 195 This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : Ten​ ant Cisco <fvTenant name="Cisco"> <fvBD arpFlood="no" multiDstPktAct="bd-flood" name="Cisco" unicastRoute="yes" unkMacUcastAct="proxy" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="Cisco"/> <fvSubnet ctrl="nd" ip="172. due to merg​ ers and ac​ qui​ si​ tions or other busi​ ness changes.1/24" name="" preferred="no" scope="private"/> </fvBD> <fvCtx knwMcastAct="permit" name="Cisco" pcEnfPref="enforced"/> <fvAp name="App1" prio="unspecified"> <fvAEPg name="EPG1"> <fvRsBd tnFvBDName="Cisco"/> </fvAEPg> </fvAp> </fvTenant> Multiple Private Networks with Intra-Tenant Communication An​ other use case that may be de​ sir​ able to sup​ port is the op​ tion to have a sin​ gle ten​ ant with mul​ ti​ ple pri​ vate net​ works. This may be a re​ sult of need​ ing to pro​ vide multi-ten​ ancy at a net​ work level. This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages: • Ability to have overlapping subnets within a single tenant Dis​ ad​ van​ tages: • EPGs residing in overlapping subnets cannot have policy applied between one another .16. It may also be caused by need​ ing to sup​ port over​ lap​ ping sub​ nets within a sin​ gle ten​ ant.

perform the following actions: a. choose Tenants > ADD TENANT. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. The Ten​ ant has been cre​ ated. In the Name field enter a name for the bridge domain. Enter a Tenant Name. 2 In the Create Tenant dialog box. 1 On the menu bar. choose Tenants > ALL TENANTS. b.196 Tenants The ob​ ject con​ tain​ ment for this par​ tic​ u​ lar setup can be de​ picted as shown below: Multiple Private Networks with Intra-Tenant communication To cre​ ate the ten​ ant: 1 On the menu bar. 2 In the Work pane. c. choose Actions > Create Private Network. In the Name field enter a name for the Private Network. 3 Click Next. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. . perform the following actions: a. Click Next. choose the Tenant_Name. In the Create Private Network dialog box. 4 5 In the Work pane. 4 Click Finish.

In the Create Application Profile dialog box. In the Subnets field click +. choose Tenants > ALL TENANTS. d. In the Name field enter a name for the Application Profile. 9 10 In the Work pane. c.Tenants 197 d. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles. 4 5 In the Action Tab select Create Application Profile. 6 Click OK. In the Create Private Network dialog box. In the Gateway IP field enter the IP address for this subnet. e. In the Scope field select Shared. In the Name field enter a name for the second Private Network. you can move to the Ap​ pli​ ca​ tion Pro​ file. 6 Click Submit. e. 12 Click Finish. In the Name field enter a name for the second bridge domain. f. 11 Click OK. b. Click Next. In the Gateway IP field enter the IP address for this subnet. 7 Click Finish. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). Now that the two pri​ vate net​ works and bridge do​ mains have been cre​ ated. 2 In the Work pane. perform the following actions: a. . choose the Tenant_Name. 8 In the Navigation pane choose Tenant_Name > Networking > Private Networks. perform the following actions: a. f. 1 On the menu bar. choose Actions > Create Private Network. In the Subnets field click +. In the Scope field select Shared.

4 5 In the Work pane.198 Tenants To cre​ ate the two end​ point groups: 1 On the menu bar. 1 On the menu bar. The ten​ ant ad​ min​ is​ tra​ tor will have to now cre​ ate a con​ tract and fil​ ter be​ tween the two EPGs. 2 In the Work pane. you would have the knowl​ edge of the fil​ ters re​ quired to per​ mit traf​ fic across the two EPGs. you can re​ peat the process of defin​ ing . In the Bridge Domain field. 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Filters. choose Tenants > ALL TENANTS. 7 Click Submit. perform the following actions: a. In the Ethertype select IP. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name. Click +. choose the Tenant_Name. c. 6 Click Finish. d. b. In the IP Protocol column select ICMP. 4 5 In the Work pane. perform the following actions: a. select the appropiate bridge domain. In the fil​ ter. choose Actions > Create Filters. 6 Click Update. In the Name field enter a name for the End Point Group. Re​ peat these steps for the sec​ ond EPG. In the Name field enter a name for the Filter. In the Name column enter ICMP. As the ten​ ant ad​ min​ is​ tra​ tor. choose Tenants > ALL TENANTS. choose the Tenant_Name. choose Actions > Create Application EPG. e. b. In the Create Filter dialog box. In the Create Application EPG dialog box.

choose Actions > Add Provided Contract. choose the Tenant_Name. a. e. In the Action Tab select Add Consume Contract. choose Tenants > ALL TENANTS. A con​ tract is as​ signed to an EPG as ei​ ther a con​ sumed or a pro​ vided con​ tract. Select the Filter you just created. 8 Click Submit. 7 Click OK. f. Now you have to de​ fine the con​ tract that will be con​ sumed and pro​ vided by the two EPGs. 1 On the menu bar. perform the following actions: a. 2 In the Work pane.. 5 In the Add Provided Contract dialog box. d. 4 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Contracts. b. 6 Click Update. c. Click Subjects field click +. In the Name field enter a name for the Subject. Select the created contract. 2 In the Work pane. As​ sign the Con​ tract be​ tween the EPGs. In the Filter Chain field click +. perform the following actions: a. 4 5 In the Work pane. b. In the Create Contract dialog box. In the Scope field select Global. Each EPG that you have cre​ ated will then ei​ ther con​ sume or pro​ vide that con​ tract to es​ tab​ lish a re​ la​ tion​ ship be​ tween both EPGs. choose Tenants > ALL TENANTS. choose Actions > Create Contract. Click Submit. 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contract. 6 7 In the Navigation pane under the second EPG select Contracts.Tenants 199 var​ io ​us dif​ fer​ ent net​ work pro​ to​ cols as re​ quired for your ap​ pli​ ca​ tions. In the Name field enter a name for the Filter. choose the Tenant_Name. Select the created contract .

200 Tenants 8 Click Submit. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : Ten​ ant Cisco # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco1' cd 'Cisco1' moset network 'Cisco1' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco1' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' .

1.shared' moconfig commit This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : Ten​ ant Cisco <fvTenant dn="uni/tn-Cisco" name="Cisco"> <vzBrCP name="ICMP" scope="tenant"> <vzSubj consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" .2.1:24' moset scope 'private.16.1/24' cd '172.1/24' cd '172.2.Tenants 201 moconfig commit # fv-rscon cd '/aci/tenants/Cisco/application-profiles/App1/applicationepgs/EPG1/contracts/consumed-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs/EPG1/subnets' mocreate '172.16.16.1:24' moset scope 'private.16.shared' moconfig commit # application-epg cd '/aci/tenants/Cisco/application-profiles/App1/application-epgs' mocreate 'EPG2' cd 'EPG2' moset bridge-domain 'Cisco1' moconfig commit # fv-rsprov cd '/aci/tenants/Cisco/application-profiles/App/applicationepgs/EPG2/contracts/provided-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/CCO/application-epgs/EPG2/subnets' mocreate '172.1.

1.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD2"/> </fvAEPg> <fvAEPg matchT="AtleastOne" name="App"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-202/pathep-[eth1/2]"/> <fvSubnet ip="172.2.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsProv matchT="AtleastOne" tnVzBrCPName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> Multiple Private Networks with Inter-Tenant Communication .1/24" scope="private.202 Tenants revFltPorts="yes"> <vzRsSubjFiltAtt tnVzFilterName="icmp"/> </vzSubj> </vzBrCP> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvCtx knwMcastAct="permit" name="CiscoCtx2" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx2"/> </fvBD> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="Web"> <fvRsCons tnVzBrCPName="ICMP"/> <fvRsPathAtt encap="vlan-1201" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-201/pathep-[eth1/16]"/> <fvSubnet ip="172.16.1/24" scope="private.16.

In the Name field enter a name for the first Tenant. 3 Click Next. .Tenants 203 Multiple Private Networks with Inter-Tenant Communication This use case may be typ​ i​ cal for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate mul​ ti​ ple ten​ ants with the abil​ ity to sup​ port in​ ter-ten​ ant com​ mu​ ni​ ca​ tions. This method has the fol​ low​ ing ad​ van​ tages and dis​ ad​ van​ tages: Ad​ van​ tages • Each tenant container can be managed separately • Allows for maximum isolation between tenants Dis​ ad​ van​ tages • Tenant address space must be unique From a con​ tain​ ment and re​ la​ tion​ ship per​ spec​ tive. this topol​ ogy looks as fol​ lows: Multiple Private Networks with Inter-Tenant Communication To cre​ ate the ten​ ant: 1 In the GUI Navigate to Tenants > ADD TENANT. 2 In the Create Tenant dialog box perform the following actions: a.

choose Tenants > ALL TENANTS. The ten​ ant has been cre​ ated. 5 In the Create Private Network dialog box. 4 In the Work pane. In the Gateway IP field enter the IP address for this subnet. f. 2 In the Work pane. choose Actions > Create Private Network. Click Next. 6 Click OK. c. choose Actions > Create Application Profile. . 6 Click Submit. In the Name field enter a name for the bridge domain. In the Subnets field click +. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. In the Create Application Profile dialog box. To cre​ ate the ap​ pli​ ca​ tion pro​ file: 1 On the menu bar. 3 In the Navigation Pane choose Tenant_Name > Networking > Private Networks. In the Name field enter a name for the Private Network. perform the following actions: a. 2 In the Work pane. e. choose the Tenant_Name. 3 In the Navigation Pane choose Tenant_Name > Networking > Application Profiles. b. perform the following actions: a. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). 4 5 In the Work pane. d. 1 On the menu bar. choose the Tenant_Name. 7 Click Finish. choose Tenants > ALL TENANTS.204 Tenants 4 Click Finish. In the Name field enter a name for the Application Profile. In the Scope field select Shared.

In the Name field enter a name for the End Point Group. 4 In the Work pane. In the Subnets field click +. 2 In the Work pane. choose Tenants > ALL TENANTS. choose Tenants > ALL TENANTS. In the Name field enter a name for the first Tenant. choose the Tenant_Name. perform the following actions: a. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name 4 5 In the Work pane. d. To cre​ ate the sec​ ond ten​ ant and ap​ pli​ ca​ tion pro​ file: 1 In the GUI Navigate to Tenants > ADD TENANT. In the Create Application EPG dialog box. choose Actions > Create Application EPG. choose the Tenant_Name. choose Actions > Create Private Network. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks. Now the ten​ ant ad​ min​ is​ tra​ tor can cre​ ate the pri​ vate net​ work. 4 Click Finish. In the Name field enter a name for the bridge domain. Click Next. e. . b. 1 On the menu bar. c. 5 In the Create Private Network dialog box. 2 In the Work pane. perform the following actions: a. In the Gateway IP field enter the IP address for this subnet. 2 In the Create Tenant dialog box perform the following actions: a. In the Name field enter a name for the Private Network. In the Bridge Domain field. 3 Click Next. The ten​ ant has been cre​ ated. 6 Click Finish. b. select the appropiate bridge domain.Tenants 205 To cre​ ate the end​ point group: 1 On the menu bar.

choose Tenants > ALL TENANTS. In the Name field enter a name for the Application Profile. 6 Click OK. In the Create Application Profile dialog box. choose Tenants > ALL TENANTS.206 Tenants f. 2 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles > Application Profile_Name 4 5 In the Work pane. 3 In the Navigation pane choose Tenant_Name > Networking > Application Profiles. 1 On the menu bar. select the appropiate bridge domain. The ten​ ant ad​ min​ is​ tra​ tor will have to now cre​ ate a con​ tract and fil​ ter be​ tween the two EPGs. choose Actions > Create Application EPG. choose Actions > Create Application Profile. choose the Tenant_Name. Note: The shared subnet type causes what is known in ACI as a route leak between two Private Networks (VRF). 7 Click Finish. choose Tenants > ALL TENANTS. 2 In the Work pane. 6 Click Finish. 6 Click Submit. 4 5 In the Work pane. b. . perform the following actions: a. To cre​ ate the end​ point group: 1 On the menu bar. In the Scope field select Shared. 2 In the Work pane. In the Create Application EPG dialog box. choose the Tenant_Name. In the Name field enter a name for the End Point Group. In the Bridge Domain field. choose the Tenant_Name. perform the following actions: a. To cre​ ate the ap​ pli​ ca​ tion pro​ file: 1 On the menu bar.

e. d. c. choose Actions > Create Filters. 4 5 In the Work pane. b. b. . In the Filter Chain field click +. In the IP Protocol column select ICMP. choose Tenants > ALL TENANTS. 8 Click Submit. Select the filter you just created. In the Ethertype select IP. As the ten​ ant ad​ min​ is​ tra​ tor you would have the knowl​ edge of the fil​ ters re​ quired to per​ mit traf​ fic across the two EPGs. Click Subjects field click +. In the Scope field choose Global. perform the following actions: a. 4 In the Work pane. 6 Click Update. f. perform the following actions: a. 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Contracts. 2 In the Work pane. 1 On the menu bar. Click +. c. 6 Click Update. In the Name field enter a name for the filter. In the fil​ ter you can re​ peat the process of defin​ ing var​ io ​us dif​ fer​ ent net​ work pro​ to​ cols as re​ quired for your ap​ pli​ ca​ tions. 5 In the Create Filter dialog box. Now you have to de​ fine the con​ tract that will be con​ sumed and pro​ vided by the two EPGs. d. e. In the Name column enter ICMP. In the Create Contract dialog box. choose the Tenant_Name. In the Name field enter a name for the Filter. In the Name field enter a name for the Subject. 7 Click OK. choose Actions > Create Contract.Tenants 207 3 In the Navigation pane choose Tenant_Name > Networking > Security Policies > Filters. 7 Click Submit.

8 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contracts. Click Submit. The con​ fig​ ur​ a​ tion for this use case can be ap​ plied via the fol​ low​ ing CLI con​ fig​ ur​ a​ tion: CLI : TEN​ ANT Cis​ co1 # tenant cd '/aci/tenants' mocreate 'Cisco1' moconfig commit # bridge-domain cd '/aci/tenants/Cisco1/networking/bridge-domains' mocreate 'Cisco1' cd 'Cisco1' moset network 'Cisco1' moconfig commit . 9 In Work pane. 10 Click Submit. Select the created Contract. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application Profile Name > Application EPGs > EPG_Name > Contracts. 7 Select the second created Tenant. b. a. choose Action > Add Consume Contract. 4 In the Work pane. 2 In the Work pane. choose Tenants > ALL TENANTS.208 Tenants As​ sign the Con​ tract be​ tween the EPGs. A con​ tract is as​ signed to an EPG as ei​ ther a con​ sumed or a pro​ vided con​ tract. choose Actions > Add Provided Contract. 6 In the GUI Navigate to Tenants > ALL TENANTS. perform the following actions: a. 1 On the menu bar. Each EPG that you have cre​ ated will then ei​ ther con​ sume or pro​ vide that con​ tract to es​ tab​ lish a re​ la​ tion​ ship be​ tween both EPGs. 5 In the Add Provided Contract dialog box. Select the created Contract.

16.16.Tenants 209 # private-network cd '/aci/tenants/Cisco1/networking/private-networks' mocreate 'Cisco1' moconfig commit # application-profile cd '/aci/tenants/Cisco1/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco1/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco1' moconfig commit # fv-rsprov cd '/aci/tenants/Cisco/application-profiles/CCO/applicationepgs/App/contracts/provided-contracts' mocreate 'ICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco/application-profiles/CCO/application-epgs/App/subnets' mocreate '172.shared' moconfig commit # contract cd '/aci/tenants/Cisco/security-policies/contracts' mocreate 'ICMP' cd 'ICMP' moset scope 'global' moconfig commit # contract-subject cd '/aci/tenants/Cisco/security-policies/contracts/ICMP/subjects' mocreate 'icmp' moconfig commit # vz-rssubjfiltatt cd '/aci/tenants/Cisco/security-policies/contracts/ICMP/subjects/icmp/common-filters' mocreate 'icmp' moconfig commit .1/24' cd '172.1.1:24' moset scope 'private.1.

2.1:24' moset scope 'shared-subnet' moconfig commit # imported-contract cd '/aci/tenants/Cisco1/security-policies/imported-contracts' mocreate 'CiscoInterTenantICMP' cd 'CiscoInterTenantICMP' moset contract 'tenants/Cisco/security-policies/contracts/ICMP' moconfig commit .16.1/24' cd '172.2.210 Tenants CLI : TEN​ ANT Cis​ co2 # tenant cd '/aci/tenants' mocreate 'Cisco' moconfig commit # bridge-domain cd '/aci/tenants/Cisco/networking/bridge-domains' mocreate 'Cisco' cd 'Cisco' moset network 'Cisco' moconfig commit # private-network cd '/aci/tenants/Cisco/networking/private-networks' mocreate 'Cisco' moconfig commit # application-profile cd '/aci/tenants/Cisco/application-profiles' mocreate 'App1' moconfig commit # application-epg cd '/aci/tenants/Cisco2/application-profiles/App1/application-epgs' mocreate 'EPG1' cd 'EPG1' moset bridge-domain 'Cisco' moconfig commit # fv-rsconsif cd '/aci/tenants/Cisco1/application-profiles/CCO/applicationepgs/Web/contracts/consumed-contract-interfaces' mocreate 'CiscoInterTenantICMP' moconfig commit # fv-subnet cd '/aci/tenants/Cisco1/application-profiles/CCO/application-epgs/Web/subnets' mocreate '172.16.

Tenants 211 This con​ fig​ ur​ a​ tion can also be ap​ plied using the fol​ low​ ing XML posted to the APIC REST API: XML : TEN​ ANT Cis​ co1 <fvTenant dn="uni/tn-Cisco1" name="Cisco1"> <vzBrCP name="ICMP" scope="global"> <vzSubj consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" revFltPorts="yes"> <vzRsSubjFiltAtt tnVzFilterName="icmp"/> </vzSubj> </vzBrCP> <vzCPIf dn="uni/tn-Cisco1/cif-ICMP" name="ICMP"> <vzRsIf consMatchT="AtleastOne" name="icmp" provMatchT="AtleastOne" revFltPorts="yes"> <vzRsSubjFiltAtt tDn="uni/tn-Cisco2/brc-default"/> </vzRsIf> </vzCPIf> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx2"/> </fvBD> <fvBD arpFlood="yes" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="EPG1"> .

212 Tenants <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-202/pathep-[eth1/2]"/> <fvSubnet ip="172.16.1.1/24" scope="private.shared"/> <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsProv matchT="AtleastOne" tnVzBrCPName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> XML : TEN​ ANT Cis​ co2 <fvTenant dn="uni/tn-Cisco2" name="Cisco2"> <fvCtx knwMcastAct="permit" name="CiscoCtx" pcEnfPref="enforced"/> <fvBD arpFlood="yes" mac="00:22:BD:F8:19:FF" name="CiscoBD2" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvBD arpFlood="yes" name="CiscoBD" unicastRoute="yes" unkMacUcastAct="flood" unkMcastAct="flood"> <fvRsCtx tnFvCtxName="CiscoCtx"/> </fvBD> <fvAp name="CCO"> <fvAEPg matchT="AtleastOne" name="EPG2"> <fvRsPathAtt encap="vlan-1202" instrImedcy="immediate" mode="native" tDn="topology/pod-1/paths-201/pathep-[eth1/2]"/> <fvSubnet ip="172.shared"/> .16.1/24" scope="private.1.

Tenants 213 <fvRsDomAtt instrImedcy="lazy" resImedcy="lazy" tDn="uni/physPhysDomainforCisco"/> <fvRsBd tnFvBDName="CiscoBD"/> <fvRsConsIf matchT="AtleastOne" tnVzBrCPIfName="ICMP"/> </fvAEPg> </fvAp> </fvTenant> .

.

215 Working with Contracts .

.

Working with Contracts 217 Section Content • Contracts Contract Configuration Parameters Create/Modify/Remove Contracts Create Contracts Modify Contracts Remove Contracts Verify Contracts • Apply/Remove EPG Contracts Apply a Contract to an EPG Remove a Contract from EPG Verify Contract on EPG • Apply/Remove External Network Contracts Apply a Contract to an External Network Remove a Contract from an External Network Verify External Network Contracts • Apply/Remove Private Network Contracts Apply a Contract to a Private Network (vzAny) Remove a Contract From a Private Network (vzAny) Verify Private Network Contracts • Filters Filter Entry Configuration Parameters Create Filters Modify Filters Remove Filters Verify Filters .

or Delete Taboo Contracts Create Taboo Contracts Modify Taboo Contracts Delete Taboo Contracts Verify Taboo Contracts • Apply/Remove Taboo Contracts Apply Taboo Contract to an EPG Remove Taboo Contract from EPG Verify Taboo Contracts Applied to EPG • Inter-Tenant Contracts Configuration Parameters • Create/Modify/Remove Export Contracts Export Contract Modify Exported Contracts Remove Exported Contracts Verify Exported Contracts • Contracts Use Cases Inter-Tenant Contracts Tenant Cisco-1/EPG-1 Tenant Cisco-2/EPG-2 • Inter-Private Network Contracts Communication Tenant Cisco-1/EPG-1 Tenant Cisco-1/EPG-2 . Modify.218 Working with Contracts • Taboo Contracts Taboo Contract Configuration Parameters Create.

Working with Contracts 219 • Single Contract Bidirectional Reverse Filter • Single Contract Unidirectional with Multiple Filters • Multiple Contracts Uni-Directional Single Filter .

.

Con​ tracts are com​ prised of the fol​ low​ ing items: • Subjects . The ex​ am​ ple below. TCP flags and ports) matches based upon filters • Labels .A group of filters for a specific application or service • Filters . . shows how con​ tracts would con​ trol traf​ fic flow be​ tween EPGs in a 3-tiered ap​ pli​ ca​ tion.Used to classify traffic based upon layer 2 to layer 4 attributes (such as • Actions . The Web EPG pro​ vides a con​ tract which is con​ sumed by the Ap​ pli​ ca​ tion EPG. These con​ tracts are built using a provider-con​ sumer model where one EPG pro​ vides the ser​ vices it wants to offer and an​ other EPG con​ sumes them.Permit. There is no con​ tract re​ quired for in​ tra-EPG com​ mu​ ni​ ca​ tion: in​ tra-EPG com​ mu​ ni​ ca​ tion is al​ lowed by de​ fault. mark. and service graph to perform Ethernet type. deny. and the Ap​ pli​ ca​ tion EPG pro​ vides a con​ tract which the Data​ base EPG would con​ sume. EPGs may acts a both a provider and con​ sumer of the same con​ tract eas​ ily by mak​ ing the con​ tract bidi​ rec​ tional.Used optionally to group objects such as subjects and EPGs for the purpose of further defining policy enforcement EPGs can only com​ mu​ ni​ cate with other EPGs based upon the con​ tract rules de​ fined. protocol type. redirect.Working with Contracts 221 Contracts Con​ tracts pro​ vide a way for the ACI ad​ min​ is​ tra​ tor to con​ trol traf​ fic flow within the ACI fab​ ric be​ tween EPGs. log. copy.

. • Application-profile .This contract will be applied for endpoint groups within the same tenant.222 Working with Contracts Contract Policies Between End Point Groups Con​ tracts gov​ ern the fol​ low​ ing types of end​ point group com​ mu​ ni​ ca​ tions: • Between application EPGs • Between application EPGs and external networks • Between application EPGs and in-band management EPG.This contract will be applied for endpoint groups associated with the same private network.This contract will be applied for endpoint groups in the • Tenant . for example if inband management is configured for the ACI fabric and certain EPGs are to be allowed to access it Contract Configuration Parameters When con​ fig​ ur​ ing con​ tracts you can de​ fine the fol​ low​ ing op​ tions: Con​ tract Scope .The scope of a ser​ vice con​ tract be​ tween two or more par​ tic​ i​ pat​ ing peer en​ ti​ ties. same application profile. The states are: • Context .

You can as​ sign the same tag name to mul​ ti​ ple ob​ jects and you can as​ sign one or more tag names to an ob​ ject.Class 1 Differentiated Services Code Point (DSCP) value. QoS Class . choose Tenants > ALL TENANTS. Match . choose the Tenant_Name. The de​ fault is Un​ spec​ i​ fied.This contract will be applied for endpoint groups throughout the fabric. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts.Working with Contracts 223 • Global . 2 In the Work pane.Class 3 DSCP value. Tags . The de​ fault is Con​ text.The search key​ word or term that is as​ signed to the ap​ pli​ ca​ tion pro​ file. • Level2 . The op​ tions are: • AtleastOne • AtmostOne • None • All The de​ fault is Atlea​ st​ O ​ne.Class 2 DSCP value. • Level3 . A tag al​ lows you to group mul​ ti​ ple ob​ jects by a de​ scrip​ tive name. Create/Modify/Remove Contracts Create Contracts 1 On the menu bar. The pri​ or​ ity level can be: • Unspecified • Level1 .The pri​ or​ ity level of the ser​ vice con​ tract.The sub​ ject match cri​ te​ ria across con​ sumers. .

In the Create Contract Subject dialog box. i. perform the following actions: 1. 5 Click Update. c. 7 Click Submit. Enter a Contract Subject Name. perform the following actions: 1. 8 Click Submit. 2. a. 7 Click OK. see the "Filters" section. b. i. 6 Click Update. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. c. Click + next to the Subject to add a Contract Subject. see the "Filters" section. Click + in the Filter Chain field. Choose a QoS Class (optional). Click + next to the Subject field. choose Tenants > ALL TENANTS. Modify Contracts 1 On the menu bar. Click + next to Filter Chain. Note: For information regarding filter creation. 2 In the Work pane. In the Create Contract Subject dialog box. 6 Click OK. Choose a Qos Class (optional). choose Actions > Create Contract. Enter a Contract Subject Name.​ For information regarding filter creation. Choose a Contract Scope (optional). d. perform the following actions: a.224 Working with Contracts 4 5 In the Work pane. choose the Tenant_Name. Choose a Contract Scope (optional). In the Create Contract dialog box. Enter a Contract Name. to add a Contract Subject. 2. b. choose the Policy tab. 4 In the Work pane. Remove Contracts .

2 In the Work pane. choose Actions > Add Provided Contract or Actions > Add Consumed Contract. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts. 5 In the Add Contract dialog box. 4 In the Work pane. Enter a Contract_Name. 2 In the Work pane. Remove a Contract from EPG 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. choose Actions > Delete. perform the following actions: a. . Note: Choose the action depending on how the contract is to be deployed. Verify Contracts REST :: /api/node/class/vzBrCP. 4 In the Work pane.xml CLI :: moquery -c vzBrCP Apply/Remove EPG Contracts Apply a Contract to an EPG 1 On the menu bar. choose the Tenant_Name. choose Tenants > ALL TENANTS. Choose a QOS policy (optional).Working with Contracts 225 Remove Contracts 1 On the menu bar. choose Tenants > ALL TENANTS. choose Tenants > ALL TENANTS. 6 Click Submit. Choose a Label (optional). c. b.

b. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name . Note: Make a selection depending on how the contract is to be deployed. choose Actions > Delete. 5 Click Update. 2 In the Work pane. c.226 Working with Contracts 2 In the Work pane. choose Tenants > ALL TENANTS. 4 In the Work pane. click + next to either Add Provided Contract or Add Consumed Contract.> Application EPGs > EPG_Name > Contracts > Contract_Name. 4 In the Work pane.xml CLI :: moquery -c fvRsProv Consumer ​ REST :: /api/node/class/fvRsCons.xml CLI :: moquery -c fvRsCons Apply/Remove External Network Contracts Apply a Contract to an External Network 1 On the menu bar. choose the Tenant_Name. Choose a Match Criteria. Choose a Contract_Name. Verify Contract on EPG Provider REST :: /api/node/class/fvRsProv. a. . 3 In the Navigation pane choose Tenant_Name > Networking > External Routed Networks > Routed Outside_Name > Networks > External_Network_Instance_Profile. choose the Tenant_Name. Choose a QOS Type.

choose Tenants > ALL TENANTS. rather than to each EPG. It eases con​ tract man​ age​ ment by al​ low​ ing the con​ tract con​ fig​ ur​ a​ tion for all end​ point groups within a pri​ vate net​ work from a sin​ gle lo​ ca​ tion as well as op​ ti​ miz​ ing hard​ ware re​ source con​ sump​ tion. 3 In the Navigation pane choose Tenant_Name > Networking > External Routed Networks > Routed Outside_Name > Networks > External_Network_Instance_Profile.Working with Contracts 227 Remove a Contract from an External Network 1 On the menu bar. con​ tracts can be ap​ plied di​ rectly to the pri​ vate net​ work. This con​ cept is also re​ ferred as "vzAny" end​ point group​ . choose the Tenant_Name. 2 In the Work pane.xml CLI :: moquery -c fvRsCons Apply/Remove Private Network Contracts In order to apply con​ tracts to all end​ point groups within a pri​ vate net​ work. Verify External Network Contracts Provider REST :: /api/node/class/fvRsProv. they can apply the con​ tracts to this one vzAny group under the pri​ vate net​ work. Apply a Contract to a Private Network (vzAny) 1 On the menu bar. 2 In the Work pane. choose the Tenant_Name. 4 In the Work pane. if an ACI Ad​ min​ is​ tra​ tion has 100 EPGs that are all part of the same pri​ vate net​ work. For in​ stance. . choose the Contract_Name and click x. choose Tenants > ALL TENANTS.xml CLI :: moquery -c fvRsProv Consumer REST :: /api/node/class/fvRsCons.

choose the Tenant_Name. Note: Make a selection depending on how the contract is to be deployed. Enter a Contract_Name. Click Update. 2 In the Work pane. 4 In the Work pane. Verify Private Network Contracts REST :: /api/node/class/vzBrCP. choose Tenants > ALL TENANTS. Choose a Match Criteria. c. 4 In the Work pane. Choose a QOS Type. click + next to either Add Provided Contract or Add Consumed Contract. a.228 Working with Contracts 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network_Name > EPG Collection for Context.xml CLI :: moquery -c vzBrCP . Remove a Contract From a Private Network (​ vzAny) 1 On the menu bar. b. 3 In the Navigation pane choose Tenant_Name > Networking > Private Networks > Private_Network_Name > EPG Collection for Context. choose the Contract_Name and click x.

Arp Flag . The Ether​ Types are: • ARP • FCOE • IP • MAC Security • MPLS Unicast • Trill • Unspecified The de​ fault is ARP. an EPG provider dic​ tates the pro​ to​ cols and ports in both the in and out di​ rec​ tions. Each fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. Filter Entry Configuration Parameters When con​ fig​ ur​ ing a Fil​ ter. the fol​ low​ ing op​ tions can be de​ fined: Name . The fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties.The Ether​ Type of the fil​ ter entry. Fil​ ters are Layer 2 to Layer 4 fields. The fil​ ter entry is a com​ bi​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties.The IP pro​ to​ col for a fil​ ter entry. Ac​ cord​ ing to its re​ lated con​ tract. See the Use Cases below for an ex​ am​ ple.Working with Contracts 229 Filters A fil​ ter pol​ icy is a group of re​ solv​ able fil​ ter en​ tries. TCP/IP header fields such as Layer 3 pro​ to​ col type. IP Pro​ to​ col .The name of a fil​ ter entry. Ether​ Type . and so on. Con​ tract sub​ jects con​ tain as​ so​ ci​ a​ tions to the fil​ ters (and their di​ rec​ tions) that are ap​ plied be​ tween EPGs that pro​ duce and con​ sume the con​ tract. . Layer 4 ports.The Ad​ dress Res​ ol​ u​ tion Pro​ to​ col flag for a fil​ ter entry.

The port can be for the fol​ low​ ing server types: • Unspecified • ftpData • smtp • DNS • HTTP • POP3 • HTTPS • RTSP ti​ na​ tion port range.The start of the source port range.The end of the des​ range is de​ ter​ mined by the server type. The range is 0 to 0xffff. The start of the port range is de​ ter​ mined by the server type. The range is 0 to 0xffff. The port state set​ tings are: • Unspecified • ftpData .The start of the source port range. The end of the port Des​ ti​ na​ tion Port: From . The port can be for the fol​ low​ ing server types: • Unspecified • ftpData • SMTP • DNS • HTTP • POP3 • HTTPS • RTSP Source Port: To . The port can be for the fol​ low​ ing server types: Source Port: From . The start of the port range is de​ ter​ mined by the server type. The end of the port range is de​ ter​ mined by the server type.230 Working with Contracts Allow Frag​ ment .The end of the source port range. The range is 0 to 0xffff. The range is 0 to 0xffff.

The fil​ ter entry is a com​ bi​ TCP Ses​ sion Rules . 5 In the Create Filter dialog box.The start of the des​ ti​ na​ tion port range. Create Filters 1 On the menu bar. Enter an Filter Name. perform the following actions: a. choose Tenants > ALL TENANTS.The TCP ses​ na​ tion of net​ work traf​ fic clas​ si​ fi​ ca​ tion prop​ er​ ties. The port range is de​ ter​ mined by the server type. choose Actions > Create Filter. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters. 4 In the Work pane.Working with Contracts 231 • SMTP • DNS • HTTP • POP3 • HTTPS • RTSP Des​ ti​ na​ tion Port: To . sion rules for a fil​ ter entry. . b. choose the Tenant_Name. Click + to add a Filter Entry. The des​ ti​ na​ tion port can be set for the fol​ low​ ing server types: • unspecified • ftpData • smtp • dns • http • pop3 • https • rtsp The de​ fault is un​ spec​ i​ fied. The range is 0 to 0xffff. 2 In the Work pane.

2. Enter a Filter Entry Name. In the drop-down list select an IP Protocol (optional). 7 Click Submit. b. 3. choose Tenants > ALL TENANTS. i. 6. Modify Filters 1 On the menu bar. d. 7. In the drop-down list select the Source Port To (optional). click the Allow Fragment check box (optional). 10. In the drop-down list select an Ethertype. If required. 6 Click Submit. 5. g. In the drop-down list select the TCP Session Rules (optional). In the drop-down list select the Source Port From (optional). 4 In the Work pane. In the drop-down list select the Destination Port To (optional).232 Working with Contracts i. In the drop-down list select an ARP Flag (optional). In the Filter Entry dialog box. In the drop-down list select the Destination Port From (optional). In the drop-down list select an ARP Flag (optional). h. perform the following actions: 1. 9. In the drop-down list select an Ethertype (optional). 8. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters > Filter_Name. In the drop-down list select the Destination Port From (optional). c. In the drop-down list select the Source Port From (optional). In the drop-down list select the Destination Port To (optional). In the drop-down list select the Source Port To (optional). 6 Click Update. 2 In the Work pane. e. If required. 5 Click Update. f. double click the filter entry. In the drop-down list select an IP Protocol (optional). . check the Allow Fragment check box (optional). 4. In the drop-down list select the TCP Session Rules (optional). a.

choose Actions > Delete. choose Tenants > ALL TENANTS. 3 In the Navigation pane choose Tenant_Name > Security Policies > Filters > Filter_Name.xml CLI :: moquery -c vzFilter .Working with Contracts 233 Remove Filters 1 On the menu bar. choose the Tenant_Name. 2 In the Work pane. 4 In the Work pane. Verify Filters REST :: /api/node/class/vzFilter.

.

match​ ing a fil​ ter. La​ bels en​ able clas​ si​ fi​ ca​ tion of the ob​ jects which can and can​ not com​ mu​ ni​ cate with one an​ other (op​ tional).The fil​ ter di​ rec​ tives as​ signed to the taboo con​ tract. Taboo rules are ap​ plied in the hard​ ware be​ fore the rules of reg​ ul​ ar con​ tracts are ap​ plied. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts. or Delete Taboo Contracts Create Taboo Contracts 1 On the menu bar. Taboo Contract Configuration Parameters When con​ fig​ ur​ ing Taboo Con​ tracts you can de​ fine the fol​ low​ ing op​ tions: Name . To im​ it​ ate the tra​ di​ tional net​ work​ ing con​ cepts. an "al​ low-all-traf​ fic" con​ tract can be ap​ plied. Create.Working with Contracts 235 Taboo Contracts There may be times when the ACI ad​ min​ is​ tra​ tor might need to deny traf​ fic that is al​ lowed by an​ other con​ tract. Di​ rec​ tive . choose Tenants > ALL TENANTS.The net​ work do​ main name label.The name of the con​ tract or con​ tract ob​ ject. Sub​ jects . choose the Tenant_Name. 2 In the Work pane. Modify. and so forth). a spe​ cific EPG. . with taboo con​ tracts con​ fig​ ured to re​ strict cer​ tain types of traf​ fic. Taboos can be used to drop traf​ fic match​ ing a pat​ tern (any EPG. Taboo con​ tracts are not rec​ om​ mended as part of the ACI best prac​ tices but they can be used to tran​ si​ tion from tra​ di​ tional net​ work​ ing to ACI. Taboos are a spe​ cial type of con​ tract that an ACI ad​ min​ is​ tra​ tor can use to deny spe​ cific traf​ fic that would oth​ er​ wise be al​ lowed by an​ other con​ tract.

Delete Taboo Contracts 1 On the menu bar. Choose Directives. 2 In the Work pane. ii. 1. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts > Taboo_Contract_Name. ii. i. perform the following actions: i. choose Tenants > ALL TENANTS. Click + in the Filter Chain field. Click + to next to the Subject field to add a Taboo Subject. Enter a Taboo Contract Subject Name. b. 2. 3 In the Navigation pane choose Tenant_Name > Security Policies > Taboo Contracts > Taboo_Contract_Name. 6 Click Update. perform the following actions: a. Enter a Filter Name. 8 Click Submit. choose the Tenant_Name. choose policy.236 Working with Contracts 4 5 In the Work pane. choose Tenants > ALL TENANTS. choose Action > Create Taboo Contract. 2 In the Work pane. 5 Click Submit. 4 In the Work pane. . Click + to next to the Subject field. In the Create Taboo Contract Subject dialog box. b. 7 Click OK. Choose Directives. In the Create Taboo Contract dialog box. Modify Taboo Contracts 1 On the menu bar. a. Enter a Filter Name. Enter a Taboo Contract Name.

6 Click Submit. . 2 In the Work pane. a. choose Actions > Add Taboo Contract. Choose the Taboo Contract. 4 In the Work pane.Working with Contracts 237 4 In the Work pane. 5 In the Add Taboo Contract dialog box. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts. 2 In the Work pane. Remove Taboo Contract from EPG 1 On the menu bar. choose Tenants > ALL TENANTS. choose Action > Delete. Verify Taboo Contracts REST :: /api/node/class/vzTaboo. choose the Taboo Contract_Name > Actions > Delete.xml CLI :: moquery -c vzTaboo Apply/Remove Taboo Contracts Apply Taboo Contract to an EPG 1 On the menu bar. 4 In the Work pane. choose the Tenant_Name. choose the Tenant_Name. 3 In the Navigation pane choose Tenant_Name > Application Profiles > Application_Profile_Name > Application EPGs > EPG_Name > Contracts. choose Tenants > ALL TENANTS.

238 Working with Contracts Verify Taboo Contracts Applied to EPG Provider REST :: /api/node/class/fvRsProv.xml CLI :: moquery -c fvRsCons .xml CLI :: moquery -c fvRsProv C ​onsumer REST :: /api/node/class/fvRsCons.

Choose the Global Contract. In​ ter​ face con​ tracts are a spe​ cial type of con​ tract that an ACI ad​ min​ is​ tra​ tor can use to allow spe​ cific traf​ fic through the use of a con​ tract ex​ port. How​ ever. the source EPG will be of type provider.Working with Contracts 239 Inter-Tenant Contracts There may be times when the ACI ad​ min​ is​ tra​ tor might need to allow traf​ fic be​ tween two ten​ ants. b. Ten​ ant . . The con​ tract in essence is ex​ ported in the source ten​ ant and im​ ported into the tar​ get ten​ ant. Enter an Export Contract Name. Sim​ il​ ar to tra​ di​ tional con​ tracts. Name .The Ten​ ant name of the tar​ geted Ex​ port con​ tract.Name of a ser​ vice con​ tract to be shared be​ tween two or more par​ tic​ ip ​at​ ing peer en​ ti​ ties. choose Tenants > ALL TENANTS. Configuration Parameters When Im​ port​ ing a Con​ tract. choose the Tenant_Name. 4 In the Work pane.The name of the con​ Global Con​ tract . perform the following actions: a. 2 In the Work pane. in the tar​ get ten​ ant. Some use case ex​ am​ ples show the com​ plete process in the next chap​ ter. Create/Modify/Remove Export Contracts Export Contract 1 On the menu bar. the fol​ low​ ing op​ tions can be de​ fined: tract in​ ter​ face. 5 In the Export Contract dialog box. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts. choose Actions > Export Contract. the con​ tract is im​ ported as type con​ tract in​ ter​ face.

6 Click Finish.xml CLI :: moquery -c vzCPif . Choose the Global Contract. choose Actions > Delete. choose Tenants > ALL TENANTS. b.240 Working with Contracts c. a. 4 In the Work pane. Enter the Tenant Name. choose policy. Remove Exported Contracts 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Imported Contracts > Contact_Name. 2 In the Work pane. choose the Tenant_Name. 4 In the Work pane. Verify Exported Contracts REST :: /api/node/class/vzCPif. choose Tenants > ALL TENANTS. 5 Click Finish. c. Modify Exported Contracts 1 On the menu bar. 3 In the Navigation pane choose Tenant_Name > Security Policies > Contracts > Contract_Name. choose the Tenant_Name. Enter an Export Contract Name. Enter the Tenant Name. 2 In the Work pane.

To use the same con​ tract.. tract In​ ter​ face will be used to as​ so​ ci​ ate an EPG from the des​ ti​ na​ tion A Con​ sumed Con​ ten​ ant with the im​ ported con​ tract. as with most com​ pa​ nies. Inter-Tenant Contracts ACME Inc. Com​ mu​ ni​ ca​ tion be​ tween EPGs that be​ long to dif​ fer​ ent ten​ ants is only al​ lowed when they share the same con​ tract. 4 Single Contract Unidirectional with multiple Filters. There are four com​ mon sce​ nar​ ios: 1 Inter-Tenant Contracts. Re​ view the con​ tracts sec​ tion on Con​ tract Scop​ ing for a more de​ tailed dis​ cus​ sion. In ten​ ant Cisco-1 .Working with Contracts 241 Contracts Use Cases These use cases all as​ sume the ob​ jec​ tive is for a host in EPG-1 to talk to a host in EPG2. EPG-1 in ten​ ant Cisco-1 re​ quires com​ mu​ ni​ ca​ tion with EPG-2 in ten​ ant Cisco-2. In the use case below. This is ac​ com​ plished by uti​ liz​ ing con​ tact in​ ter​ faces. and whether the sys​ tem is more fo​ cused on ob​ ject reuse or ten​ ant au​ ton​ omy. These ser​ vices will be used across most of their ten​ ants and so ACME Inc. How these sce​ nar​ ios are im​ ple​ mented will de​ pend on the op​ er​ at​ ional model cho​ sen. an end​ point group starts con​ sum​ ing all the sub​ jects rep​ re​ sented by the in​ ter​ face. must allow this traf​ fic across the whole fab​ ric. 5 Multiple Contracts Unidirectional with single Filter. That con​ tract will ap​ pear under the Im​ ported Con​ tract sec​ tion in the Se​ cu​ rity Poli​ cies of the des​ ti​ na​ tion ten​ ant. 3 Single Contract Bidirectional forwarding with reverse filter. makes use of shared ser​ vices such as DNS for name res​ o​ lu​ tion and Ac​ tive Di​ rec​ tory for user man​ age​ ment. achiev​ ing bidi​ rec​ tional traf​ fic. Note: A con​ tract con​ sump​ tion in​ ter​ face rep​ re​ sents one or more sub​ jects de​ fined under the con​ tract. it will need to be ex​ ported from the source ten​ ant to the ap​ pro​ pri​ ate des​ ti​ na​ tion ten​ ant. 2 Inter-Private Network Contracts. By as​ so​ ci​ at​ ing to an in​ ter​ face.

Tenant Cisco-2/EPG-2 1 Confirm the exported contract is listed under Imported Contracts.242 Working with Contracts the user will ex​ port the in​ tended con​ tract in​ ter​ faces. 3 Add the Contract under EPG1 .subnet scope shared.subnet scope shared. 2 Create the host subnet (default Gateway IP) under EPG1 .subnet scope private/public. The user will then con​ firm the im​ ported con​ tract in ten​ ant Cisco-2 and se​ lect the con​ tract as con​ sumed. 4 Create the host subnet under the bridge domain . 3 Add the Interface Contract under EPG2 . . To ad​ ver​ tise the routes from the source VRF to the in​ tended VRF. 4 Create the host subnet (default Gateway IP) under the bridge domain .subnet scope private/public. 2 Create the host subnet (default Gateway IP) under EPG2 . In ten​ ant Cisco-1 the user will ex​ port the in​ tended con​ tract and se​ lect provider to pro​ vide the con​ trast to EPG-2.contract type provider.contract type consumed. the user must cre​ ate the sub​ net within the EPG. Exporting Contracts Between Tenants Tenant Cisco-1/EPG-1 1 Create an Export Contract under security policies.

Single Contract Bidirectional Reverse Filter This use case is use​ ful when im​ ple​ ment​ ing a con​ tract with the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions and with the op​ tion to apply the re​ verse fil​ ter. This is ac​ com​ plished by uti​ liz​ ing the sub​ net field within the EPG. EPG-1 in VRF Cisco-1 re​ quires com​ mu​ ni​ ca​ tion with EPG-2 in VRF Cisco-2. This is . Tenant Cisco-1/EPG-1 1 Create the host subnet (default Gateway IP) under EPG1 . 2 Add the Contract under EPG1 .subnet scope shared.contract scope Tenant.contract type provider. Tenant Cisco-1/EPG-2 1 Create the host subnet (default Gateway IP) under EPG2 . the route will be leaked to the VRF noted within the Ten​ ant scoped con​ tract.contract type provider. By cre​ at​ ing the sub​ net under the EPG and se​ lect​ ing shared.subnet scope shared.Working with Contracts 243 Inter-Private Network Contracts Communication In the use case below. Exporting Contracts Between Private Networks 1 Create the contract under Security Policies . 2 Add the Contract under EPG2 .

e. EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of www and EPG-2 is con​ sum​ ing the con​ tract. www. i. Sample Contract using Single Bi-directional Subject. EPG-1 is pro​ vid​ ing a ser​ vice to EPG-2. This al​ lows the Web Client in EPG-2 to ac​ cess the Web Server in EPG-1. Default Bi-directional Contract with Reverse Filter Re​ sult: A sin​ gle con​ tract with (1) Sub​ ject and (1) Fil​ ter with a sin​ gle provider and a sin​ gle con​ sumer. In the use case below.244 Working with Contracts the most com​ mon of the use cases and al​ lows for a sin​ gle sub​ ject/fil​ ter to be im​ ple​ mented with a sin​ gle Provider/Con​ sumer re​ la​ tion​ ship. In this ex​ am​ ple. Single Filter with Reverse Filtering Ports Enabled Single Contract Unidirectional with Multiple Filters .

EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of icmp and EPG-2 is con​ sum​ ing the con​ tract. When uti​ liz​ ing a sin​ gle sub​ ject with​ out the use of “Apply Both Di​ rec​ tions. Sample Single Contract.” the user must then con​ fig​ ure two fil​ ters. icmp. Single Unidirectional Subject.Working with Contracts 245 Single Contract Unidirectional with Multiple Filters This use case in​ volves im​ ple​ ment​ ing a con​ tract with​ out the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions. Single Unidirectional Subject. Single Contract. When se​ lect​ ing this op​ tion the user no longer has the op​ tion to se​ lect the re​ verse fil​ ter op​ tion. In this ex​ am​ ple. In the use case below. This al​ lows the Host in EPG-1 to ac​ cess the Host in EPG-2 via icmp. one in each di​ rec​ tion. Multiple Filters Re​ sult: A sin​ gle con​ tract with (1) Sub​ ject (2) Fil​ ters and a sin​ gle provider and a sin​ gle con​ sumer. Multiple Filters .

EPG-1 is pro​ vid​ ing a ser​ vice to EPG-2. Each con​ tract will have a sin​ gle provider and a sin​ gle con​ sumer ref​ er​ enc​ ing the same con​ tract. That is. Single Filters Re​ sult: Two con​ tracts with (1) Sub​ ject (1) Fil​ ters. . The dif​ fer​ ence here is that the con​ tract is ex​ plic​ itly ap​ plied in BOTH di​ rec​ tions. but is also the most com​ pre​ hen​ sive. This al​ lows the Web Client in EPG-2 to ac​ cess the Web Server in EPG-1. EPG-1 is pro​ vid​ ing a con​ tract with a sub​ ject of www and EPG-2 is con​ sum​ ing the con​ tract. This al​ lows the end-user the most gran​ u​ lar​ ity when de​ ploy​ ing con​ tracts. and with​ out the op​ tion to apply the re​ verse fil​ ter. Unidirectional Subjects. In the use case below. Multiple Contracts.246 Working with Contracts Multiple Contracts Uni-Directional Single Filter This use case is use​ ful when im​ ple​ ment​ ing a con​ tract with the op​ tion to apply the con​ tract sub​ ject in both di​ rec​ tions.

Working with Contracts 247 Sample Contract with Single Bi-Directional Subjects and single Filter .

.

249 Layer 4 to Layer 7 Services .

.

Layer 4 to Layer 7 Services 251 Section Content • Understanding Layer 4 to Layer 7 Integration Service Insertion Design Principles Applying Service Graphs to EPG Communications Rendering Service Graphs Integration Support Function • Services Deployment Guide Reference Import Device Package Create a Device Modify a Device Create Layer 4 to Layer 7 Service Graph Template Apply a Service Graph Template to EPGs • Service Graph Monitoring Monitoring a Service Graph Instance Monitoring and Resolving Service Graph Faults Monitoring a Virtual Device • ASAv Sample Configuration Verify ASAv VM configuration Remove GW IP from Relevant Fabric Bridge Domains Create a Layer 4 to Layer 7 Device Create a Layer 4 to Layer 7 Service Graph Template Apply the Service Graph Template Associate the Web and App Bridge domains to the Interfaces Verifying service node configuration Display the Access List configuration pushed-down from APIC Display the Interface configuration pushed-down from APIC Display the current Interface names .

.

load bal​ ancers. which can cause change man​ age​ ment to be​ come chal​ leng​ ing. Use​ ful in​ fra​ struc​ ture im​ ple​ men​ ta​ tions also uti​ lize this fast fab​ ric in a smart way to also in​ te​ grate Layer 4 to Layer 7 ser​ vices.Layer 4 to Layer 7 Services 253 Understanding Layer 4 to Layer 7 Integration When ACME Inc. de​ signed their new ap​ pli​ ca​ tion. fire​ walls might be placed di​ rectly in​ line as a "bump in the line" or might be an ad​ junct ser​ vice de​ vice near a gate​ way. ser​ vice in​ ser​ tion would re​ quire some in​ line de​ vice place​ ment or redi​ rec​ tion in order to get traf​ fic to the ser​ vice de​ vices. As an ex​ am​ ple. Fire​ walls are typ​ ic ​ally con​ fig​ ured per de​ vice by build​ ing up blocks of sta​ tic con​ fig​ ur​ a​ tion. One of the key tech​ nol​ ogy in​ no​ va​ tions in Cisco's Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) is pol​ icy-based man​ age​ ment of ser​ vice in​ ser​ tion through ap​ pli​ ca​ tion of a Ser​ vice Graph. such as fire​ walls. The main pur​ pose of data cen​ ter fab​ ric equip​ ment is fast and ef​ fi​ cient for​ ward​ ing of traf​ fic from ingress to the fab​ ric. they knew they would need to in​ cor​ po​ rate some ser​ vices. we will dis​ cuss the high level overview of how the process works and how ACME will uti​ lize Ser​ vice Graphs for ef​ fi​ cient man​ age​ ment of Layer 4 to Layer 7 ser​ vices. but can be​ come in​ flex​ ib ​le and frag​ ile. These sta​ tic con​ fig​ ur​ a​ tion blocks build up over time to cre​ ate a sit​ ua ​t​ ion where the con​ fig​ ur​ a​ tion works. In tra​ di​ tional in​ fra​ struc​ ture. be​ tween phys​ ic ​al and vir​ tual hosts within the fab​ ric and egress back out of the fab​ ric. IDS/IPS. Some pos​ si​ ble Layer 4 to Layer 7 ser​ vices in​ clude: • Firewalls • Load balancers • Traffic inspection appliances • SSL offload functions • Application flow acceleration functions In​ te​ grat​ ing ser​ vices with Cisco ACI Ser​ vice Graphs will pro​ vide ACME with the fol​ low​ ing ben​ e​ fits: • Policy based configuration management . Through the rest of this chap​ ter. or other types of higher-layer ser​ vice de​ vice.

al​ low​ ing multi-ten​ ant map​ ping of fab​ ric for​ ward​ ing and ten​ ant-spe​ cific ser​ vice con​ fig​ ur​ a​ tions. and can be ex​ tended to in​ clude ser​ vice in​ ser​ tion via at​ tach​ ment of a Ser​ vice Graph to a con​ tract.254 Layer 4 to Layer 7 Services • Flexible configuration state abstraction through the ACI object model • Integrated configuration management using the APIC GUI. cur​ rently only VMware Hy​ per​ vi​ sors and VLAN trans​ port modes are sup​ ported. the ap​ pli​ ances do not need to be placed at any spe​ cific lo​ ca​ tion within the net​ work fab​ ric. a con​ tract must be put in to place. This con​ tract es​ sen​ tially con​ trols com​ mu​ ni​ ca​ tions flow be​ tween EPGs. With vir​ tual ser​ vices ap​ pli​ ances. Where ap​ plic​ ab ​le. Ser​ vices ap​ pli​ ances can be phys​ ic ​al or vir​ tual and can be con​ nected to any leaf under man​ age​ ment of the ACI fab​ ric. Applying Service Graphs to EPG Communications To allow com​ mu​ ni​ ca​ tions be​ tween EPGs within an ACI fab​ ric. The Ser​ vice Graph then ties the con​ tract to the re​ solved ser​ vice de​ vice with the pol​ icy-based con​ fig​ ur​ a​ tion in place. This con​ tract may take the form of a spe​ cific con​ sumer/provider re​ la​ tion​ ship de​ fined by spec​ if​ ied pro​ to​ col and port. and thus. REST API or Python scripts. phys​ ic ​al ap​ pli​ ances can also be run in mul​ ti​ ple con​ text mode. traf​ fic flow to/from ser​ vice ap​ pli​ ances is man​ aged very ef​ fi​ ciently. all based on a consistent ACI object model • Complex topology modeling with logical flow stitching allowing abstracted links • Policy-based provisioning allowing rapid complex topology deployment • Configuration synchronization allowing dynamic workload provisioning and de- between multiple service devices provisioning without manual intervention • Application centric template-based configuration management and object • Infrastructure multi-tenancy within the fabric and the service devices reuse to shorten infrastructure implementation timelines Service Insertion Design Principles With the spine-leaf ar​ chi​ tec​ ture and holis​ tic fab​ ric man​ age​ ment as​ pects of ACI. There could also be an "allow all" con​ tract that al​ lows com​ pletely open com​ mu​ ni​ ca​ tions. .

such as Python or a RESTful API post through POSTman These pol​ icy ob​ jects can be cre​ ated. ports. con​ fig​ ur​ a​ tion of IP ad​ dresses on the fire​ wall and load bal​ ancer in​ ter​ faces. and other val​ ues. the ser​ vice graph ob​ jects are then trans​ lated into spe​ cific de​ vice con​ fig​ ur​ a​ tions that gets pushed the ser​ vice nodes through a process called ren​ der​ ing. When an ap​ pli​ ca​ tion pro​ file is de​ ployed and end​ points are at​ tached to the leaf switches. ACME has a few cookie cut​ ter tem​ plates for fire​ wall and load-bal​ anc​ ing ser​ vices. cre​ ation of the VLAN on these de​ vices to cre​ ate the path for the func​ tions. When the de​ vice pack​ age is im​ ported. ob​ ject-groups. and per​ for​ mance of all the work nec​ es​ sary to make sure that the path be​ tween EPGs is the path de​ fined in the ser​ vice graph. com​ mu​ ni​ ca​ tions with the ser​ vice de​ vices is sup​ ported by im​ port​ ing de​ vice pack​ ages. This pol​ icy can be copied. ex​ posed func​ tions. reused and repack​ aged as nec​ es​ sary. As it re​ lates to the Layer 4 to Layer 7 ser​ vices.Layer 4 to Layer 7 Services 255 Rendering Service Graphs The ACI al​ lows users to de​ fine a pol​ icy using the fol​ low​ ing ways: • APIC GUI • API with a programmatic tool. Though the ini​ tial de​ f​ in ​i​ t​ ion of these tem​ plates can be po​ ten​ tially cum​ ber​ some. the ACI fab​ ric has full un​ der​ stand​ ing of what a de​ vice can do. how it con​ nects to the fab​ ric. and reused. These de​ vice pack​ ages carry the de​ vice de​ scrip​ tion. ma​ nip​ ul​ ated. sub​ se​ quently reusing the tem​ plates is very straight​ for​ ward sim​ ply by re​ plac​ ing IP ad​ dresses. This ab​ stracted process of con​ fig​ ur​ a​ tion man​ age​ ment works like a pol​ icy tem​ plate where you can de​ fine the ex​ pected be​ hav​ ior. . these ob​ jects ex​ press the in​ tent of use for that ob​ ject in re​ la​ tion to the ap​ pli​ ca​ tion. then link two groups and sub​ ject their re​ la​ tion​ ship to that pol​ icy. As is the case with many cus​ tomers. The ren​ der​ ing in​ volves al​ lo​ ca​ tion of the nec​ es​ sary bridge do​ mains. The APIC also sets up the net​ work for​ ward​ ing path to make sure the cor​ rect for​ ward​ ing ac​ tion is taken to get traf​ fic flow to the ser​ vice nodes for treat​ ment. and have con​ fig​ ur​ a​ tion script con​ tent. Integration Support Function In the ACI model.

​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ unified-fabric/​ aci_​ ecosystem. An up to date list​ ing of part​ ners that lever​ age the API are avail​ able at: http://​ www.​ html . Some of the De​ vice Pack​ ages can be down​ loaded at: http://​ www.​ cisco. and how to trans​ late pol​ icy in​ tent to a de​ vice-spe​ cific con​ fig​ ur​ a​ tion.​ html De​ vice pack​ ages from mul​ ti​ ple ven​ dors that lever​ age the rich open API that ACI pro​ vides are avail​ able from ven​ dors such as Cit​ rix and F5 at the time of writ​ ing of this book. This de​ vice pack​ age is a ven​ dor-de​ vel​ oped pack​ age that is read​ ily avail​ able from the orig​ in ​al ven​ dor or from Cisco on the soft​ ware down​ load page.256 Layer 4 to Layer 7 Services how to build path for​ ward​ ing to bring traf​ fic into and get traf​ fic back from the de​ vice.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centricinfrastructure/​ solution-overview-c22-732445.

L4-L7 Workflow For in​ for​ ma​ tion about de​ ploy​ ing Layer 4 to Layer 7 ser​ vices. the de​ vices will be added through a process of cre​ at​ ing a log​ i​ cal de​ vice clus​ ter and cre​ at​ ing a re​ la​ tion​ ship be​ tween this log​ i​ cal de​ vice and the phys​ i​ cal ap​ pli​ ance. This can be done with a phys​ i​ cal or VM de​ vice. as shown in this sec​ tion of the De​ ploy​ ment Guide. Create a Device Once the de​ vice pack​ age is im​ ported. but are very sim​ i​ lar. . see the Cisco APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide: http://​ www. ACME will im​ port the ven​ dor/model-spe​ cific de​ vice pack​ ages.​ html.​ cisco.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy. The key sec​ tions of the De​ ploy​ ment Guide are listed below: Import Device Package To begin the process of Ser​ vice Node in​ te​ gra​ tion.Layer 4 to Layer 7 Services 257 Services Deployment Guide Reference The di​ a​ gram below shows a high level overview of the Layer 4 to Layer 7 ser​ vices work​ flow when at​ tempt​ ing to in​ te​ grate a de​ vice. The con​ fig​ u​ ra​ tion steps dif​ fer slightly for phys​ i​ cal de​ vices and vir​ tual de​ vices.

Create Layer 4 to Layer 7 Service Graph Template This sec​ tion of the De​ ploy​ ment Guide ex​ plains how to cre​ ate a ser​ vice graph. the process to apply a ser​ vice graph tem​ plate to EPGs can be found in this sec​ tion of the De​ ploy​ ment Guide.258 Layer 4 to Layer 7 Services Modify a Device You can mod​ ify a de​ vice's con​ fig​ ur​ a​ tion through the GUI as de​ scribed in this sec​ tion of the De​ ploy​ ment Guide. . Apply a Service Graph Template to EPGs Once the ap​ pli​ ca​ tion End​ point Groups (EPGs) have been cre​ ated.

To sup​ port these ef​ forts. and the current state of the graph policy. Func​ tions of a Graph in​ stance. re​ sources al​ lo​ cated to a func​ tion and pa​ ra​ me​ ters spec​ if​ ied for a func​ tion. there are a few tech​ niques that can be em​ ployed.​ com/​ c/​ en/​ us/​ td/​ Layer 4 to Layer 7 Ser​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy/​ b_​ L4L7_​ Deploy_​ chapter_​ 01010. A state of "Applied" means the graph has been applied and is active in the fabric and the service device. including a list of the deployed service graphs. in​ clud​ ing: • Monitoring a Service Graph Instance • Monitoring and Resolving Service Graph Faults • Monitoring a Virtual Device Monitoring a Service Graph Instance Once a ser​ vice graph is con​ fig​ ured and as​ so​ ci​ ated with a con​ tract that is at​ tached to an EPG.​ html#​ task_​ F2BFF7545D9142EFB208C10F5DFBB1B4 . the associated contracts. has de​ ployed Ser​ vice Graphs for ap​ pli​ ca​ tion ser​ vice in​ ser​ tion. choose Tenants > ALL TENANTS. choose the tenant. the re​ quire​ ment of ob​ serv​ ing the Ser​ vice Graph be​ comes an op​ er​ at​ ional im​ per​ at​ ive. The Work pane displays information about the deployed graph instances. To mon​ it​ or a ser​ vice graph in​ stance: 1 On the menu bar. choose Tenant_Name > L4-L7 Services > Deployed Graph Instances.​ cisco. there are some pri​ mary mon​ it​ or​ ing as​ pects that should be con​ sid​ ered: State of the Ser​ vice Graph. 2 In the Work pane. 3 In the Navigation pane.Layer 4 to Layer 7 Services 259 Service Graph Monitoring Once ACME Inc. see the Cisco APIC vices De​ ploy​ ment Guide at: http://​ www. For fur​ ther de​ tails of the pos​ si​ ble states and other rel​ e​ vant states.

Mon​ it​ or​ ing the vir​ tual de​ vices tells you what de​ vice clus​ ters are as​ so​ ci​ ated.260 Layer 4 to Layer 7 Services Monitoring and Resolving Service Graph Faults To mon​ it​ or a ser​ vice graph's faults: 1 On the menu bar.​ html#​ concept_​ 307C0CA3EB57469EAF7EF87AAE5A240F Monitoring a Virtual Device After you con​ fig​ ure a ser​ vice graph and at​ tach the graph to an end​ point group (EPG) and a con​ tract. The Work pane displays the faults that are related to the active service graph. and the health of the de​ vices in a de​ vice clus​ ter. choose Tenants > ALL TENANTS.​ cisco. the func​ tions in use and the func​ tion pa​ ra​ me​ ters passed to the de​ vices. 2 In the Work pane. see Ta​ bles 1 and 2 in the Cisco APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide at: http://​ www. choose the tenant. choose Tenants > ALL TENANTS. which VLANs are con​ fig​ ured for a de​ vice clus​ ter. choose Tenant_Name > L4-L7 Services > Deployed Graph Instances. the sta​ tis​ tics from the de​ vices. 4 In the Work pane. To mon​ it​ or a vir​ tual de​ vice: 1 On the menu bar. 3 In the Navigation pane.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy/​ b_​ L4L7_​ Deploy_​ chapter_​ 01010. The Work pane displays information about the virtual devices. choose the tenant. To un​ der​ stand the faults listed and pos​ si​ ble res​ o​ lu​ tions. such as faults and health scores. . choose the Faults tab. choose Tenant_Name > L4-L7 Services > Virtual Devices. you can mon​ it​ or the vir​ tual de​ vices as​ so​ ci​ ated with the ser​ vice graphs of a ten​ ant. 2 In the Work pane. 3 In the Navigation pane.

Issue the following command: # ping {ASAv management 0/0 interface} . This ex​ am​ ple de​ tails the process they fol​ lowed to con​ fig​ ure the ASAv fire​ wall vir​ tual ser​ vice ap​ pli​ ance as a sin​ gle node in routed mode. As a routed node. SSH to the APIC cluster IP address b.Layer 4 to Layer 7 Services 261 ASAv Sample Configuration One of the ser​ vice nodes that ACME Inc. a. 2 Enter enable mode. The high level steps are: • Create Logical and Concrete device clusters • Define a firewall ruleset/graph • Deploy the graph between 2 EPGs Verify ASAv VM configuration The first set of steps per​ formed is to ver​ ify ASAv VM con​ fig​ ur​ a​ tion and up​ load the ASA de​ vice pack​ age (or ver​ ify if it has al​ ready been done) 1 Log in to the ASAv VM. the ASAv will be​ come the de​ fault gate​ way for the hosts in the 2 EPGs that are con​ nected by the con​ tract that this ser​ vice graph gets as​ so​ ci​ ated to. will in​ te​ grate is an ASAv de​ ployed in sin​ glenode. routed FW mode. 3 Issue the following command: # show ip int brief 4 Verify that the ASAv has a correct IPv4 address on the management 0/0 interface. 5 Verify connectivity from the APIC to the management 0/0 interface of the ASAv.

click Flood. perform the following actions: a. d. c. Click Submit. 6 In the APIC GUI. c.262 Layer 4 to Layer 7 Services c.050 ms If the re​ sponse is dif​ fer​ ent. Remove GW IP from Relevant Fabric Bridge Domains Once the ASAv pack​ age and VM are ver​ if​ ied. 7 In the Navigation Pane expand Tenant_Name > L4-L7 Services > Packages > L4-L7 Service Device Types. Follow the prompts to upload the device package. 3 In the Navigation pane. click Flood. choose Tenant_Name > Networking > Bridge Domains > BD-EPG1. then perform the following actions: a. 2 In the Work pane. For the L3 Unknown Multicast Flooding radio buttons. Ad​ dress the con​ nec​ tiv​ ity prob​ lems be​ fore mov​ ing on. b. on the menu bar. choose Tenants > ALL TENANTS.16.10: icmp_seq=1 ttl=64 time=0. Right click on the Device Types. choose Tenant > Tenant_Name. 8 If an ASA device package is not listed. the next step is to re​ move SVI/GW IP ad​ dresses from the fab​ ric Bridge Do​ mains so the Layer 3 routed fire​ wall can be​ come the de​ fault gate​ way. b. choose the tenant. The return response should be similar to the following example: 64 bytes from 172. Uncheck the Unicast Routing check box. Choose Import Device Package. then there is likely some sort of con​ nec​ tiv​ ity issue.10. Re​ peat this process for the Bridge Do​ mains of the af​ fected EPGs. To re​ move the rout​ ing de​ fault gate​ way func​ tion on the EPG1 and EPG2 bridge do​ mains: 1 On the menu bar. For the L2 Unknown Unicast radio buttons. 4 In the Work pane. Create a Layer 4 to Layer 7 Device .

n. choose Tenants > ALL TENANTS. b. Enter a meaningful name. The fol​ low​ ing steps will cre​ ate a Layer 4 to Layer 7 de​ vice: 1 On the menu bar. i. In the Profile drop-down list. Enter a name that is relevant to the tenant and fuction.d/m.b.Layer 4 to Layer 7 Services 263 Create a Layer 4 to Layer 7 Device Per​ form the fol​ low​ ing steps to add log​ ic ​al and con​ crete de​ vice clus​ ters to a ten​ ant: 1 On the menu bar. ii.c. Expand Interface Specific Configuration. such as "TXX-ASAvFP". 4 In the Work pane. choose WebServiceProfileGroup or WebPolicyForRoutedMode. . 7 Verify the external and internal interfaces IPv4 addresses. h. f. perform the following actions: a.o. e. choose the tenant. g. Click SUBMIT. 3 In the Navigation Pane choose Tenant_Name > L4-L7 Services. 5 In the Create a L4-L7 Function Profile dialog box. 2 In the Work pane. choose Create Function Profile Group. choose the All Parameters tab. such as "TXX-FP-Group". You will configure IP addressing under Interface Related Configuration for both external and internal interfaces (externalIf and interalIf). k. d. In the Create L4-L7 Services Function Profile Group dialog box. In the Feature and Parameters section. c. Enter: ipv4 address in a. perform the following actions: i.p format j. 2 In the Work pane. Expand Interface Related Configuration for externalIf. 6 Choose L4-7 Services > Function Profiles > ALL PARAMETERS. Repeat steps 4f . choose Tenants > ALL TENANTS. choose the tenant. In the Profile Group drop-down list. Double-click IPv4 Address. click Create a L4-L7 Function Profile.4j as needed. Click Update.

Click NEXT. 3 In the Navigation Pane choose Tenant_Name > L4-L7 Services > L4-L7 Devices. Management IP address: ASAv IP address ii. Management Port: https iii.1 Model: ASAv c. choose Tenants > ALL TENANTS. enter interface values accordingly: 1. vii. 4 Expand both TXX-ASAv-Cluster and TXX-ASAv-Cluster_Device_1 to view the logical and physical interfaces. 2 In the Work pane. 5 In the Create L4-L7 Devices dialog box. choose the tenant. Credentials: {uid/pwd} h.264 Layer 4 to Layer 7 Services 3 In the Navigation pane. Click FINISH. VM: Tenant ASAv Controller (in the dropdown box) iv. specify the following values: i. click Create a L4-L7 virtual device. Name: GigabitEthernet0/1 vNIC: Network Adapter 3 Direction: consumer v. choose Tenant_Name > L4-L7 Services. Mode: Single Node d. Function Type: Goto e. Ver​ ify the Log​ ic ​al and Con​ crete De​ vice Clus​ ters have been con​ fig​ ured: 1 On the menu bar. Virtual Interfaces: Create two entries. Under Device 1. APIC to Device: Out of Band g. Connectivity VMM Domain: Txx-vCenter f. vi. Name: GigabitEthernet0/0 vNIC: Network Adapter 2 Direction: provider 2. Device Package: CISCO-ASA-1. perform the following actions: a. Click UPDATE after each entry. 5 Select TXX-ASAv-Cluster_Device_1 to see a graphic view of the concrete device. click + twice. Enter a meaningful name: Txx-ASAv-Cluster b. . 4 In the Work pane.

choose EPG1. e. which is the functional profile you created previously. In the Profile drop-down list. perform the following actions: a.Layer 4 to Layer 7 Services 265 Note This com​ pletes the Log​ ic ​al and Con​ crete de​ vice cre​ ation. choose Tenants > ALL TENANTS. In the Device Function drop-down list. choose the tenant. choose Tenants > ALL TENANTS. 4 In the Work pane. . choose Single Node . Create a Layer 4 to Layer 7 Service Graph Template 1 On the menu bar. 3 In the Navigation pane. Apply the Service Graph Template 1 On the menu bar. 2 In the Work pane. In the Type drop-down list. In the Name field. choose Actions > Create a L4-L7 Service Graph Template. enter “TXX-ASAv-L3-Routed-Template”. In the Consumer EPG / External Network drop-down list. perform the following actions: a. 6 You can verify that the template was created successfully by expanding Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates > Txx-ASAvL3-Routed-Template > Function Node . choose TXX-ASAv-FP. 3 In the Navigation pane. d. Click Submit. 5 In the Create L4-L7 Devices dialog box. choose Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates. c. choose EPG2. 2 In the Work pane.1/Firewall.Firewall in Routed Mode. choose Tenant_Name > L4-L7 Services > L4-L7 Service Graph Templates. 4 Right click Txx-ASAv-L3-Routed-Template and choose Apply L4-L7 Service Graph Template. choose CISCO-ASA-1.Firewall. In the Provider EPG / External Network drop-down list. choose the tenant. 5 In the Apply L4-L7 Service Graph Template to EPGs dialog box. b. b.

j. l. 6 Choose TXX-EPG1-to-EPG2-TXX-ASAv-L3-Router-Template-TXXProduction. This unassigns the inbound access list from the internal interface. The Devices Selection Polices folder and Deployed Graph Instances folder are now populated. Associate the Web and App Bridge domains to the Interfaces 1 Expand Devices Selection Policies. In the L4-L7 Devices drop-down list. Click Update. g. k. choose Txx-ASAv-Cluster. choose access-list-inbound. click Create A New One. Check the No Filter (Allow All Traffic) check box. choose Txx-ASAv-L3Routed-Template. Click Next. the configuration will not push to the ASA.266 Layer 4 to Layer 7 Services c. Expand Device Config > Interface Related Configuration externalIf > Interface Related Configuration to verify the IP address assignment. Expand Device Config > Interface Related Configuration internalIf > Interface Specific Configuration to verify the IP address assignment with the mask. choose the All Parameters tab. q. . In the Features and Parameters section. In the Contract Name field. Double click Device Config > Interface Related Configuration internalIf > Access Group > Inbound Access List. In the Service Graph Template drop-down list. If the mask is missing. Click Reset. e. f. m. n. 3 Select External. You can see that the Consumer and Provider EPGs are associated with the EPG1 and EPG2 server EPGs. Click Finish. i. enter “TXX-EPG1-to-EPG2”. 4 Assign the Bridge Domain to BD-EPG1. 2 Expand TXX-EPG1-to-EPG2-TXX-ASAv-L3-Router-Template-Firewall. h. In the Value drop-down list. o. Double click Device Config > Interface Related Configuration externalIf > Access Group > Inbound Access List. For the Contract radio buttons. This inbound access list is not desirable for the lab traffic-flow. d. p.

the re​ sult​ ing con​ fig​ ur​ a​ tion pushed to the ser​ vice node can be ver​ if​ ied by sim​ ple show com​ mands on the real ser​ vice node de​ vice. 12 Click SUBMIT.Layer 4 to Layer 7 Services 267 5 Repeat the process to assign the Bridge Domain to BD-EPG2. 15 CTX Terms: TXX-ASAv-L3Routed Template/T1/In. The ASAv will now have IP ad​ dresses asigned in the Ser​ vice Graph Pro​ file. 6 Click SUBMIT. Issue the fol​ low​ ing com​ mand: # show run | grep access This will show the ac​ cess list con​ fig​ ur​ a​ tion and you can re​ late that to the con​ fig​ ur​ a​ tion that was done in the APIC. 8 Expand TXX-ASAv-L3-Router-Template. This can be ver​ if​ ied by going into the ASAv VM con​ sole and issue the fol​ low​ ing com​ mand: # show ip int brief Verifying service node configuration Once the De​ vice has been in​ te​ grated and the Ser​ vice Graph has been con​ fig​ ured and as​ so​ ci​ ated. 7 Expand Layer 4 to Layer 7 Service Graph Templates. 16 Click SUBMIT. . 14 Filters: common/default. 9 Expand Function Node – Firewall > choose external. 11 CTX Terms: TXX-ASAv-L3Routed Template/T1/Out. Display the Access List configuration pushed-down from APIC SSH to the ASAv ser​ vice node for ac​ cess list val​ id ​a​ tion. 10 Filters: common/default. 13 Repeat the process for internal.

Issue the fol​ low​ ing com​ mand: # show nameif This will show the in​ ter​ face names pushed from the APIC and will show the re​ lated in​ ter​ face names to the log​ ic ​al in​ ter​ face names that were con​ fig​ ured in the APIC above.268 Layer 4 to Layer 7 Services Display the Interface configuration pushed-down from APIC SSH to the ASAv ser​ vice node for in​ ter​ face con​ fig​ ur​ a​ tion val​ id ​a​ tion. Display the current Interface names SSH to the ASAv ser​ vice node for in​ ter​ face name con​ fig​ ur​ a​ tion val​ id ​a​ tion. Issue the fol​ low​ ing com​ mand: # show run interface This will show the in​ ter​ face con​ fig​ ur​ a​ tion where the IP ad​ dress con​ fig​ ur​ a​ tion was pushed from the APIC. .

269 Health Scores .

.

Health Scores 271 Section Content • Understanding Health Scores • Understanding Faults • How Are Health Scores Calculated • Health Score Use Cases Using Health Scores for Proactive Monitoring Using Health Scores for Reactive Monitoring .

.

can be rep​ re​ sented in a sin​ gle con​ sol​ id ​ated health score. Ad​ di​ tion​ ally. The vis​ ib ​il​ ity pro​ vided by health scores give the op​ er​ at​ or a quick at-a-glance as​ sess​ ment of the cur​ rent sta​ tus of the en​ tire sys​ tem. de​ vices. This vis​ i-​ bil​ ity has a num​ ber of prac​ ti​ cal use cases. and avail​ abil​ ity of the sys​ tem they are re​ spon​ si​ ble for op​ er​ at​ ing. data through​ out the fab​ ric is col​ lected. and de​ scribe the re​ la​ tion​ ship be​ tween the var​ io ​us de​ vices and links in at​ tempt to pro​ vide a cor​ re​ la​ tion. their re​ la​ tion​ ships. but with health scores. per​ for​ mance. as well as the real time sta​ tus of their uti​ liza​ tion. Most ob​ jects in the model will have an as​ so​ ci​ ated health score. and avail​ abil​ ity read​ ily avail​ able. and there​ fore the cur​ rent sta​ tus of all of the ob​ jects in​ clud​ ing links. and cor​ re​ lated by the APIC in real time and then pre​ sented in an un​ der​ stand​ able for​ mat. per​ for​ mance. which pro​ vides Health Scores that make in​ for​ ma​ tion on sta​ tus. . this in​ for​ ma​ tion by it​ self is of lit​ tle to no value with​ out ad​ di​ tional data on its ef​ fect on the over​ all health of the net​ work. and how var​ io ​us faults im​ pact the cal​ cu​ la​ tion of the health score. or any sub​ set of the sys​ tem. To ad​ dress these chal​ lenges they can now uti​ lize the Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI). To man​ ua ​lly col​ lect and cor​ re​ late in​ for​ ma​ tion would have pre​ vi​ ously been a long and te​ dious task. which can be found from the Dash​ board or Pol​ icy tabs of the ob​ ject from the GUI. and in this chap​ ter we will clas​ sify these use cases as re​ ac​ tive and proac​ tive. all health scores are in​ stan​ ti​ ated from the health​ Inst class and can be ex​ tracted through the API. It is worth not​ ing that while pro​ vid​ ing such an​ swers may be easy as it re​ lates to an in​ de​ pen​ dent de​ vice or link. Tra​ di​ tional net​ work mon​ it​ or​ ing and man​ age​ ment sys​ tems at​ tempt to pro​ vide a model of in​ fra​ struc​ ture that has been pro​ vi​ sioned. ACI also pro​ vides the flex​ ib ​il​ ity to mon​ it​ or some as​ pects of how the health scores are cal​ cu​ lated.Health Scores 273 Understanding Health Scores ACME's Op​ er​ at​ ions team has been chal​ lenged on a reg​ ul​ ar basis to an​ swer basic ques​ tions re​ gard​ ing the cur​ rent sta​ tus. com​ puted. The ob​ ject model at the heart of ACI is in​ her​ ent to the in​ fra​ struc​ ture.

ACI health scores pro​ vide a quick check as to whether an issue being re​ ported is con​ firmed in a degra​ da​ tion of the health score. In this tab is a tree that will show the var​ i​ ous ob​ jects in a tree form to re​ veal faults.274 Health Scores Tenant Health Score In a re​ ac​ tive ca​ pac​ ity. and EPGs are im​ pacted by that fail​ ure. im​ me​ di​ ately pro​ vid​ ing feed​ back as to which ten​ ants. if you nav​ i​ gate to the ap​ pli​ ca​ tion pro​ file it has a Health tab. . As a ex​ am​ ple. Health scores also pro​ vide a real-time cor​ re​ la​ tion in the event of a fail​ ure sce​ nario. If so. the root cause of the issue can be found by ex​ plor​ ing the faults and how these get rolled up in the larger model. ap​ pli​ ca​ tions.

Ide​ ally. and other ca​ pac​ ity plan​ ning ex​ er​ cises. ACI health scores can help iden​ tify po​ ten​ tial bot​ tle​ necks in terms of hard​ ware re​ sources. this is not al​ ways re​ al​ is​ tic given the dy​ namic na​ ture of data cen​ ter en​ vi​ ron​ ments. with the goal of in​ creas​ ing the av​ er​ age health score of a given set of com​ po​ nents over time. and end​ points have fail​ ures. band​ width uti​ liza​ tion. how​ ever. . In​ stead the health score should be seen as a met​ ric that will change over time. Op​ er​ a​ tions teams also stand a bet​ ter chance of iden​ ti​ fy​ ing is​ sues be​ fore they im​ pact cus​ tomers or users. equip​ ment. Links.Health Scores 275 Object with a fault Proac​ tively. the health of all ap​ pli​ ca​ tion and in​ fra​ struc​ ture com​ po​ nents would al​ ways be at 100%.

.

All devices (Fabric Switches. It maintains a comprehensive. Faults.Health Scores 277 Understanding Faults From a man​ age​ ment point of view we look at the Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) from two per​ spec​ tives: 1 Policy Controller . All ob​ jects within ACI can be queried. events. up-to-date run-time representation of the administrative or configured state. 2 Telemetry device . a fault is rep​ re​ sented as a mu​ ta​ ble. In this model. Virtual Switches. and per​ sis​ tent MO. in​ clud​ ing faults. events and statistics to the APIC. managed and applied.Where all fabric configuration is created. state​ ful. integrated Layer 4 to Layer 7 devices) in an ACI fabric report faults. Fault Lifecycle . and sta​ tis​ tics in the ACI fab​ ric are rep​ re​ sented as a col​ lec​ tion of Man​ aged Ob​ jects (MOs) within the over​ all ACI Ob​ ject Model/Man​ age​ ment In​ for​ ma​ tion Tree (MIT).

they have a par​ ent. In most cases.xml?query-target=self&rspsubtree-include=faults As you can see. Man​ The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the health score for a ten​ ant named "3tier​ app": https://hostname/api/node/mo/uni/tn-3tierapp. The Fault “code” is an al​ phanu​ mer​ ic ​al string in the form FXXX. If the same con​ di​ tion is de​ tected mul​ ti​ ple times while the cor​ re​ spond​ ing fault MO is ac​ tive. a fault MO is au​ to​ mat​ ic ​ally cre​ ated.xml?query-target=self&rsp-subtreeinclude=stats The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the faults for a leaf node: https://hostname/api/node/mo/topology/pod-1/node-103. and Sys​ tem Mes​ sages age​ ment Guide. To re​ move a fault. with prop​ erty fil​ ters. no ad​ di​ tional in​ stances of the fault MO are cre​ ated. A fault MO re​ mains in the sys​ tem until the fault con​ di​ tion is cleared.xml?query-target=self&rsp-subtreeinclude=health The fol​ low​ ing ex​ am​ ple is a REST query to the fab​ ric that re​ turns the sta​ tis​ tics for a ten​ ant named "3tier​ app": https://hostname/api/node/mo/uni/tn-3tierapp. the fault con​ di​ tions are de​ fined by the fault rules of the par​ ent ob​ ject class. if the same con​ di​ tion is de​ tected mul​ ti​ ple times for the same af​ fected ob​ ject. see the Cisco APIC Faults. the con​ di​ tion rais​ ing the fault must be cleared. For more in​ for​ ma​ tion about fault codes.278 Health Scores When a spe​ cific con​ di​ tion oc​ curs. es​ ca​ lated. RN. the sys​ tem cre​ ates a fault MO as a child ob​ ject to the MO that is pri​ mar​ ily as​ so​ ci​ ated with the fault. MOs can be queried by class and DN. only one fault is raised while a counter for the re​ cur​ rence of that fault will be in​ cre​ mented. In other words. Fault MOs ap​ pear as reg​ ul​ ar MOs in MIT. such as a com​ po​ nent fail​ ure or an alarm. whether by con​ fig​ ur​ a​ - . and so on. pag​ i​ na​ tion. Events. There can be at most one fault with a given code under an MO. de-es​ ca​ lated. and deleted by the sys​ tem as spe​ cific con​ di​ tions are de​ tected. and so on. For a fault ob​ ject class. a DN.

Sever​ ity pro​ vides an in​ di​ ca​ tion of the es​ ti​ mated im​ pact of the con​ di​ tion on the ca​ pa​ bil​ ity of the sys​ tem or com​ po​ nent to pro​ vide ser​ vice. some of which are user configurable For ex​ am​ ple.Health Scores 279 tion. in which case the fault can be deleted by the user by ac​ knowl​ edg​ ing it. or tem​ per​ at​ ures. . An ex​ cep​ tion to this is if the fault is in the cleared or re​ tained state. you can set fault thresh​ olds on sta​ tis​ ti​ cal mea​ sure​ ments such as health scores. data traf​ fic. or a change in the run time state of the fab​ ric. Pos​ si​ ble val​ ues are: • Warning (possibly no impact) • Minor • Major • Critical (system or component completely unusable) The cre​ ation of a fault MO can be trig​ gered by in​ ter​ nal processes such as: • Finite state machine (FSM) transitions or detected component failures • Conditions specified by various fault policies.

.

You can see how all of these scores are ag​ gre​ gated by look​ ing at how man​ aged ob​ ject scores are cal​ cu​ lated. which is di​ rectly by the faults they have as​ so​ ci​ ated with them. For ex​ am​ ple. Each fault is weighted de​ pend​ ing on the level of im​ por​ tance. Sys​ tem and pod health scores are a weighted av​ er​ age of the leaf health scores. All health scores are cal​ cu​ lated using the num​ ber and im​ por​ tance of faults that apply to it. but con​ tain health scores of log​ i​ cal com​ po​ nents within that ten​ ant. di​ vided by the total num​ ber of learned end points. as well as an over​ all health score for the over​ all sys​ tem. Crit​ i​ cal faults might have a . In other words: Health Score calculation Ten​ ant health scores are sim​ i​ lar. it will only be weighted by the end points that are in​ cluded in that ten​ ant.Health Scores 281 How Health Scores Are Calculated Health scores exist for sys​ tems and pods. ten​ ants. mul​ ti​ plied by the spine co​ ef​ fi​ cient which is de​ rived from the num​ ber of spines and their health scores. man​ aged ob​ jects (such as switches and ports).

​ github. but it is pos​ si​ ble a fault may be oc​ cur​ ring that is af​ fect​ ing more than that one ten​ ant. low. Faults that have been iden​ ti​ fied as not im​ pact​ ing might even be re​ as​ signed a per​ cent​ age value of 0% so that it does not af​ fect the health score com​ pu​ ta​ tion. such as fire​ walls. pod.​ io/​ aci-troubleshooting-book/.​ Cen​ . and in​ tru​ sion pre​ ven​ tion/de​ tec​ tion sys​ tems. a fab​ ric admin will be able to see all health scores. Keep in mind. For ex​ am​ ple. it is pos​ si​ ble to change these val​ ues to bet​ ter match your en​ vi​ ron​ ment. In most cases. or “none” fault lev​ els. medium. the ten​ ant admin should be able to drill into the health scores that are vis​ ib ​le to them. but there should be a basic un​ der​ stand​ ing of whether faults should have high. and over​ all sys​ tem health scores. In this case the fab​ ric ad​ min​ is​ tra​ tor may have to start trou​ bleshoot​ ing. The ten​ ant and fab​ ric ad​ mins may also see health scores of any layer four through seven de​ vices.282 Health Scores high fault level at 100%. load bal​ ancers. see the Trou​ bleshoot​ ing Cisco Ap​ pli​ ca​ tion tric In​ fra​ struc​ ture book: http://​ datacenter. along with faults within our VMM do​ mains will all roll up into our ten​ ant. be​ cause of the role-based ac​ cess con​ trol. not all ad​ min​ is​ tra​ tors will be able to see all of the health scores. These. For more in​ for​ ma​ tion on how to use faults. while warn​ ings might have a low fault level at only 20%. Though faults in ACI come with de​ fault val​ ues. Luck​ ily there is re​ ally no need to un​ der​ stand the cal​ cu​ la​ tions of the health scores to use them ef​ fec​ tively. but a ten​ ant admin would only be able to see the health scores that per​ tain to the ten​ ants to which they have ac​ cess.

Most ob​ jects will have a Health tab which can be used to ex​ plore the re​ la​ tion​ ship be​ tween ob​ jects. ACI health scores will allow them to start pre​ vent​ ing is​ sues. If you see that one of the leaf switches is at 100% (green for good) one week. there is time to re​ solve the issue be​ fore any bot​ tle​ necks on the net​ work are no​ tice​ able. the fault it​ self will con​ tain in​ for​ ma​ tion about pos​ si​ ble re​ me​ di​ at​ ion steps. Using Health Scores for Reactive Monitoring Re​ ac​ tively. it is pos​ si​ ble to send these no​ ti​ fi​ ca​ tions to ap​ pli​ ca​ tion own​ ers. and so on. they are es​ sen​ tially base​ lines to which you can make com​ par​ isons later. this baselin​ ing method can be used as a ca​ pac​ ity plan​ ning tool. Other ways health scores can be used to proac​ tively mon​ it​ or your ACI en​ vi​ ron​ ment are by giv​ ing vis​ ib ​il​ ity of cer​ tain com​ po​ nents to other groups. In this sce​ nario. Data​ base Ad​ min​ is​ tra​ tor. or fire​ wall. In these cases adding an​ other load bal​ ancer.Health Scores 283 Health Score Use Cases Using Health Scores for Proactive Monitoring While ACME ad​ min​ is​ tra​ tors have tra​ di​ tion​ ally spent a lot of time re​ act​ ing to is​ sues on the net​ work. Health scores not only act as in​ di​ ca​ tors of faults. you can drill down to see what changed. and the next week the leaf is show​ ing a warn​ ing. Upon no​ ti​ fi​ ca​ tion that a health score has been de​ graded. The same sce​ nario can ob​ served with a load bal​ ancer or fire​ wall that is get​ ting over​ loaded. it is pos​ si​ ble the links are over​ sub​ scribed and so it can be time to ei​ ther move some of of the work​ load to an​ other leaf or maybe to add more band​ width by con​ nect​ ing more ca​ bles. This would pro​ vide mon​ i-​ tor​ ing of the en​ vi​ ron​ ment across the net​ work that has not pre​ vi​ ously been avail​ able and which is not able to be re​ trieved by any other means. This pro​ vides the abil​ ity to “dou​ ble-click to root cause”. VMware ad​ min​ is​ tra​ tors. . or maybe even op​ ti​ miz​ ing the rules may be needed to make traf​ fic flow more ef​ fi​ cient. As shown in the above ex​ am​ ples. Since you can ex​ port the scores and faults. and their as​ so​ ci​ ated faults. Once the root cause faults have been iden​ ti​ fied. Since it is still only a warn​ ing. an op​ er​ at​ or can use the GUI to eas​ ily nav​ ig ​ate the re​ la​ tion​ ships and faults that are con​ tribut​ ing to that health score. health scores can be used to di​ ag​ nose prob​ lems with the ACI fab​ ric.

.

285 Monitoring .

.

Tenant and Fabric Policies Stats Collection Policies Stats Export Policies Diagnostics Policies Call Home/SNMP/syslog Event Severity and Fault Severity Assignments Fault Lifecycle Policies TCAM Policy Usage Create TCAM Policy Monitor TCAM Prefix Usage Health Score Evaluation Policy Communication Policy • Proactive Monitoring .Infrastructure Monitoring APICs CPU utilization and Memory Disk Utilization Physical and Bond Interface Statistics APIC Fan Status Temperature Status Power Supply Status ​Monitoring Leaf Switches Monitoring Switch CPU Utilization Monitoring Switch Memory Utilization Monitoring File System Health Monitoring CoPP (Control Plane Policing) Statistics Physical Interface Statistics and Link State Module Status Switch Fan Status .Monitoring 287 Section Content • Proactive Monitoring .

288 Monitoring Power Supply Status LLDP Neighbor Status GOLD Diagnostic Results • Proactive Monitoring Use Cases Monitoring Workload Bandwidth EPG Level Statistics • Reactive Monitoring Reactive Monitoring Tools Switch Port Analyzer (SPAN) Traceroute Atomic Counters Traffic Map Enhanced Troubleshooting Wizard IPing Audit Logs • Reactive Monitoring Use Cases ​Loss of Connectivity to Endpoint Users Report that an Application Running in the Fabric is Slow .

ACI pro​ vides an APIC which will do all of the sta​ tis​ tics gath​ er​ ing. and add phys​ ic ​al test servers to VPCs on the leaf switches. A Fab​ ric Wide pol​ icy would be cre​ ated as a de​ fault pol​ icy to be ap​ plied to all ten​ ants. the ten​ - . but is often ne​ glected be​ cause putting out fires in the net​ work usu​ ally takes pri​ or​ ity. can be used to drill into any of the com​ po​ nents and pro​ vides the abil​ ity to click on a Stats tab to dis​ play on-de​ mand sta​ tis​ tics. it has still been nec​ es​ sary to man​ ua ​lly spec​ ify which de​ vices are to be mon​ it​ ored and how they should be mon​ it​ ored. At this point the APIC is al​ ready gath​ er​ ing sta​ tis​ tics for the VMM do​ main and the phys​ ic ​al de​ vices. as well as to trou​ bleshoot or pre​ dict any is​ sues that may be aris​ ing. For ex​ am​ ple. Fab​ ric. or Ac​ cess. When plan​ ning to move an ap​ pli​ ca​ tion from a legacy net​ work to the ACI in​ fra​ struc​ ture. Add test VMs to port groups on ei​ ther a DVS or AVS as​ so​ ci​ ated with the APIC. it is sen​ si​ ble to start by test​ ing be​ fore going straight to pro​ duc​ tion. CLI. This could also be in a test​ ing ten​ ant which is com​ pletely sep​ ar​ ate from the pro​ duc​ tion en​ vi​ ron​ ment. Even when they have used tools to gather data on layer one through seven de​ vices. Ten​ ant. The next step is to con​ fig​ ure a pol​ icy for trend analy​ sis. to over​ ride that pol​ icy for a par​ tic​ ul​ ar ten​ ant. whether ac​ cessed through the GUI. There are four dif​ fer​ ent scopes for sta​ tis​ tics gath​ er​ ing: Com​ mon or Fab​ ric Wide.Monitoring 289 Proactive Monitoring Tenant and Fabric Policies Proac​ tive mon​ it​ or​ ing is a very im​ por​ tant piece of the net​ work ad​ min​ is​ tra​ tor's job. this will save net​ work ad​ min​ is​ tra​ tors both time and frus​ tra​ tion. How​ ever. Sta​ tis​ tics gath​ er​ ing has been a some​ what man​ ual and even re​ source in​ ten​ sive process for ACME in the past. The APIC. or API. the human error and ef​ fort is min​ im ​al. Since sta​ tis​ tics are gath​ ered au​ to​ mat​ ic ​ally and poli​ cies are used and can be re-used in other places. since the APIC makes it in​ cred​ ib ​ly easy to gather sta​ tis​ tics and per​ form analy​ ses. and pro​ vides the abil​ ity to proac​ tively mon​ it​ or your en​ tire en​ vi​ ron​ ment with​ out all of the has​ sle of main​ tain​ ing a third party mon​ it​ or​ ing tool. but more im​ por​ tantly it en​ ables the setup of poli​ cies to keep per​ sis​ tent data to an​ al​ yze trends in the en​ vi​ ron​ ment. or often for​ got to add a new de​ vice to their Net​ work Mon​ it​ or​ ing Sys​ tem (NMS). SNMP and a third party tool may have been used to mon​ it​ or the CPU of switches or band​ width uti​ liza​ tion on ports. but they strug​ gled with en​ ter​ ing cor​ rect SNMP in​ for​ ma​ tion on each de​ vice. How​ ever.

but a dou​ ble click on them will change the admin state or re​ ten​ tion pe​ ri​ ods. just click on the pol​ icy that spec​ if​ ies a 5 minute gran​ ul​ ar​ ity and change the re​ ten​ tion pe​ riod to 2 hours. Click Submit. Create Tenant Monitor policy To cre​ ate a ten​ ant mon​ it​ or​ ing pol​ icy: 1 On the menu bar. a Ten​ ant pol​ icy is cre​ ated to gather sta​ tis​ tics. Even if this ten​ ant is shared with other ap​ pli​ ca​ tions. choose Actions > Create Monitoring Policies.290 Monitoring ant pol​ icy will over​ ride the Fab​ ric pol​ icy. cus​ tomers. and syslog • Event Severity Assignment Policies • Fault Lifecycle Policies Stats Collection Policies Click​ ing on Stats Col​ lec​ tion Poli​ cies will dis​ play the de​ fault re​ ten​ tion pe​ ri​ ods and admin states (En​ abled/Dis​ abled) for ALL Mon​ it​ ored Ob​ jects. choose Tenants > ALL TENANTS. SNMP. choose Tenant_Name > Monitoring Policies. In the fol​ low​ ing test​ ing ex​ am​ ple. 2 In the Work pane. 5 In the Create Monitoring Policies dialog box. A mon​ it​ or​ ing ob​ ject tells the APIC which com​ po​ nents to gather . For ex​ am​ ple. choose Tenant_Name > Monitoring Policies > Policy_Name to display the following information: • Stats Collection Policies • Stats Export Policies • Diagnostics Policies • Callhome. choose the Tenant_Name. 3 In the Navigation pane. Most likely the de​ faults will be kept. test cases. b. but be re​ tained for 2 hours. it will pro​ vide a real world ex​ am​ ple of how the ap​ pli​ ca​ tion will be​ have in a pro​ duc​ tion en​ vi​ ron​ ment. perform the following actions: a. In the Name field enter a name for the Monitoring Policy. 4 In the Work pane. It is sim​ il​ arly pos​ si​ ble to change the poli​ cies for spe​ cific Mon​ it​ or​ ing Ob​ jects. to have it poll a com​ po​ nent every 5 min​ utes. 6 In the Navigation pane.

use the Bridge Do​ main (infra. Click Update. The retention policy may either be inherited or explicitly specified as enabled or disabled as well. Stats Export Policies . b. perform the following actions: a. Click + to add the policy. choose the Tenant_Name. 4 In the Work pane. 2 In the Work pane. choose the Tenant_Name. Put a checkmark next to the Monitoring Objects to be included. Leaf Port. choose Tenants > ALL TENANTS. There are sev​ eral op​ tions and this will all de​ pend on what is im​ por​ tant to mon​ it​ or in the en​ vi​ ron​ ment. in the Stats Collection Policy dialog box. c. and/or Taboo Con​ tract. To add a pol​ icy to a Mon​ it​ or​ ing Ob​ ject: 1 On the menu bar. For ex​ am​ ple. c.Monitoring 291 sta​ tis​ tics about. Select the Monitoring Object. e. 3 In the Navigation pane choose Tenant_Name > Monitoring Policies > Monitoring Policy_Name > Stats Collection Policies a. Leave the state as inherited to stick with the defaults as set for ALL. d.​ RSOInfraBD) Mon​ it​ or​ ing Ob​ ject. choose Tenants > ALL TENANTS. to change the in​ for​ ma​ tion gath​ ered for Bridge Do​ mains. To add mon​ it​ or​ ing ob​ jects: 1 On the menu bar. VXLAN Pool. b. changes might be made to Mon​ it​ or​ ing Ob​ ject poli​ cies for Ten​ ant. Select the granularity with which it is to poll. Click Submit. Click on the pull down menu to se​ lect a mon​ it​ or​ ing ob​ ject and add a re​ ten​ tion pol​ icy to it. or explicitly select enabled or disabled. and remove any checkmarks next to Monitoring Objects to be left out. Click on the Pencil icon to edit the Monitoring Objects. f. 2 In the Work pane. 3 In the Navigation pane choose Monitoring Policies > Monitoring Policy_Name > Stats Collection Policies. For this ex​ am​ ple.

choose the Tenant_Name. in the Stats Export Policy dialog box. . Choose either JSON or XML as the format. Click + under Export Destinations to specify a server where this information is to be collected. Click + to add the policy. Click Ok. f. 3 In the Navigation pane choose Tenant_Name > Monitoring Policies > Monitoring Policy_Name > Stats Export Policies. d. 2 In the Work pane. 5 Click Submit. or it may be dictated by the tool used to read it. Select ALL or a specific monitoring object from the drop-down list. perform the following actions: a. choose Tenants > ALL TENANTS. lo​ cated under the mon​ it​ or​ ing pol​ icy. Much like the Stats Col​ lec​ tion Poli​ cies. Another wizard will pop up to enable specification of the protocols and credentials used to connect to this server. c. Next to the Mon​ it​ or​ ing Ob​ ject is the Pen​ cil but​ ton which en​ ables se​ lec​ tion of the mon​ it​ or​ ing ob​ jects to be con​ fig​ ured with di​ ag​ nos​ tics poli​ cies. 4 In the Work pane. b. Diagnostics Policies Next are the di​ ag​ nos​ tics poli​ cies in the nav​ ig ​a​ tion pane on the left. e. There's really no difference other than personal preference. To cre​ ate a Stats Ex​ port Pol​ icy: 1 On the menu bar.292 Monitoring Stats Export Policies It is de​ sir​ able to col​ lect these on​ go​ ing sta​ tis​ tics as well as to see how this data be​ haves over time. it is pos​ si​ ble to cre​ ate a pol​ icy for ALL mon​ it​ or​ ing ob​ jects. Now define the Stats Export Policy in the wizard. This is a re​ ally slick fea​ ture that al​ lows the setup of di​ ag​ nos​ tics test for the Mon​ it​ or​ ing Ob​ jects that were spec​ if​ ied in the Stats Col​ lec​ tion Poli​ cies. or leave it uncompressed. Choose to compress it using GZIP.Boot-Up di​ ag​ nos​ tics or On​ go​ ing di​ ag​ nos​ tics. or se​ lect spe​ cific mon​ it​ or​ ing ob​ jects and spec​ ify where this in​ for​ ma​ tion will be saved. There are two dif​ fer​ ent kind of poli​ cies for con​ fig​ ur​ a​ tion . Use the Stats Ex​ port Poli​ cies op​ tion in the left nav​ ig ​a​ tion pane. g.

Peripherals. Cisco Call Home is a fea​ ture in many Cisco prod​ ucts that will pro​ vide email or webbased no​ ti​ fi​ ca​ tion alerts in sev​ eral dif​ fer​ ent for​ mats for crit​ ic ​al events. in the Diagnostic Policies dialog box. Select one of the Monitoring Objects. b. This al​ lows ad​ min​ is​ tra​ tors to re​ solve is​ sues be​ fore they turn into out​ ages. v. and System Memory. CPU. 3 In the Work pane. i. Boot-Up runs the tests while the devices are booting. 2 In the Navigation pane choose Tenant_Name > Monitoring Policies > default > Diagnostics Policies. perform the following actions: Note: Click on the Pencil Icon and put checks next to the Monitoring Objects which diagnostics tests are to be added to. choose Fabric > Fabric Policies. iv. or recommended tests. The Call Home/SNMP/sys​ log pol​ icy will allow alert​ ing to be con​ fig​ ured in a flex​ ib ​le man​ ner. and Ongoing will run the tests as often as specified within the wizard. iii. Click + to add an Object.Monitoring 293 To con​ fig​ ure di​ ag​ nos​ tic poli​ cies: 1 On the menu bar. select either Boot-Up or Ongoing. The di​ ag​ nos​ tics found here can be use​ ful in find​ ing failed com​ po​ nents be​ fore they cause major is​ sues within your en​ vi​ ron​ ment. Call Home/SNMP/syslog There are a few dif​ fer​ ent ways to setup no​ ti​ fi​ ca​ tion or alert poli​ cies. There are five different diagnostics tests available: ASIC. a. full tests. Dif​ fer​ ent log​ ging lev​ els may be se​ lected for no​ ti​ fi​ ca​ tions and alert lev​ els spec​ if​ ied for Mon​ it​ or​ ing Ob​ jects from which alerts are to be re​ ceived. . ii. Click Submit. In the wizard give it a name and select the admin state. SNMP or sys​ log poli​ cies can also be used with cur​ rent no​ ti​ fi​ ca​ tion sys​ tems. Double-click on each to obtain the option of specifying no tests. Internal Connectivity.

but there is a major fault you'd also like to be no​ ti​ fied about im​ me​ di​ ately. 2 In the Work pane. Select a Monitoring Object. choose Tenants > ALL TENANTS. you can change the sever​ ity for that par​ tic​ ul​ ar fault code. Note: Squelched gives it a weight of 0%. which will dictate the fault codes for which you are changing the fault severity. perform the following actions: a. if only crit​ ic ​al faults are being no​ ti​ fied. Once a fault is de​ tected it is in the "soak​ ing" state. the de​ fault sever​ ity as​ sign​ ments for Events and Faults will be kept. choose the Tenant_Name. Fault Lifecycle Policies Fault Life​ cy​ cle is the term Cisco uses to de​ scribe the life of a fault. Most likely." It is only in this state briefly and moves on to the "clear​ ing time" state. Warning. Select the particular fault code for which severity is to be changed. Inherit. c. Click + to add an Object. 4 In the Work pane. Critical. re​ ferred to as the "soak​ ing in​ ter​ val" it will move on to the "raised" state." Lastly it moves on to the "re​ tain​ ing" state and does not get re​ moved until the end of the "re​ tain​ ing in​ ter​ val. Select the severity: Cleared. 3 In the Navigation pane. Minor. For ex​ am​ ple. d. Major. After the fault clears it's in a state called "raised clear​ ing. The Event Sever​ ity As​ sign​ ment Poli​ cies are con​ fig​ ured in the same way. "Raised" means the fault is still pre​ sent after the soak​ ing in​ ter​ val. 5 Click Update. but there are ex​ am​ ples where an ACI ad​ min​ is​ tra​ tor may de​ cide the event or fault is more or less se​ vere than the de​ fault value. After a cer​ tain amount of time." . Squelched. choose Tenant_Name > Monitoring Policies > Monitoring_Policy > Fault Lifecycle Policies.294 Monitoring Event Severity and Fault Severity Assignments Event and fault sever​ it​ ies can be changed for events raised by Mon​ it​ or​ ing Ob​ jects. Info. in the Fault Severity Assignment Policies dialog box. b. meaning it does not affect health scores. 1 On the menu bar. It re​ mains in the "clear​ ing time" state for the amount of time spec​ if​ ied in the "clear​ ing in​ ter​ val.

which will dictate the fault codes for which you are changing the default intervals. Note: The default for the Clearing Interval is 120 seconds. b. sources and des​ ti​ na​ tions be​ come one entry for a given EPG. There is a a sys​ tem wide view of avail​ able TCAM re​ sources. 3 In the Navigation pane. TCAM Policy Usage The phys​ ic ​al ternary con​ tent-ad​ dress​ able mem​ ory (TCAM) in which pol​ icy is stored for en​ force​ ment is an ex​ pen​ sive com​ po​ nent of switch hard​ ware and there​ fore tends to lower pol​ icy scale or raise hard​ ware costs. 4 In the Work pane. ACME will have other poli​ cies to con​ fig​ ure in the fab​ ric as out​ lined in the fol​ low​ ing sec​ tions. m is the num​ ber of des​ ti​ na​ tions. and Soaking Interval (all in seconds). where n is the num​ ber of sources. pol​ icy is ap​ plied based on the EPG rather than the end​ point it​ self. Click +. Retention Interval. 2 In the Work pane. which re​ duces the num​ ber of total en​ tries re​ quired. choose the Tenant_Name. choose Tenants > ALL TENANTS. perform the following actions: a. Select a Monitoring Object. the Retention Interval is 3600 seconds. . At this point there will be a fully work​ ing ten​ ant mon​ it​ or​ ing pol​ icy. This pol​ icy size can be ex​ pressed as n*m*f. and f is the num​ ber of pol​ icy fil​ ters. in the Fault Lifecycle Policies dialog box. choose Tenant_Name > Monitoring Policies > Monitoring_Policy > Fault Lifecycle Policies.Monitoring 295 To change Fault Life​ cy​ cle In​ ter​ vals: 1 On the menu bar. c. and the Soaking Interval is 120 seconds. Specify times for the Clearing Interval. Within the Cisco ACI fab​ ric. and you will see a table sum​ ma​ riz​ ing ca​ pac​ i-​ tiy for all nodes. TCAM is a fab​ ric re​ source that should be mon​ it​ ored. To see this click on Fab​ ric > In​ ven​ tory > Pod1 and then se​ lect the Op​ er​ a​ tional tab in the work pane. Within the Cisco ACI fab​ ric.

or you an​ tic​ i​ pate the pos​ si​ bil​ ity of con​ sis​ tently being over​ sub​ scribed. choose Monitor Policies > default > Stats Collection Policies. There is a Fab​ ric Re​ source Cal​ cu​ la​ tion tool on Github that will help with es​ ti​ ma​ tion of nor​ mal op​ er​ at​ ing pa​ ra​ me​ ters: https://​ github. Create TCAM Policy Monitor 1 On the menu bar. If your en​ vi​ ron​ ment has a high rate of change. The ar​ chi​ tec​ ture/de​ sign team should ar​ tic​ u​ late what the as​ sump​ tions were for TCAM uti​ liza​ tion. the de​ fault mon​ i​ tor​ ing poli​ cies will alert you to a re​ source short​ age and lower over​ all fab​ ric health score.296 Monitoring Switch TCAM capacity dashboard TCAM is a crit​ i​ cal sys​ tem re​ source in an ACI fab​ ric and should be mon​ i​ tored for uti​ liza​ tion. you may want to set dif​ fer​ ent thresh​ olds.​ com/​ datacenter/​ FabricResourceCalculation As a gen​ eral rule. choose Fabric > Fabric Policies. . 2 In the Navigation pane.

choose Fabric > Fabric Policies. In the Penalty of fault severity major dropdown menu. Click + under Config Thresholds. 1 On the menu bar. 2 In the Navigation pane. Select the Monitoring Object Equipment Capacity Entity (eqptcapacity. select the desired %. b. . perform the following actions: a. choose Monitor Policies > Common Policies > Health Score Evaluation Policy > Health Score Evaluation Policy. select the desired %. select the blue pencil icon next to policy CAM entries usage current value. d. In the Penalty of fault severity critical dropdown menu. in the Stats Collection Policies dialog box. b. choose Fabric > Fabric Policies. 3 In the Work pane. c.Entity). Select the Stats TypeLayer3 Entry. c. Health Score Evaluation Policy 1 On the menu bar. select the desired %. d. d.Entity). In the Penalty of fault severity warning dropdown menu. perform the following actions: a. In the Penalty of fault severity minor dropdown menu. choose Monitor Policies > default > Stats Collection Policies. In the Thresholds For Collection 5 Minute window. in the Properties dialog box. in the Stats Collection Policies dialog box. Select the Monitoring Object Equipment Capacity Entity (eqptcapacity. c. b. 3 In the Work pane. In the Thresholds For Collection 5 Minute window. select the desired %. TCAM Prefix Usage This pro​ ce​ dure man​ ages a TCAM Pre​ fix Usage. perform the following actions: a.Monitoring 297 3 In the Work pane. Click + under Config Thresholds. Select the Stats Type Policy Entry. 2 In the Navigation pane. select the blue pencil icon next to policy CAM entries usage current value.

From the HTTPS Admin State dropdown menu select the desired state. Select the desired HTTP redirect state. From the Telnet Port dropdown menu select the desired port. b.298 Monitoring 4 Click Submit. 2 In the Navigation pane. e. choose Fabric > Fabric Policies. From the HTTP Port dropdown menu select the desired port. 5 Click Submit. From the SSH Admin State dropdown menu select the desired state. f. 3 In the Work pane. d. From the HTTPS Port dropdown menu select the desired port. . h. expand Pod Policies > Policies > Communication. choose Actions > Create Communication Policy. 4 In the Create Communication Policy dialog box. From the Telnet Admin State dropdown menu select the desired state. i. g. Enter Communication Policy Name. From the HTTP Admin State dropdown menu select the desired state. j. Select the desired HTTPS redirect state. perform the following actions: a. c. Communication Policy 1 On the menu bar.

x is avail​ able at: http://​ www. Monitoring APICs CPU utilization and Memory GUI The eas​ ie ​st way to quickly ver​ ify the health of the con​ trollers is the APIC. and fab​ ric and ten​ ant poli​ cies show us a nar​ rower view of fab​ ric and ten​ ant health.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. REST API calls and CLI.Monitoring 299 Proactive Monitoring . In this ex​ am​ ple. The screen​ shot shows where this in​ for​ ma​ tion is vis​ ib ​le. A full list of MIBs sup​ ported on the 1. sug​ gest rec​ om​ mended thresh​ olds for trig​ ger​ ing alerts or alarms from Net​ work Mon​ it​ or​ ing Sys​ tems (NMS). When log​ ging into the sys​ tem dash​ board.​ cisco. and when pos​ si​ ble. GUI. ACME will also need to set up poli​ cies to mon​ it​ or spe​ cific re​ sources. This sec​ tion of the book at​ tempts to cover some key per​ for​ mance in​ di​ ca​ tors that should be mon​ it​ ored. the health of the APICs and the health of the clus​ ter it​ self are dis​ played right at the dash​ board. SNMP. . There are sev​ eral meth​ ods to ob​ tain this data.​ html It is im​ por​ tant to note that this list changes as newer soft​ ware ver​ sions are made avail​ able.Infrastructure While health scores pro​ vide a com​ pre​ hen​ sive view of the health sta​ tus of var​ io ​us ob​ jects. Op​ er​ at​ ors can use a wide array of tools in​ clud​ ing Sys​ log. pro​ ce​ dures for mon​ it​ or​ ing them. and ref​ er​ ences are pro​ vided to ways of ob​ tain​ ing the data when pos​ si​ ble. two out of the three APICs are in a sub-op​ ti​ mal state and APIC 1 is also ex​ pe​ ri​ enc​ ing is​ sues. and more vari​ ables may be ex​ posed via SNMP MIBs in the fu​ ture.

300 Monitoring System health dashboard The nor​ mal state for these is to have them all green in a "fully fit" state im​ ply​ ing the APICs are syn​ chro​ nized with each other.CPU uti​ liza​ tion maxMemAl​ loc . The pro​ cEn​ tity ob​ jects con​ tain the fol​ low​ ing use​ ful prop​ er​ ties: cpuPct . REST API Con​ trollers pro​ vide in​ for​ ma​ tion re​ gard​ ing the cur​ rent sta​ tus of CPU and mem​ ory uti​ liza​ tion by cre​ at​ ing in​ stances of the pro​ cEn​ tity class. A more de​ tailed drill​ down is avail​ able by click​ ing on Sys​ tem > Con​ trollers. pro​ cEn​ tity is a con​ tainer of processes in the sys​ tem.The max​ i​ mum mem​ ory al​ lo​ cated for the sys​ tem mem​ Free .xml? . This ob​ ject holds de​ tailed in​ for​ ma​ tion about var​ i​ ous processes run​ ning on the APIC.The max​ i​ mum amount of avail​ able mem​ ory for the sys​ tem Sam​ ple Usage: This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: http[s]://apic_ip/api/node/class/procEntity.

3 0.7 0. 0 zombie 0. 131954932k total. 0k used.34 svc_ifc_reader. 0k total.0%wa.0%st 1952656k cached PID USER PR NI 32102 root 20 0 556m 205m 85m S 3.2 38:11.27.7 0. 124481752k free.0 0. The GUI pro​ vides ready ac​ cess to disk space uti​ liza​ tion of all par​ ti​ tions on the sys​ tem and can be used for mon​ i​ tor​ ing this in​ for​ ma​ tion.24 nginx.74 svc_ifc_scripth 20 0 834m 419m 94m S 1.37 svc_ifc_observe 32128 ifc 20 0 639m 315m 69m S 1.6 20:03.3%id. Cpu(s): Mem: 0. load average: 4.04 svc_ifc_applian 32120 ifc 20 0 660m 343m 86m S 2. 0.0%hi.7 0.0 0.92 svc_ifc_topomgr 32105 root 20 0 659m 258m 85m S 1. 409540k buffers TIME+ COMMAND Disk Utilization GUI There are sev​ eral disks and file sys​ tems pre​ sent on the APICs.2 17:08.3 27:58.bin 1291 root VIRT 0k free. 99.3%us.35 svc_ifc_bootmgr 32113 ifc 20 0 1083m 721m 69m S 1.19. The disk uti​ liza​ tion can be viewed by click​ ing on Sys​ tem > Con​ trollers > Apic-X > Stor​ age The work pane dis​ plays the uti​ liza​ tion of all par​ ti​ tions in the sys​ tem. Swap: 4 users. user@apic1:~> top top . 4. 0 stopped.73 svc_ifc_dbgr.bi 32121 ifc 20 0 631m 286m 86m S 2. 0.3 20:35. .2 17:41.11:41:51 up 16:50.Monitoring 301 CLI The Linux Top util​ ity also comes built into the APIC con​ trollers and can be used for trou​ bleshoot​ ing and/or ver​ if​ i​ ca​ tion. 0. 353 sleeping. 0.4%sy.2 17:13.2 16:28. 7473180k used. 32132 ifc 20 0 657m 252m 71m S 1.3 0. 4. Tasks: 354 total.29 1 running.0%ni. RES SHR S %CPU %MEM 0.0%si.7 0.

xml? Physical and Bond Interface Statistics APICs use a bonded in​ ter​ face that is typ​ ic ​ally dual-homed to two leaves for con​ nec​ tiv​ ity to the ACI fab​ ric and have the abil​ ity to use a bonded in​ ter​ face that can be dual homed to the out-of-band man​ age​ ment net​ work. Bond0 is the bond in​ ter​ face used to con​ nect to the fab​ ric it​ self (to con​ nect to leaves that con​ nect into the fab​ ric). The GUI should be used as a sin​ gle source of truth for file sys​ tem uti​ liza​ tion.302 Monitoring CLI user@apic1:~> df Filesystem 1K-blocks /dev/dm-1 41282880 10518960 Used Available Use% Mounted on 28666872 tmpfs 4194304 56456 4137848 tmpfs 65977464 964 65976500 1% /tmp 10518960 28666872 27% /data 13860672 25327104 36% /firmware 27% / 2% /dev/shm /dev/mapper/vg_ifc0-data 41282880 /dev/mapper/vg_ifc0-firmware 41284928 /dev/mapper/vg_ifc0-data2 583149656 1281104 552246280 1% /data2 * Note that not all file sys​ tems are vis​ ib ​le from the CLI as some re​ quire root ac​ cess to reach the mount points. REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: http[s]://apic-ip/api/node/class/eqptStorage. Bond1 is the bond in​ ter​ face used to con​ nect to the out-of-band seg​ ment (to con​ nect to an OOB seg​ ment that al​ lows setup of the APIC it​ self). .

GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-1/sys. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs. The CLI also pro​ vides in​ for​ ma​ tion on de​ tailed in​ ter​ face sta​ tis​ tics such as RX and TX coun​ ters. nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > Equip​ ment-Fans REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-1.json?querytarget=subtree&target-subtree-class=eqptFan CLI .json?querytarget=subtree&target-subtree-class=l3EncRtdIf APIC Fan Status The fol​ low​ ing sec​ tion de​ scribes method​ olo​ gies to re​ trieve the sta​ tus of the fan trays on the APICs. nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > In​ ter​ faces CLI Both "if​ con​ fig" and the "ip link" CLI com​ mands can be used to ver​ ify link state.Monitoring 303 The bond in​ ter​ faces rely on un​ der​ ly​ ing phys​ ic ​al in​ ter​ faces and it is im​ por​ tant to note that the GUI pro​ vides link in​ for​ ma​ tion for both the phys​ ic ​al and log​ ic ​al bond in​ ter​ faces.

nav​ ig ​ate to Sys​ tem > Con​ trollers > Apic-x > Equip​ ment-Sen​ sors REST API .16.16.--------. admin@172.16.179 Warning: Permanently added '172. Max. Max.176.304 Monitoring CLI The Fan sta​ tus for the APICs can be mon​ it​ ored using the CLI on the CIMC port of the APIC.176.-------- Min. Min. If this has not been setup pre​ vi​ ously.179's password: C220-FCH1807V02V# scope sensor C220-FCH1807V02V /sensor # show fan Name Sensor Reading Units Status ----------. login to the CIMC using the cre​ den​ tials used for set​ ting up the CIMC (may not be the same as the cre​ den​ tials used for APIC).--------- FAN1_TACH1 Normal 7490 RPM 1712 N/A 1284 N/A FAN1_TACH2 Normal 7490 RPM 1712 N/A 1284 N/A FAN2_TACH1 Normal 7490 RPM 1712 N/A 1284 N/A FAN2_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN3_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A FAN3_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN4_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A FAN4_TACH2 Normal 7276 RPM 1712 N/A 1284 N/A FAN5_TACH1 Normal 7704 RPM 1712 N/A 1284 N/A Temperature Status To mon​ it​ or the tem​ per​ at​ ure state of the var​ io ​us sen​ sors avail​ able on the APICs use the fol​ low​ ing steps. To ob​ tain this. Warning Warning Failure Failure ----.------. user@apic1:~> ssh -l admin 172. The CIMC port is the in​ te​ grated lights-out man​ age​ ment port that can be used to re​ cover an APIC in the event of a cat​ as​ trophic fail​ ure.176. the de​ fault user​ name is admin and the de​ fault pass​ word is pass​ word. GUI To view in​ ter​ face sta​ tus for the in​ ter​ faces on the APICs.-------.179' (RSA) to the list of known hosts.--------.

0 RISER1_INLET_TMP Normal 45.json?querytarget=subtree&target-subtree-class=eqptSensor CLI C220-FCH1807V02V /sensor # show temperature C220-FCH1807V02V /sensor # show temperature Name Sensor Reading Units Status -----------------.0 N/A 85.0 C N/A 65.0 C N/A 65.0 N/A 90.-------. Warning Warning Failure Failure ------.0 C N/A 65.0 N/A 85.5 C N/A 81.0 N/A 85. Max.0 P2_TEMP_SENS Normal 50.0 C N/A 85.0 N/A 70.0 N/A 70. Max.0 C N/A 60.0 DDR3_P2_E1_TEMP Normal 43.0 N/A 70.------.0 N/A 85.0 N/A 65.0 PSU1_TEMP Normal 37.0 N/A 85.0 C N/A 65.0 N/A 70.0 VICP81E_0_TMP3 Normal 56.0 N/A 85. .0 RISER2_INLET_TMP Normal 41.-------.0 DDR3_P1_C1_TEMP Normal 44.0 FP_TEMP_SENSOR Normal 37.0 C N/A 60.0 C N/A 60.0 N/A 85.0 N/A 86.Monitoring 305 REST API This in​ for​ ma​ tion can be re​ trieved for all APIC con​ trollers using the fol​ low​ ing REST call: https://{{apic-ip}}//api/node/mo/topology/pod-1/node-1.0 DDR3_P1_A1_TEMP Normal 42.-------- Min.0 C N/A 60.0 C N/A 65.0 RISER2_OUTLETTMP Normal 41.0 DDR3_P1_D1_TEMP Normal 44.0 DDR3_P1_B1_TEMP Normal 43.0 C N/A 65.0 DDR3_P2_H1_TEMP Normal 41.5 C N/A 81.0 DDR3_P2_F1_TEMP Normal 43.0 N/A 70.0 N/A 85.0 C N/A 65.0 C N/A 65.0 N/A 86.0 Power Supply Status To mon​ it​ or the tem​ per​ at​ ure state of the var​ io ​us sen​ sors avail​ able on the APICs use the fol​ low​ ing steps.0 N/A 85.0 RISER1_OUTLETTMP Normal 50.0 DDR3_P2_G1_TEMP Normal 42.0 C N/A 60.0 C N/A 60.--------.0 PCH_TEMP_SENS Normal 51.0 C N/A 80.--------- P1_TEMP_SENS Normal 49. Min.

0 POWER_USAGE Normal 160 Watts N/A N/A N/A 800 PSU1_POUT Normal 136 Watts N/A 624 N/A 648 PSU1_PIN Normal 160 Watts N/A 720 N/A 744 PSU1_STATUS Normal present PSU2_STATUS Normal absent PSU1_PWRGD Normal good PSU1_AC_OK Normal good Monitoring Leaf Switches Leaf switches in the ACI fab​ ric typ​ ic ​ally equate to the Nexus 9300 fam​ ily of switches (with the ex​ cep​ tion of the Nexus 9336 switch). To ac​ cess the dash​ board to mon​ it​ or switches nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-* . Max.-------. Max.-------. Note that un​ like tra​ di​ tional two or three tier de​ signs where the "WAN" layer at​ taches to ei​ ther the dis​ tri​ bu ​t​ ion/ag​ gre​ ga​ tion layer. The leaf switches pro​ vide first hop con​ nec​ tiv​ ity to any​ thing that at​ taches to the fab​ ric.-------- -------- Min.0 N/A 86.--------- P1_TEMP_SENS Normal 49.--------. Warning Warning Failure Failure ------.306 Monitoring CLI C220-FCH1807V02V /sensor # show psu Name Sensor Reading Units Status -----------------. all ex​ ter​ nal con​ nec​ tiv​ ity is pro​ vided through a set of leaf switches that pro​ vide high den​ sity 10gig or 40gig con​ nec​ tiv​ ity to el​ e​ ments out​ side the fab​ ric.5 C N/A 81. in the ACI fab​ ric. Min.

. Monitoring Switch CPU Utilization There are sev​ eral meth​ ods to poll CPU uti​ liza​ tion and trend it over dif​ fer​ ent pe​ ri​ ods of time. The fol​ low​ ing sec​ tions de​ scribe a few of the meth​ ods avail​ able. REST API Spine and Leaf switches CPU uti​ liza​ tion can be mon​ i​ tored using the fol​ low​ ing classes. re​ me​ di​ ate faults on the switch. and trou​ bleshoot the switch from a hard​ ware per​ spec​ tive. based on the de​ sired timescale and gran​ u​ lar​ ity. and the sub tabs on the right hand side (topol​ ogy/gen​ eral/stats/health/faults/trou​ bleshoot​ ing/his​ tory) can be used to quickly drill down into the var​ i​ ous prop​ er​ ties of the switch to un​ der​ stand how the switch is de​ ployed from a hard​ ware con​ fig​ u​ ra​ tion stand​ point. This class up​ dates every 10 sec​ onds.Monitoring 307 Leaf switch monitoring dashboard No​ tice that the dash​ board for this switch de​ faults to pre​ sent​ ing the health score of the switch at a node level on the dash​ board. proc:SysCPU5min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 5 minute sam​ pling in​ ter​ val.

This class up​ dates every day. proc:SysCPU1mo A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 month sam​ pling in​ ter​ val. This class up​ dates every hour. re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 proc:SysCPU1d A class that rep​ day sam​ pling in​ ter​ val.8% svc_ifc_opflexe 4391 2355 4423 532 0.3% python 4292 3841 134758 28 0. This class up​ dates every day. proc:SysCPU1w A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 week sam​ pling in​ ter​ val.xml? CLI Leaf-1# show proc cpu sort PID Runtime(ms) Invoked uSecs 1Sec Process ----- ----------- -------- ----- ------ ----------- 4012 69510 493837 140 1. http[s]://apic_ip//api/node/class/procSysCPU1d. This class up​ dates every day. proc:SysCPU1qtr A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 quar​ ter sam​ pling in​ ter​ val. This class up​ dates every day.3% t2usd_tor 4065 7239 27609 262 1. This class up​ dates every 5 min​ utes. ACME would like to see the av​ er​ age CPU uti​ liza​ tion of all of the fab​ ric switches over the last day.4% nginx 4067 1911 206 9278 0. proc:SysCPU1year A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 year sam​ pling in​ ter​ val. This class up​ dates every 15 min​ utes.4% svc_ifc_policye 4302 1904 1862 1022 0. proc:SysCPU1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 1 hour sam​ pling in​ ter​ val.308 Monitoring proc:SysCPU15min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem cpu in a 15 minute sam​ pling in​ ter​ val.3% svc_ifc_observe .

.3% svc_ifc_dbgrele 4846 119693 36527 3276 0...3..1% pfmclnt 4864 2361 2812 839 0.0% plog 4866 2792 3269 854 0. Leaf-1# show processes cpu history 1 1 33 1 746554661885833055376572545534667663554785033943645665335644 100 90 80 70 60 50 40 # 30 ## 20 ## 10 # ### ####### ######## # ## ##### ## #### # # #### ## 0..0% policy_mgr 4301 1860 19152 97 0...5........1...1.1% ospf 13606 435 211 2065 0....4. 0 5 0 5 0 5 0 CPU% per second (last 60 seconds) # = average CPU% 5 0 5 .0% snmpd 4297 6667 4542 1467 0..0% bgp 4296 6263 7413 844 0.0% dc3_sensor 4299 8559 8225 1040 0.2% stats_manager 3923 15406 2645 5824 0.3% svc_ifc_eventmg 4310 1802 689 2616 0...3% svc_ifc_confele 4123 1407 251 5606 0....2.....2..4.3.5.0% mcecm In order to ob​ tain a his​ tor​ ic ​al view of CPU uti​ liza​ tion from the CLI it may be nec​ es​ sary to jump into an al​ ter​ na​ tive shell from the switch bash prompt.5......Monitoring 309 4311 1811 1018 1779 0.1% ospfv3 4865 2402 2717 884 0. This shell is called vsh (or v-shell)..0% isis 5025 1611 1743 924 0.

.7....5...4...5...1..3.5..3.3..2......1.6.....5.........6..4....1.5.....4.2.3.1. 0 5 0 5 0 5 0 5 CPU% per hour (last 72 hours) * = maximum CPU% # = average CPU% 0 5 0 5 0 .............2..... 0 5 0 5 0 5 0 5 0 5 CPU% per minute (last 60 minutes) * = maximum CPU% # = average CPU% 1 440 030 100 * 90 * 80 * 70 * 60 * 50 * 40 *** 30 *** 20 *** 10 ### 0...4........5....310 Monitoring 32 13311134111111111311111111131 11111113 1111 11231 1111111 749513800432206328353370732175609342000769025791144192680117 100 90 80 70 60 50 40 * * 30 * ** 20 ** **** * ** ** * * * ** * * * * * *** ** * ** ** ** * 10 ############################################################ 0.......2.......

proc:Sys​ Mem1year A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 year sam​ pling in​ ter​ val. re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ proc:Sys​ Mem1mo A class that rep​ ory in a 1 month sam​ pling in​ ter​ val.xml? . This class up​ dates every 10 sec​ onds. This class up​ dates every day. REST API Spine and Leaf switches mem​ ory uti​ liza​ tion can be mon​ it​ ored using the fol​ low​ ing classes. This class up​ dates every day. ACME would like to mon​ it​ or mem​ ory over the last day. This class up​ dates every day. This class up​ dates every hour. proc:Sys​ Mem1d A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 day sam​ pling in​ ter​ val. This class up​ dates every 15 min​ utes. This class up​ dates every day. proc:Sys​ Mem15min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 15 minute sam​ pling in​ ter​ val. based on the de​ sired timescale and gran​ ul​ ar​ ity. This class up​ dates every 5 min​ utes. proc:Sys​ Mem1w A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 week sam​ pling in​ ter​ val. proc:Sys​ Mem5min A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 5 minute sam​ pling in​ ter​ val. proc:Sys​ Mem1qtr A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 quar​ ter sam​ pling in​ ter​ val. and would use the fol​ low​ ing REST call: http[s]://apic_ip/api/node/class/procSysMem1d. proc:Sys​ Mem1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for Sys​ tem mem​ ory in a 1 hour sam​ pling in​ ter​ val.Monitoring 311 Monitoring Switch Memory Utilization There are sev​ eral meth​ ods to poll mem​ ory uti​ liza​ tion and trend it over dif​ fer​ ent pe​ ri​ ods of time. The fol​ low​ ing sec​ tions de​ scribe a few of the meth​ ods avail​ able.

80 93.21 Processes : 513 total. See http://​ www. Memory usage: 5 minutes: 1.​ cisco.14 2. It is nor​ mal to see a higher % uti​ liza​ tion on some of the mount points in the file sys​ tem hi​ er​ ar​ chy.x. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive.1% user. 16401072K total.312 Monitoring CLI Leaf-1# show system resources Load average: 1 minute: 1.5% kernel.​ html Monitoring File System Health CLI Cur​ rently. The crit​ ic ​al vol​ umes to keep track of in terms of uti​ liza​ tion are /volatile.4% idle 9054020K used. 7347052K free Current memory status: OK SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. 2 running CPU states : 4. 15 minutes: 0. the CLI is only way to mon​ it​ or the uti​ liza​ tion of the file sys​ tem on the leaves.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mibsupport. boot​ flash and logflash Leaf-1# df df: `/nxos/tmp': No such file or directory df: `/var/home': No such file or directory df: `/var/tmp': No such file or directory df: `/nginx': No such file or directory df: `/debugfs': No such file or directory df: `/recovery': No such file or directory df: `/cfg0': No such file or directory df: `/cfg1': No such file or directory df: `/logflash/INXOS_SYSMGR/startup-cfg': No such file or directory df: `/mnt/plog': No such file or directory Filesystem 1K-blocks Used Available Use% Mounted on .

plugin 716800 665728 51072 93% /controller/sbin 93% /dev 4% /controller 716800 665728 51072 none 3145728 336664 2809064 none 51200 2288 48912 none 2097152 682360 1414792 33% /isan/plugin/0/isan/utils none 2097152 682360 1414792 33% /isan/plugin/0/lc/isan/utils none 2097152 682360 1414792 33% /isan/plugin/0/lc/isan/lib none 2097152 682360 1414792 33% /isan/plugin/0/isan/lib none 2097152 682360 1414792 33% /isan/lib none 2097152 682360 1414792 33% /isan/plugin/0/lib none 2097152 682360 1414792 33% /isan/utils rootfs 716800 665728 51072 93% /lc/isan/utils rootfs 716800 665728 51072 93% /lib rootfs 716800 665728 51072 93% /mnt/cfg 60485 5356 52006 10% /mnt/cfg/0 /dev/sda5 11% /dev/shm 5% /etc .old.Monitoring 313 rootfs 512000 1064 510936 1% / rootfs 512000 1064 510936 1% / none 512000 1064 510936 1% /isan none 512000 1064 510936 1% /var none 51200 2288 48912 5% /etc none 51200 108 51092 1% /var/log none 3145728 336664 2809064 11% /dev/shm 512000 0 512000 0% /volatile 7782036 1080636 6306088 15% /bootflash none /dev/sda4 /dev/sda5 60485 5356 52006 10% /mnt/cfg/0 /dev/sda6 60485 5356 52006 10% /mnt/cfg/1 /dev/sda3 120979 13349 101384 /dev/sda7 512000 1064 510936 /dev/sda9 15748508 591216 14357292 4% /mnt/ifc/cfg /dev/sda8 15748508 991204 13957304 7% /mnt/ifc/log /dev/sda8 15748508 991204 13957304 7% /var/log/dme/oldlog /dev/sda9 15748508 591216 14357292 716800 665728 51072 93% /mnt/ifc/cfg/mgmt/opt/controller/sbin 93% /controller/sbin rootfs rootfs /dev/sda8 rootfs /dev/sda4 rootfs /dev/sda9 rootfs rootfs 716800 665728 51072 15748508 991204 13957304 716800 665728 51072 7782036 1080636 6306088 12% /mnt/pss 1% /logflash 4% /controller 7% /data/techsupport 93% /bin 15% /bootflash 716800 665728 51072 15748508 591216 14357292 93% /data/challenge.

314 Monitoring /dev/sda6 60485 5356 52006 10% /mnt/cfg/1 716800 665728 51072 93% /mnt/ifc 15748508 591216 14357292 716800 665728 51072 /dev/sda8 15748508 991204 13957304 /dev/sda3 120979 13349 101384 rootfs /dev/sda9 rootfs rootfs /dev/sda8 none rootfs none 4% /mnt/ifc/cfg 93% /mnt/ifc/cfg/mgmt/opt/controller/sbin 7% /mnt/ifc/log 12% /mnt/pss 716800 665728 51072 15748508 991204 13957304 93% /sbin 1572864 39444 1533420 3% /tmp 716800 665728 51072 93% /usr 7% /data/techsupport 51200 108 51092 15748508 991204 13957304 51200 108 51092 rootfs 716800 665728 51072 93% /var/log/dme rootfs 716800 665728 51072 93% /var/log/dme/nginx rootfs 716800 665728 51072 93% /usr/share/vim /dev/sda7 11811760 375608 10836140 4% /var/log/dme/log /dev/sda8 15748508 991204 13957304 7% /var/log/dme/oldlog none 512000 1064 510936 1% /var/run/mgmt/log none 512000 1064 510936 1% /var/run/utmp 11811760 375608 10836140 40960 8 40952 11811760 375608 10836140 rootfs 716800 665728 51072 none 512000 0 512000 /dev/sda8 none /dev/sda7 none /dev/sda7 1% /var/log 7% /var/log/dme/oldlog 1% /var/log/messages 4% /var/sysmgr 1% /var/sysmgr/startup-cfg 4% /logflash/core 93% /usb 0% /volatile Monitoring CoPP (Control Plane Policing) Statistics CLI CoPP is en​ abled by de​ fault and the pa​ ra​ me​ ters can​ not be changed at this time. To show the CoPP pol​ icy that is pro​ gramed by the sys​ tem use the fol​ low​ ing CLI com​ mand: Leaf-1# show copp policy COPP Class COPP proto COPP Rate COPP Burst . CoPP sta​ tis​ tics are avail​ able through the CLI.

Monitoring 315 mcp mcp 500 500 ifc ifc 5000 5000 igmp igmp 1500 1500 nd nd 1000 1000 cdp cdp 1000 1000 pim pim 500 500 dhcp dhcp 1360 340 lacp lacp 1000 1000 ospf ospf 2000 2000 arp arp 1360 340 lldp lldp 1000 1000 acllog acllog 500 500 stp stp 1000 1000 eigrp eigrp 2000 2000 coop coop 5000 5000 traceroute traceroute 500 500 isis isis 1500 5000 icmp icmp 500 500 bgp bgp 5000 5000 To show drops against spe​ cific CoPP classes. use the fol​ low​ ing CLI com​ mand: Leaf-1# show copp policy stats COPP Class COPP proto COPP Rate COPP Burst AllowPkts AllowBytes DropPkts DropBytes mcp mcp 500 500 0 0 0 0 ifc ifc 5000 5000 195072 161613961 0 0 igmp igmp 1500 1500 3 192 0 0 nd nd 1000 1000 6 564 0 0 cdp cdp 1000 1000 494 140543 0 0 pim pim 500 500 0 0 0 0 dhcp dhcp 1360 340 4 1400 0 0 lacp lacp 1000 1000 0 0 0 0 ospf ospf 2000 2000 0 0 0 0 arp arp 1360 340 1284 90068 0 0 lldp lldp 1000 1000 5029 1717208 0 0 acllog acllog 500 500 0 0 0 0 stp stp 1000 1000 0 0 0 0 eigrp eigrp 2000 2000 0 0 0 0 coop coop 5000 5000 4722 470546 0 0 .

in the APIC GUI. vir​ tual port-chan​ nels. nav​ i​ gate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > In​ ter​ faces > Phys​ i​ cal In​ ter​ faces. . the oper state col​ umn dis​ plays the op​ er​ a​ tional state of the link. To ac​ cess in​ ter​ face sta​ tis​ tics. Note that there are other tabs avail​ able in the work pane that ref​ er​ ence other types of in​ ter​ faces like port-chan​ nels. In the work pane. routed in​ ter​ faces. in the APIC GUI. and then click on the Stats tab in the work pane on the right-hand side. etc. loop​ backs. Physical interface throughput statistics Note that click​ ing on the check icon en​ ables you to se​ lect ad​ di​ tional sta​ tis​ tics that can be graphed sim​ i​ lar to the fol​ low​ ing screen​ shot.316 Monitoring traceroute traceroute 500 500 0 0 0 0 isis isis 1500 5000 17141 2167565 0 0 icmp icmp 500 500 0 0 0 0 bgp bgp 5000 5000 864 73410 0 0 Physical Interface Statistics and Link State GUI To ac​ cess in​ ter​ face link state in​ for​ ma​ tion. nav​ i​ gate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > In​ ter​ faces > Phys​ i​ cal in​ ter​ faces > Eth X/Y.

Monitoring 317 Granular physical interface statistics REST API For cus​ tomers that pre​ fer the REST API in​ ter​ face to poll for in​ ter​ face sta​ tis​ tics. There are sev​ eral such coun​ ters that are avail​ able (e. It is ex​ pected that the reader has a good un​ der​ stand​ ing of the ob​ ject model and is able to nav​ i​ gate through the model to ob​ tain the in​ for​ ma​ tion de​ siered using the ex​ am​ ple below. RX/TX. 5 minute rate. as the chil​ dren can be de​ rived from it. in​ for​ ma​ tion pro​ vided in pre​ ced​ ing sec​ tions and the tools de​ scribed therein An ex​ am​ ple of the base API call for phys​ i​ cal in​ ter​ face sta​ tis​ tics is: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/phys-[eth1/1].). 30 sec​ ond rates. mul​ ti​ cast pack​ ets.g.json . uni​ cast pack​ ets. etc. As a pointer. input/out​ put / du​ plex. sev​ eral ob​ jects are avail​ able. the par​ ent man​ aged ob​ ject is pro​ vided below.

8771 (bia 7c69. From the prior com​ mand.json Vi​ sore al​ lows the op​ er​ at​ or to dig deeper into the hi​ er​ ar​ chi​ cal tree. the op​ er​ at​ or could see chil​ dren of the in​ ter​ face ob​ ject. address: 7c69. output flow-control is off Auto-mdix is turned off Rate mode is dedicated Switchport monitor is off EtherType is 0x8100 EEE (efficient-ethernet) : n/a Last link flapped 04:19:13 Last clearing of "show interface" counters never 1 interface resets 30 seconds input rate 169328 bits/sec. BW 1000000 Kbit. rxload 1/255 Encapsulation ARPA. Dedicated Interface Hardware: 1000/10000/auto Ethernet.f60f. DLY 1 usec reliability 255/255. 115 packets/sec . medium is broadcast Port mode is trunk full-duplex. media type is 1G Beacon is turned off Auto-Negotiation is turned on Input flow-control is off. One of the child ob​ jects in​ cludes the fol​ low​ ing: topology/pod-1/node-101/sys/phys-[eth1/1]/dbgEtherStats&#10 CLI The show in​ ter​ face eth x/y com​ mand can be used to mon​ it​ or in​ ter​ faces from the CLI. to de​ ter​ mine the total ingress bytes on Leaf 101 port Eth1/1. txload 1/255. such as ingress and egress bytes. the ACME op​ er​ at​ or could issue the fol​ low​ ing API call: /topology/pod-1/node-101/sys/phys-[eth1/1]. Other sup​ ported com​ mands in​ clude "show in​ ter​ face port-chan​ nel x/y" Leaf-1# show int e1/1 Ethernet1/1 is up admin state is up.318 Monitoring For ex​ am​ ple.8771) MTU 9000 bytes. 1000 Mb/s. 97 packets/sec 30 seconds output rate 424528 bits/sec.f60f.

the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for in​ ter​ faces See: http://​ www. they have a su​ per​ vi​ sor com​ po​ nent which refers to the CPU com​ plex. viz. output rate 365544 bps. which pro​ vides up​ link con​ nec​ tiv​ ity to the spines.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.​ cisco. 114 pps RX 2474537 unicast packets 8434 multicast packets 2482973 input packets 0 jumbo packets 0 runts 0 storm suppression bytes 0 giants 0 input error 0 watchdog 2 broadcast packets 1686129815 bytes 0 CRC 0 no buffer 0 short frame 0 bad etype drop 0 input with dribble 0 overrun 0 underrun 0 bad proto drop 0 ignored 0 if down drop 712 input discard 0 Rx pause TX 1673907 unicast packets 1674489 output packets 575 multicast packets 7 broadcast packets 455539518 bytes 0 jumbo packets 0 output error 0 collision 0 deferred 0 lost carrier 0 no carrier 0 babble 0 late collision 0 output discard 0 Tx pause SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.x.​ html Module Status Even though the leaves are con​ sid​ ered fixed switches. 134 pps. the NFE (Net​ work For​ ward​ ing En​ gine ASIC) which pro​ vide the front panel ports. and the ALE or ALE2 (Ap​ pli​ ca​ tion Leaf En​ gine ASIC) de​ pend​ ing on the gen​ er​ at​ ion of switch hard​ ware. there are two data plane com​ po​ nents.Monitoring 319 Load-Interval #2: 5 minute (300 seconds) input rate 644416 bps. . The fol​ low​ ing meth​ ods can be used to de​ ter​ mine the sta​ tus of the mod​ ules in the switch. From a for​ ward​ ing per​ spec​ tive.

https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/supslot-1/sup https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/lcslot-1/lc CLI The show mod​ ule com​ mand can be used to ob​ tain the sta​ tus of the base mod​ ule and the up​ link mod​ ule.-----------------.2050 Mod MAC-Address(es) Serial-Num --- -------------------------------------- ---------- 1 7c-69-f6-0f-87-71 to 7c-69-f6-0f-87-ad SAL17267Z9U .320 Monitoring GUI To ac​ cess mod​ ule sta​ tus for the NFE and the CPU com​ plex. in the APIC GUI. REST API The fol​ low​ ing REST API call(s) can be used to mon​ it​ or the state of the su​ per​ vi​ sor and the mod​ ule. To ac​ cess mod​ ule sta​ tus for the ALE/ALE2.1(0.152) 0. Leaf-1# show module Mod Ports Module-Type --- ----- ----------------------------------. in the APIC GUI. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Mod​ ule > Su​ per​ vi​ sor mod​ ules and the sta​ tus of the mod​ ule is dis​ played in the work pane. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Mod​ ule > Line mod​ ules and the sta​ tus of the mod​ ule is dis​ played in the work pane.---------- Model Status 1 48 1/10G Ethernet Module N9K-C9396PX active GEM 12 40G Ethernet Expansion Module N9K-M12PQ ok Mod Sw Hw --- -------------- ------ 1 11.

x.​ html Switch Fan Status The fol​ low​ ing sec​ tion de​ scribes method​ olo​ gies to re​ trieve the sta​ tus of the fan trays on the leaf switches. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-1 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-2 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/ftslot-3 CLI The fol​ low​ ing CLI's can be used to mon​ it​ or the state of the fans on a leaf switch. in the APIC GUI. Leaf-1# show environment fan Fan: . GUI To ac​ cess fan sta​ tus for the leaf switch.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.Monitoring 321 Mod Online Diag Status --- ------------------ 1 pass SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. REST API The fol​ low​ ing REST API call(s) and their child ob​ jects can be used to mon​ it​ or the state of the fans on a leaf switch (note that there are 3 slots on this par​ tic​ ul​ ar switch). See: http://​ www. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for mod​ ules.​ cisco. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Fan Tray and the sta​ tus of the mod​ ules is dis​ played in the work pane.

in the APIC GUI.​ cisco. https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/psuslot-1 https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/ch/psuslot-2 . nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Chas​ sis > Power Sup​ ply Units and the sta​ tus of the mod​ ules is dis​ played in the work pane. Power Supply Status The fol​ low​ ing sec​ tions de​ scribe method​ olo​ gies to re​ trieve the sta​ tus of the power sup​ plies on the leaf switches GUI To ac​ cess power sup​ ply sta​ tus for the leaf switch.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.x.​ html). REST API The fol​ low​ ing REST API call(s) and their child ob​ jects can be used to mon​ it​ or the state of the fans on a leaf switch (note that there are 3 slots on this par​ tic​ ul​ ar switch).322 Monitoring -----------------------------------------------------Fan Model Hw Status -----------------------------------------------------Fan1(sys_fan1) N9K-C9300-FAN1-B -- ok Fan2(sys_fan2) N9K-C9300-FAN1-B -- ok Fan3(sys_fan3) N9K-C9300-FAN1-B -- ok Fan_in_PS1 -- -- unknown Fan_in_PS2 -- -- ok Fan Speed: Zone 1: 0x5f Fan Air Filter : Absent SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for fan trays (http://​ www.

Per module power not available Power Usage Summary: -------------------Power Supply redundancy mode (configured) Non-Redundant(combined) Power Supply redundancy mode (operational) Non-Redundant(combined) Total Power Capacity (based on configured mode) 648 W Total Power of all Inputs (cumulative) 648 W Total Power Output (actual draw) 168 W Total Power Allocated (budget) N/A Total Power Available for additional modules N/A SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for power sup​ plies.Monitoring 323 CLI The fol​ low​ ing CLI com​ mands can be used to mon​ it​ or the state of the fans on a leaf switch: Leaf-1# show environment power Power Supply: Voltage: 12.0 Volts Power Supply Model Actual Total Output Capacity (Watts ) (Watts ) Status ------- ------------------- ----------- ----------- 1 UCSC-PSU-650W 0 W 648 W shut 2 UCSC-PSU-650W 168 W 648 W ok Actual Power Draw Allocated Module Model -------------- Status (Watts ) (Watts ) ----------- ----------- 168 W 456 W Powered-Up -------- ------------------- 1 N9K-C9396PX -------------- fan1 N9K-C9300-FAN1-B N/A N/A Powered-Up fan2 N9K-C9300-FAN1-B N/A N/A Powered-Up fan3 N9K-C9300-FAN1-B N/A N/A Powered-Up N/A . .x.

(T) Telephone.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. (C) DOCSIS Cable Device (W) WLAN Access Point. (O) Other Device ID Local Intf Hold-time Capability Port ID apic2 Eth1/1 120 apic3 Eth1/4 120 4c:4e:35:09:77:2f 5548-2 Eth1/7 120 B Eth1/3 Spine-1 Eth1/49 120 BR Eth4/1 Spine-2 Eth1/50 120 BR Eth4/1 Spine-3 Eth1/51 120 BR Eth1/3 Spine-4 Eth1/52 120 BR Eth1/3 Spine-5 Eth1/53 120 BR Eth1/5 90:e2:ba:4b:fa:d4 .324 Monitoring See: http://​ www. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-X > Pro​ to​ cols > LLDP > Neigh​ bors > eth x/y A full list​ ing of all LLDP neigh​ bors on the in​ ter​ face can be ob​ tained in the work pane. (P) Repeater.​ html LLDP Neighbor Status The APIC pro​ vides a sin​ gle pane of glass to query and de​ ter​ mine all LLDP neigh​ bors in a fab​ ric. In the above work​ flow click​ ing on Neigh​ bors (in​ stead of eth x/y) gives you a list of all LLDP neigh​ bors on the switch. REST API The fol​ low​ ing rest API call can be used to ob​ tain the same in​ for​ ma​ tion: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/lldp/inst/if-[eth1/1] CLI Leaf-1# show lldp neighbors Capability codes: (R) Router.​ cisco. (B) Bridge. (S) Station. To ob​ tain a list of LLDP neigh​ bors on an in​ ter​ face.

V . the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for LLDP.IGMP.Repeater.Supports-STP-Dispute Device-ID Local Intrfce Hldtme Capability Platform Port ID . I . D .x. B .Source-Route-Bridge S .Monitoring 325 Spine-6 Eth1/54 120 BR Eth1/5 Total entries displayed: 9 SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1.Router. T .Trans-Bridge. s .Host. r .Switch.​ cisco. In the above work​ flow click​ ing on Neigh​ bors (in​ stead of eth x/y) gives you a list of all LLDP neigh​ bors on the switch.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support. See: http://​ www.​ html APIC pro​ vides a sin​ gle pane of glass to query and de​ ter​ mine all CDP neigh​ bors in a fab​ ric.Remotely-Managed-Device. CDP Neigh​ bor Sta​ tus: GUI To ob​ tain a list of CDP neigh​ bors on an in​ ter​ face. nav​ ig ​ate to Fab​ ric > In​ ven​ tory > Pod1 > Leaf-X > Pro​ to​ cols > CDP > Neigh​ bors > eth x/y A full list​ ing of all CDP neigh​ bors on the in​ ter​ face can be ob​ tained in the work pane. REST API The fol​ low​ ing rest API call can be used to ob​ tain the same in​ for​ ma​ tion: https://{{apic-ip}}/api/node/mo/topology/pod-1/node-101/sys/cdp/inst/if-[eth1/1] CLI Leaf-1# show cdp neighbors Capability Codes: R .VoIP-Phone. H .

Then click Trou​ bleshoot​ ing in the work pane. To view the same for mod​ ules. See: http://​ www. GUI To view GOLD Di​ ag​ nos​ tic test re​ sults in the GUI for the Su​ per​ vi​ sors.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.​ html GOLD Diagnostic Results GOLD is cov​ ered in greater de​ tail in the Hard​ ware Re​ place​ ment sec​ tion. the fol​ low​ ing SNMP ob​ jects are sup​ ported from an SNMP polling per​ spec​ tive for CDP. click on Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-1 > Chas​ sis > Line Mod​ ules > Slot-x. Then click trou​ bleshoot​ ing in the work pane.x. . GOLD di​ ag​ nos​ tics pro​ vide an easy and quick way for op​ er​ at​ ions teams to con​ firm that bootup and non-dis​ rup​ tive tests that run dur​ ing nor​ mal op​ er​ at​ ions have ex​ e​ cuted prop​ erly.326 Monitoring Services-UCS-A(SSI15450J63) Eth1/5 129 S I s UCS-FI-6248UP Eth1/17 Eth1/7 123 S I s N5K-C5548UP Eth1/3 5548-2(SSI154300VL) SNMP As men​ tioned in the URL for the SNMP ref​ er​ ence guide for ACI re​ lease 1. click on Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-1 > Chas​ sis > Su​ per​ vi​ sor Mod​ ules > Slot-1.​ cisco. as well as the abil​ ity to run on de​ mand di​ ag​ nos​ tics to iso​ late po​ ten​ tial hard​ ware at fault.

34) asic-scratch-------------------> . 31) obfl-acc-----------------------> . . 25) act2-acc-----------------------> . 41) pcie-bus-----------------------> . . . 23) mem-health---------------------> . F = Fail. 33) fpga-reg-chk-------------------> . E = Error disabled) 1) bios-mem-----------------------> . . . . 2) mgmtp-lb-----------------------> . . 6) fabp-prbs: Port 1 2 3 4 5 6 7 8 9 10 11 12 ----------------------------------------.Monitoring 327 CLI Leaf-1# show diagnostic result module all Current bootup diagnostic level: bypass Module 1: 1/10G Ethernet Module (Active) Test results: (. . 24) ssd-acc------------------------> . 30) cons-dev-----------------------> . 26) ge-eeprom----------------------> . 40) rtc-test-----------------------> . = Pass. . . A = Abort. I = Incomplete. U = Untested. 4) nsa-mem------------------------> . 32) nvram-cksum--------------------> . 29) usb-bus------------------------> . 22) cpu-cache----------------------> . . .

.

g. To ac​ com​ plish these tasks re​ quires con​ fig​ ur​ ing a mon​ it​ or​ ing pol​ icy with two dis​ tinct types of poli​ cies. choose Actions > Create Monitoring Policy. Select 5 minutes from the granularity column. choose Ingress. The fol​ low​ ing will pro​ vide an ex​ am​ ple of how sta​ tis​ tics poli​ cies and thresh​ olds can be used to alert the op​ er​ at​ ors when ad​ di​ tional band​ width is re​ quired to a server. d. e. Click on the edit button next to Monitoring Object. choose Monitoring Policies. and select L1 Physical Interface Configuration. Click +. 2 In the Navigation pane. . f. First. ACME's server team has de​ ter​ mined that they would like to mon​ it​ or link uti​ liza​ tion over a 5 minute pe​ riod. and click on update. This en​ ables them to an​ swer ques​ tions about whether a server needs to be up​ graded from 10G to 40G.Monitoring 329 Proactive Monitoring Use Cases Monitoring Workload Bandwidth ACME would like to proac​ tively mon​ it​ or con​ nec​ tions to servers to de​ ter​ mine whether ad​ e​ quate band​ width is avail​ able to a given work​ load. 3 In the Work pane. 1 On the menu bar. ACME will con​ fig​ ure a stats col​ lec​ tion pol​ icy to col​ lect the av​ er​ age uti​ liza​ tion of a link over the de​ sired 5 minute in​ ter​ val. From the Monitoring Object drop-down list. Expand the newly created monitoring policy and select Stats Collection Policy. Cre​ ate an in​ ter​ face mon​ it​ or​ ing pol​ icy. 4 In the Create Monitoring Policy dialog box. or mul​ ti​ ple in​ ter​ faces need to be bonded to pro​ vide more band​ width to the server. c. choose Fabric > Access Policies. and when the av​ er​ age uti​ liza​ tion is above 80%. they would like to raise a fault for the af​ fected in​ ter​ face. Click the edit button next to Stats Type and select Ingress and Egress. perform the following actions: a. From the stats type drop-down list. Next they will con​ fig​ ure a thresh​ old pol​ icy to gen​ er​ ate a fault when the link uti​ liza​ tion ex​ ceeds var​ io ​us thresh​ olds ac​ cord​ ing to the fol​ low​ ing table. b. choose L1 Physical Interface Configuration.

For ex​ am​ ple. these poli​ cies can be fur​ ther cus​ tomized to pro​ vide a nor​ mal value. 1 Click + in the Config Thresholds column. In the Work pane. 2 In the Thresholds for Collection 5 minute window. and look for de​ vi​ at​ ions above or below that nor​ mal value. Select UCS-10G-PG. Next we will con​ fig​ ure a thresh​ old which will alert us if the uti​ liza​ tion ex​ ceeds 80% and as​ sign an de​ fault fault sever​ ity to it. b. Re​ peat these steps for the egress sta​ tis​ tics as well. Fi​ nally. choose Fabric > Access Policies. 3 Click on Ingress Link utilization average value. and clear the warn​ ing when the uti​ liza​ tion falls below 75. In . select the newly created monitoring policy. click +. a. and the reset col​ umn will spec​ ify the the level at which the fault will be cleared. so we will flag this as a warn​ ing. 80% uti​ liza​ tion does not nec​ es​ sar​ ily mean that the ap​ pli​ ca​ tion per​ for​ mance is de​ graded. 4 Enter 0 for the normal value and click the Rising radio button. As an ex​ am​ ple. choose Interface Policies > Policy Groups. EPG Level Statistics The ap​ pli​ ca​ tion owner would like to be able to mon​ it​ or net​ work-re​ lated in​ for​ ma​ tion for their ap​ pli​ ca​ tion. Note: In this ex​ am​ ple. we are in​ ter​ ested in an ab​ solute per​ cent​ age. Ad​ di​ tional lev​ els/sever​ it​ ies can also be spec​ if​ ied if de​ sired. we will be rais​ ing a warn​ ing when the uti​ liza​ tion goes above 80. we will apply the pol​ icy to the UCS-10G-PG 1 2 On the menu bar.330 Monitoring We have now cre​ ated a pol​ icy which will mon​ it​ or as​ so​ ci​ ated in​ ter​ faces and ingress traf​ fic rate at 5 minute in​ ter​ vals. we will as​ so​ ci​ ate the newly cre​ ated pol​ icy with an in​ ter​ face pol​ icy group that rep​ re​ sents the in​ ter​ faces we to mon​ it​ or with this pol​ icy. we will mon​ it​ or the amount of traf​ fic to the web tier of a given ap​ pli​ ca​ tion. The set col​ umn spec​ if​ ies the level at which the fault will be raised. such as the ag​ gre​ gate amount of traf​ fic to a spe​ cific tier. From the Monitoring Policy drop down menu. For our ex​ am​ ple. In this sce​ nario.

Monitoring 331

this ex​
am​
ple, the de​
fault mon​
i​
tor​
ing poli​
cies are ap​
pro​
pri​
ate, and they are sim​
ply ex​
tract​
ing them from the sys​
tem to be con​
sumed ex​
ter​
nally. This in​
for​
ma​
tion is use​
ful in
sce​
nar​
ios such as a new re​
lease being pushed, and to make sure that no traf​
fic anom​
alies are cre​
ated after the push.
This can be ac​
com​
plished by nav​
i​
gat​
ing to the EPG, and se​
lect​
ing the Stats tab:

EPG-level throughput statistics
Ad​
di​
tion​
ally, this in​
for​
ma​
tion can be gath​
ered from the API:
http[s]://apic_ip/api/node/mo/uni/tn-mynewproject/ap-app1/epg-web-epg.xml?querytarget=self&rsp-subtree-include=stats

Monitoring 333

Reactive Monitoring
It is cru​
cial that the ACME op​
er​
a​
tional staff are able to react to any in​
di​
ca​
tion of some​
thing going wrong. If there is a no​
ti​
fi​
ca​
tion that some​
thing has gone wrong, such as a
fault no​
ti​
fi​
ca​
tion, a low health score, or a ticket/re​
port that end-user func​
tion​
al​
ity has
been im​
pacted, knowl​
edge of the avail​
able mon​
i​
tor​
ing tools is im​
por​
tant for the iden​
ti​
fi​
ca​
tion and col​
lec​
tion of ev​
i​
dence. This ev​
i​
dence can then be used to iden​
tify and an​
a​
lyze the root cause of the prob​
lem be​
fore tak​
ing cor​
rec​
tive ac​
tion. For more in​
for​
ma​
tion
re​
gard​
ing faults and health scores please refer to those spe​
cific sec​
tions within this book.
A deep dive into the processes of trou​
bleshoot​
ing is out of the scope of this book.
bleshoot​
ing Cisco Ap​
pli​
ca​
tion Cen​
tric In​
fra​
struc​
ture: An​
a​
lyt​
i​
cal
Please refer to "Trou​
lem solv​
ing ap​
plied to the pol​
icy dri​
ven data cen​
ter" avail​
able at: http://​
datacenter.​
prob​
github.​
io/​
aci-troubleshooting-book/​

Tenant—Troubleshoot Policies
Within the APIC GUI, under each Ten​
ant you can find a Trou​
bleshoot Pol​
icy sec​
tion.
This sec​
tion will allow con​
fig​
ur​
a​
tion of poli​
cies that are spe​
cific to one ten​
ant, and the
mon​
it​
or​
ing of traf​
fic and test con​
nec​
tiv​
ity be​
tween end​
points.
As seen in the image above, the fol​
low​
ing trou​
bleshoot​
ing poli​
cies can be con​
fig​
ured:

SPAN (Switched Port ANalyzer)—Configuration of SPAN and ERSPAN sources
and destinations to be used in external monitoring of Tenant traffic flows

Endpoint-To-Endpoint Traceroute—Configuration of a path validation tool for
verifying validity of communications between Tenant endpoints in an ACI fabric

Atomic Counters—Configuration of a set of customizable counters to collect
and report on information between a definable set of objects. As shown in the
image below, a policy can be configured to collect statistics between EPs,
between EPGs, between EPGs and specific IPs and other special objects, such
as Any or External ​
traffic flows

Fabric—Troubleshoot Policies
For trou​
bleshoot​
ing within the en​
tire fab​
ric, there are the fol​
low​
ing tools and poli​
cies:

334 Monitoring

SPAN (Switched Port Analyzer)—Configuration of SPAN and ERSPAN sources

On-demand Diagnostics—Configuration of a policy for collection of diagnostic

and destinations to be used in external monitoring of fabric traffic flows
information that can be executed at a point in time and which will return a set
of valuable output for investigation

Leaf Nodes Traceroute—Configuration of a path validation tool for verifying

Traffic Map—At-a-glance hotspot map of node-to-node traffic flow in an ACI

validity of communications between ACI fabric nodes
fabric

Enhanced Troubleshooting Wizard
From ver​
sion 1.1, APIC pro​
vides a trou​
bleshoot​
ing graphic tool to find rel​
e​
vant faults
and sta​
tis​
tics, re​
cent changes, and run con​
nec​
tiv​
ity tests in a sim​
ple man​
ner.
It also gives the op​
tion to gen​
er​
ate a re​
port to save the re​
sults so it can be used as a
ref​
er​
ence.

Other Tools

iPing—A troubleshooting tool in the ACI fabric that can be used to verify
reachability of a device connected to the fabric utilizing the fabric as the
pervasive source

Audit Logs—Audit logs are continually collected on all actions taken in an ACI
fabric and can give a quick indication of which user took which actions at what
time

Reactive Monitoring Use Cases
At the end of this chap​
ter, we are going to de​
scribe two sit​
ua
​t​
ions where ACME ran
into is​
sues and how they made use of the tools pre​
vi​
ously de​
scribed.
1

No access to application: An end-user calls and reports that they can no longer
access a web application running within the fabric.

2

Users report that an application running in the fabric is slow or there is a report
of slowness to the web application running within the fabric.

Monitoring 335

Reactive Monitoring Tools
Switch Port Analyzer (SPAN)
SPAN is gen​
er​
ally used in two ways:

Proactively as part of a third party or offline analysis requirement.
Security tools (IDS/IPS, Data Loss Prevention)
Call Recording

Troubleshooting application and networking issues within the fabric.

It may be help​
ful to per​
form a cap​
ture of some par​
tic​
ul​
ar traf​
fic to see what is going on
within the stream of data. Look​
ing through traf​
fic flows will allow in​
ves​
ti​
ga​
tion of
packet and pro​
to​
col level de​
tails, such as traf​
fic re​
sets, mis​
be​
hav​
ing pro​
to​
cols, im​
proper host re​
quests or re​
sponses, or node level com​
mu​
ni​
ca​
tions. This will pro​
vide deeper in​
sight into how de​
vices are using the net​
work than sim​
ple traf​
fic flow and
fab​
ric con​
fig​
ur​
a​
tion re​
view.
Switched Port AN​
al​
yzer, or SPAN, is a stan​
dard fea​
ture that al​
lows copy and repli​
ca​
tion
of traf​
fic to a net​
work an​
al​
yzer for fur​
ther de​
cod​
ing and in​
ves​
ti​
ga​
tion. It can be used to
copy traf​
fic from one or more ports, VLANs, or end​
point groups (EPGs).
The SPAN fea​
ture process is non-dis​
rup​
tive to any con​
nected de​
vices and is fa​
cil​
it​
ated
in hard​
ware, which pre​
vents any un​
nec​
es​
sary CPU load.
SPAN ses​
sions can be con​
fig​
ured to mon​
it​
or traf​
fic re​
ceived by the source (ingress
traf​
fic), traf​
fic trans​
mit​
ted from the source (egress traf​
fic), or both. By de​
fault, SPAN
mon​
it​
ors all traf​
fic, but fil​
ters can be con​
fig​
ured to mon​
it​
or only se​
lected traf​
fic.

Multinode SPAN
APIC traf​
fic mon​
it​
or​
ing poli​
cies can en​
force SPAN at the ap​
pro​
pri​
ate places to copy
traf​
fic from mem​
bers of each End Point Group wher​
ever they are con​
nected. If a mem​
ber moves, APIC au​
to​
mat​
ic
​ally pushes the pol​
icy to the new leaf switch. For ex​
am​
ple,
when a VMo​
tion event re​
lo​
cates an End​
point to a new leaf switch, the SPAN fea​
ture
con​
fig​
ur​
a​
tion au​
to​
mat​
ic
​ally ad​
justs.

336 Monitoring

SPAN Guidelines and Restrictions

Use SPAN for troubleshooting. SPAN traffic competes with user traffic for
switch resources. To minimize the load, configure SPAN to copy only the
specific traffic that you want to analyze.

An l3extLIfP Layer 3 subinterface cannot be configured as a SPAN source. The
entire port must be used for monitoring traffic from external sources.

Tenant and access SPAN use the encapsulated remote extension of SPAN
(ERSPAN) type I, while fabric SPAN uses ERSPAN type II. For information
regarding ERSPAN headers, refer to the IETF Internet Draft at this URL: https:/​
/​
tools.ietf.org/​
html/​
draft-foschiano-erspan-00.

See the Verified Scalability Guide for Cisco ACI document for SPAN-related
limits, such as the maximum number of active SPAN sessions.

Configuring a SPAN Session
This pro​
ce​
dure shows how to con​
fig​
ure a SPAN pol​
icy to for​
ward repli​
cated source
pack​
ets to a re​
mote traf​
fic an​
al​
yzer.
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the tenant.

3

In the Navigation pane, choose Tenant_Name > Troubleshooting Policies >
SPAN > SPAN Destination Groups.

4

In the Work pane, choose Actions > Create SPAN Destination Group.

5

In the Create SPAN Destination Group dialog box, perform the following
actions:
a. In the Name field, enter a name for the SPAN destination group.
b. In the Destination EPG dropdowns, select the destination Tenant,
Application Profile, and EPG.
c. Enter the Destination IP.
d. Enter the Source IP Prefix.
e. Optionally, modify the other fields as needed.
f. Click OK.
g. If needed, add additional destinations.
h. Click Submit.

Monitoring 337

6

Under SPAN, right-click SPAN Source Groups and choose Create SPAN Source
Group.

7

In the Create SPAN Source Group dialog box, perform the following actions:
a. In the Name field, enter a name for the SPAN source group.
b. From the Destination Group drop-down list, choose the SPAN destination
group that you configured previously.
c. In the Create Sources table, click the + icon to open the Create Sources
dialog box.
d. In the Name field, enter a name for the source.
e. In the Direction field, choose the radio button based on whether you want to
replicate and forward packets that are incoming to the source, outgoing from
the source, or both incoming and outgoing.
f. From the Source EPG drop-down list, choose the EPG (identified by
Tenant/ApplicationProfile/EPG) whose packets will be replicated and
forwarded to the SPAN destination. Click OK to save the SPAN source.
g. Click Submit to save the SPAN source group.

Traceroute
Tracer​
oute is a use​
ful fea​
ture in tra​
di​
tional net​
work​
ing. In ACI this fea​
ture is im​
ple​
mented tak​
ing into ac​
count the way the fab​
ric works.
Tracer​
oute sup​
ports a va​
ri​
ety of modes, in​
clud​
ing end​
point-to-end​
point, and leaf-toleaf (tun​
nel end​
point, or TEP to TEP). It dis​
cov​
ers all paths across the fab​
ric, dis​
cov​
ers
point of exits for ex​
ter​
nal end​
points, and helps to de​
tect if any path is blocked.
A tracer​
oute that is ini​
ti​
ated from the ten​
ant end​
points shows the de​
fault gate​
way as
an in​
ter​
me​
di​
ate hop that ap​
pears at the ingress leaf switch.
Note: If tracer​
oute is done from the OS of a con​
nected server or VM, it will show the
hops for the leaves and spines as un​
known, and will keep record​
ing the in​
for​
ma​
tion
after the packet gets out of the fab​
ric. For more pre​
cise in​
for​
ma​
tion, please use tracer​
oute from the APIC (GUI or CLI)

338 Monitoring

Traceroute Guidelines and Restrictions

When the traceroute source or destination is an endpoint, the endpoint must
be dynamic and not static. Unlike a dynamic endpoint (fv:CEp), a static endpoint
(fv:StCEp) does not have a child object (fv:RsCEpToPathEp) that is required for
traceroute.

See the Verified Scalability Guide for Cisco ACI document for traceroute-related
limits.

Traceroute results will display the IP address of the remote node and interface
which it came it came in on. (Browse on the APIC GUI to Fabric | Inventory |
Fabric Management to view the IP address information for correlation.)

Traceroute cannot be used for endpoints that reside in an external EPG.

Performing a Traceroute Between Endpoints
1

On the menu bar, choose Tenants > ALL TENANTS.

2

In the Work pane, choose the tenant.

3

In the Navigation pane, choose Tenant_Name > Troubleshooting Policies >
Endpoint-to-Endpoint Traceroute Policies.

4

In the Work pane, choose Actions > Create Endpoint-to-Endpoint Traceroute
Policy.

5

In the Create Endpoint-to-Endpoint Traceroute Policy dialog box, perform the
following actions:
a. In the Name field, enter a name for the traceroute policy.
b. In the Source End Points table, click the + icon to edit the traceroute source.
c. From the Source MAC drop-down list, choose or enter the MAC address of
the source endpoint and click Update.
d. In the Destination End Points table, click the + icon to edit the traceroute
destination.
e. From the Destination MAC drop-down list, choose or enter the MAC address
of the destination endpoint and click Update.
f. In the State field, click the Start radio button.
g. Click Submit to launch the traceroute.

6

In the Navigation pane or the Traceroute Policies table, click the traceroute
policy. The traceroute policy is displayed in the Work pane.

Monitoring 339

7

In the Work pane, click the Operational tab, click the Source End Points tab,
and click the Results tab.

8

In the Traceroute Results table, verify the path or paths that were used in the trace.
a. More than one path might have been traversed from the source node to the
destination node.
b. For readability, increase the width of one or more columns, such as the Name
column.

Atomic Counters
Atomic Coun​
ters are use​
ful for trou​
bleshoot​
ing con​
nec​
tiv​
ity be​
tween end​
points, EPGs,
or an ap​
pli​
ca​
tion within the fab​
ric. A user re​
port​
ing ap​
pli​
ca​
tion may be ex​
pe​
ri​
enc​
ing
slow​
ness, or atomic coun​
ters may be needed for mon​
it​
or​
ing any traf​
fic loss be​
tween
two end​
points. One ca​
pa​
bil​
ity pro​
vided by atomic coun​
ters is the abil​
ity to place a
trou​
ble ticket into a proac​
tive mon​
it​
or​
ing mode, for ex​
am​
ple when the prob​
lem is in​
ter​
mit​
tent, and not nec​
es​
sar​
ily hap​
pen​
ing at the time the op​
er​
at​
or is ac​
tively work​
ing
the ticket.
Atomic coun​
ters can help de​
tect packet loss in the fab​
ric and allow the quick iso​
la​
tion of the
source of con​
nec​
tiv​
ity is​
sues. Atomic coun​
ters re​
quire NTP to be en​
abled on the fab​
ric.
Leaf-to-leaf (TEP to TEP) atomic coun​
ters can pro​
vide the fol​
low​
ing:

Counts of drops, ad​
mits, and ex​
cess pack​
ets

Short-term data col​
lec​
tion such as the last 30 sec​
onds, and long-term data col​
lec​
tion such as 5 min​
utes, 15 min​
utes, or more

A break​
down of per-spine traf​
fic (avail​
able when the num​
ber of TEPs, leaf or

On​
go​
ing mon​
it​
or​
ing

VPC, is less than 64)

Leaf-to-leaf (TEP to TEP) atomic coun​
ters are cu​
mu​
la​
tive and can​
not be cleared. How​
ever, be​
cause 30 sec​
ond atomic coun​
ters reset at 30 sec​
ond in​
ter​
vals, they can be used
to iso​
late in​
ter​
mit​
tent or re​
cur​
ring prob​
lems.
Ten​
ant atomic coun​
ters can pro​
vide the fol​
low​
ing:

Application-specific counters for traffic across the fabric, including drops,
admits, and excess packets

choose Tenant_Name > Troubleshooting Policies > Atomic Counter Policy. external interfaces. The required identifying information differs depending on the type of source (endpoint.*** Dropped pack​ ets are cal​ cu​ lated when there are less pack​ ets re​ ceived by the des​ ti​ na​ tion than trans​ mit​ ted by the source. Ex​ cess pack​ ets are cal​ cu​ lated when there are more pack​ ets re​ ceived by the des​ ti​ na​ tion than trans​ mit​ ted by the source. perform the following actions: a. choose Tenants > ALL TENANTS. . 3 In the Navigation pane. endpoint groups. choose Actions > Add Policy_Name Policy. Choose or enter the identifying information for the traffic destination. Choose or enter the identifying information for the traffic source. They do not take into ac​ count drops or error coun​ ters in a hard​ ware level.340 Monitoring • Modes include the following: • Endpoint to endpoint MAC address. 5 6 In the Work pane. Configuring Atomic Counters 1 On the menu bar. 2 In the Work pane. Traffic can be measured between a combination of endpoints. b. Note that a single target endpoint could have multiple IP addresses associated with it. or endpoint to endpoint IP address. c. • EPG to EPG with optional drill down • EPG to endpoint • EPG to * (any) • Endpoint to external IP address ***Atomic coun​ ters track the amount pack​ ets of be​ tween the two end​ points and use this as a mea​ sure​ ment. external interface. 4 Choose a policy (traffic topology). endpoint group. In the Add Policy dialog box. In the Name field. or IP address). enter a name for the policy. choose the tenant. and IP addresses.

Monitoring 341 d. choose the new atomic counter policy. choose Fabric > Fabric Policies. e. 2 In the Navigation pane. click + to specify filtering of the traffic to be counted. Traffic Map Low per​ for​ mance and con​ ges​ tion can be iden​ ti​ fied by use of Traf​ fic maps. In the resulting Create Atomic Counter Filter dialog box. The policy configuration is displayed in the Work pane. under the selected topology. choose Troubleshooting Policies > Traffic Map. for example) and by source and destination IP port numbers. Traf​ fic maps make use of atomic coun​ ters to con​ tin​ u​ ously mon​ i​ tor and dis​ play traf​ fic be​ tween leaf switches to help with quick de​ bug​ ging and iso​ la​ tion of ap​ pli​ ca​ tion con​ nec​ tiv​ ity is​ sues. Optional: (Optional) In the Filters table. Click Submit to save the atomic counter policy. . Configuring Traffic Map 1 On the menu bar. 7 In the Navigation pane. choose the Operational tab and choose the Traffic subtab to view the atomic counter statistics. 8 In the Work pane. specify filtering by the IP protocol number (TCP=6.

received. enter the MAC or IP address for the source endpoint. e. click the wrench icon to launch the enhanced troubleshooting wizard. • Last interval and cumulative • Sent.342 Monitoring 3 Set the drop-down menu options to view the source Leaf to destination Leaf traffic paths. 4 Clicking on a cell opens a table with all data for all trails and links. b. c. . click the check box. Run traceroute between EPs. Show any related recent changes in the path between EPs. enter the MAC or IP address for the destination endpoint. In the Name field. 3 In the next screen. Show the faults in the path between the selected EPs. Show relevant statistics for those EPs. and click search. 1 In the menu bar. Optional: If it is an external IP address. d. The En​ hanced Trou​ bleshoot​ ing Wiz​ ard pro​ vides a sin​ gle lo​ ca​ tion that in​ cludes sev​ eral com​ monly used tools and out​ puts re​ quired for trou​ bleshoot​ ing end point con​ nec​ tiv​ ity. Optional: Generate a Report to download the results. Click Start. b. highlighting the affected component. Enhanced Troubleshooting Wizard When trou​ bleshoot​ ing con​ nec​ tiv​ ity be​ tween end​ points within the fab​ ric. 2 In the Create SPAN Destination Group dialog box. In the Destination field. d. f. perform the following actions: a. you can click the following items for more information: a. dropped and excess • All spines and a specific spine switch The per​ cent​ age is shown in rel​ at​ ive terms to all traf​ fic by Source or re​ ceived by Des​ ti​ na​ tion. the En​ hanced Trou​ bleshoot​ ing Wiz​ ard may be used to quickly iden​ tify con​ nec​ tiv​ ity is​ sues. In the Source field. c. Select the Time Window duration for the debug. enter a name for the session. and click search.

256 ms 64 bytes from 10. Configure SPAN between EPs g.154): 56 data bytes 64 bytes from 10. Show the configured contracts between EPs.0.154: icmp_seq=1 ttl=55 time=0.0.59.154 PING 10. Configure Atomic Counters between EPs.245 ms .154 (10. IPing IPing is used to test and val​ i​ date con​ nec​ tiv​ ity within the from leaf node to end​ points within the fab​ ric.254 ms 64 bytes from 10.0. tak​ ing into ac​ count the pri​ vate net​ work. f. IPing is a trou​ bleshoot​ ing tool for net​ work users sim​ i​ lar to the PING com​ mand.59. Using IPing iping [ -V vrf ] [ -c count ] [ -i wait ] [ -p pattern ] [ -s packetsize ] [ -t timeout ] host Syntax Description Examples pod1-leaf1# iping -V overlay-1 10.0.154: icmp_seq=2 ttl=55 time=0.154: icmp_seq=0 ttl=55 time=0.59.59.0.59.Monitoring 343 e.0.59.

23/0. choose Common. 5 Under the History tab.59.344 Monitoring 64 bytes from 10. . 2 In the Work pane.245/0.10. Audit logs can be found in sev​ eral places within the GUI. An out​ age re​ ported on a host or ap​ pli​ ca​ tion in the fab​ ric may need to be tracked. choose Tenants > ALL TENANTS. choose the History tab. when the change was made. and a de​ scrip​ tion of the ac​ tion. choose the Events subtab to view the event log. choose Common. 7 Double-click a log entry to view additional details about the event. or data pulled for an audit re​ quire​ ment.00% packet loss round-trip min/avg/max = 0.256 ms Audit Logs At times it may be re​ quired to view changes which have taken place in the fab​ ric. 0. Viewing Audit Logs Pro​ ce​ dure (ex​ am​ ple for view​ ing audit log on a ten​ ant) 1 On the menu bar.0.241 ms 64 bytes from 10. Wher​ ever a His​ tory tab ap​ pears in the GUI Work pane. Audit logs also record when users logon and lo​ goff. 5 packets received.154: icmp_seq=4 ttl=55 time=0. 3 In the Navigation pane. fil​ tered to show only those events rel​ e​ vant to the cur​ rent GUI con​ text. the audit log can be viewed.59.154 ping statistics --5 packets transmitted. Audit logs are records of who made a change.23 ms --. 4 In the Work pane. choose the Audit Log subtab to view the audit log.0. This pro​ ce​ dure shows how to view ten​ ant events as an ex​ am​ ple.154: icmp_seq=3 ttl=55 time=0.0. 6 Under the History tab.59.

bridge do​ main and Ten​ ant. 4 In the Work pane. 2 Verify that the required contracts are in place between the EPs. hosts are up. within the same sub​ net. 3 In the Navigation pane.Monitoring 345 Reactive Monitoring Use Cases This chap​ ter will show some ex​ am​ ples of how ACME's op​ er​ at​ ions teams can use their ACI mon​ it​ or​ ing tools to react to a few com​ mon pos​ si​ ble sce​ nar​ ios. choose the Operational tab and verify that the endpoint is present. The fol​ low​ ing steps trou​ bleshoot this sit​ ua ​t​ ion: 1 Check that the EPs have been learned by the leaf switches. Note: these ex​ am​ ples as​ sume that basic low-level in​ ves​ ti​ ga​ tion has been done and the issue has been iso​ lated to an issue with traf​ fic flows across the fab​ ric. 2 In the Work pane. etc. choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name. there​ fore the de​ fault gate​ way for both EPs exist in the leaf switches. Ca​ bles and con​ nec​ tiv​ ity have been ver​ if​ ied. VMs are run​ ning. Verify the required contracts are in place between the EPs . choose the tenant. and each EP is con​ nected to dif​ fer​ ent leaf switches. These two End Points (EPs) be​ long to two dif​ fer​ ent End Point Groups (EPGs). choose Tenants > ALL TENANTS. Loss of Connectivity to Endpoint The ACME ap​ pli​ ca​ tion owner has re​ ported that two servers have lost con​ nec​ tiv​ ity. 5 Repeat this procedure for the destination EPG. Check that the EPs have been learned by the leaf switches 1 On the menu bar. processes are run​ ning. The bridge do​ main has uni​ cast rout​ ing en​ abled. mem​ ory and CPU uti​ liza​ tion has been checked.

choose Tenant_Name > Application Profiles > App_Profile_Name > Application EPGs > EPG_Name.346 Monitoring Verify the required contracts are in place between the EPs 1 On the menu bar. choose Tenants > ALL TENANTS. Each leaf has an SVI used as default gateway. choose the tenant. choose the Operational tab. 7 Click on the contract to verify the contents of the contract. noting the direction of the arrows. use iPing from each leaf to the remote endpoint using the default gateway as source. 4 In the Work pane. One method could be to use endpoint-to-endpoint traceroute to show if there are paths available between those endpoints. A good starting place is the Enhanced Troubleshooting Wizard. To check the connectivity to the remote end. a. b. . 3 In the Navigation pane. If this test is successful. 8 Inspect the contents of each filter by examining the contract under the Security Policies folder and verifying that each filter contains the appropriate filter entries. it then might be necessary to use SPAN to verify where the traffic is entering and leaving the fabric. 9 If the endpoints are discovered within each EPG and the contract relationships look correct. In the event that these things seem valid. 5 Choose the Contracts subtab. examine the troubleshooting policies. the connectivity between endpoints and leaf switches is not the problem. Another option inside the fabric could be to utilize the iPing tool to verify connectivity between the default gateway and the endpoints. 10 Alternate techniques are available to validate communications between endpoints within the fabric. and make sure frames have the right format. 6 Check for a relationship between the source EPG and destination EPG. 2 In the Work pane. This displays the filters that are present within that contract.

Atomic coun​ ters can also be de​ ployed to check if there are any drops or ir​ reg​ ul​ ar​ it​ ies be​ tween the de​ fined de​ vices. If every​ thing in the fab​ ric looks fine. they can use EP-to-EP Tracer​ oute to show the path va​ lid​ ity. from the EPs on a OS level to the switches. the end-user is try​ ing to ac​ cess the web por​ tal from a test VM.Monitoring 347 Users Report that an Application Running in the Fabric is Slow ACME test users re​ port slow re​ sponse of the web servers run​ ning in the fab​ ric. SPAN may be used to ver​ ify where the traf​ fic is en​ ter​ ing and leav​ ing the fab​ ric and make sure frames have the right for​ mat. The VM and the web por​ tal be​ long to a dif​ fer​ ent EPGs in the same bridge do​ main and ten​ ant. Cor​ rupted frames could cause drops in dif​ fer​ ent points. First the op​ er​ at​ ions staff should make sure end points are learned by the leaf switches in a con​ sis​ tent way so that the EPs are vis​ ib ​le in the op​ er​ at​ ional tab under the Ten​ ant-> Ap​ pli​ ca​ tion Pro​ file-> EPGs-> Op​ er​ at​ ional Tab Once they ver​ ify end​ points have been learned. packet drops. In this case. or in​ ter​ mit​ tent con​ nec​ tiv​ ity loss. Look​ ing through the Traf​ fic Map can also show an at-a-glance view in the fab​ ric and high​ light any pos​ si​ ble hotspots or con​ ges​ tion in cer​ tain el​ e​ ments of the fab​ ric. This per​ for​ mance issue could be caused due to la​ tency. . Check spe​ cific coun​ ters for CRC er​ rors on the in​ ter​ faces used to trans​ mit EP spe​ cific traf​ fic.

.

349 Scripting .

.

Scripting 351 Section Content • Leveraging Network Programmability Reference to Object Model Programmatic Interfaces REST Read Operations Write Operations Authentication Filters • API Inspector • Development Techniques • POSTman Installation Collections Build Login request Make Query to APIC Make Configuration Change in APIC Use API Inspector for Query Guidance • Cobra SDK and Arya Establish Session Work with Objects Cisco APIC REST to Python Adapter .

352 Scripting • ACI Toolkit ACI Toolkit Applications Endpoint Tracker ACI Lint • GitHub Source Control GitHub "It's on github" .

de​ vel​ op​ ers and even novices an ap​ proach​ able path to​ ward au​ toma​ tion. Given the com​ pre​ hen​ sive​ ness of the pro​ gram​ ma​ bil​ ity fea​ tures avail​ able on ACI. al​ low​ ing peo​ ple to free them​ selves from repet​ it​ ive tasks that could be more eas​ ily ac​ com​ plished by a ma​ chine. Their op​ er​ at​ ions teams can uti​ lize the plethora of in​ for​ ma​ tion con​ tained within the APIC to stream​ line their processes. using com​ mon tools and lan​ guages to pro​ vide net​ work en​ gi​ neers. yield​ ing a higher qual​ ity prod​ uct. This move​ ment from man​ ual to au​ to​ mated op​ er​ at​ ions changed human pro​ duc​ tiv​ ity. gather bet​ ter met​ rics and cor​ re​ late events more ac​ cu​ rately. a more cost-ef​ fec​ tive way to rapidly pro​ vi​ sion in​ fra​ struc​ ture in a timely fash​ ion ac​ cord​ ing to de​ mand. and the con​ sis​ tency pro​ vided by the abil​ ity to au​ to​ mate all of the mov​ ing parts. in​ crease in speed and in​ creased qual​ ity al​ lowed for more work to be done for less money in less time.Scripting 353 Leveraging Network Programmability The in​ dus​ trial rev​ ol​ u​ tion mod​ ern​ ized the tech​ niques used to man​ uf​ ac​ ture goods. The as​ so​ ci​ ated de​ crease in costs. ACI is able to take ad​ van​ tage of all of these ben​ e​ fits by com​ pletely ex​ pos​ ing all of the na​ tive func​ tion​ al​ ity in pro​ gram​ ma​ ble ways. . The in​ evitable move to​ ward au​ toma​ tion in the IT in​ dus​ try has pro​ vided peo​ ple and busi​ nesses a faster way to achieve their de​ sired goals. Though ACME is just get​ ting started with true De​ vOps in their IT or​ ga​ ni​ za​ tion. going from hand pro​ duc​ tion meth​ ods to mech​ an ​ized man​ uf​ ac​ tur​ ing. every​ one can ben​ e​ fit. yield​ ing faster time to res​ ol​ u​ tion and higher cus​ tomer sat​ is​ fac​ tion. they re​ al​ ize that these ben​ e​ fits will allow them to keep up with the pace of busi​ ness. Pro​ gram​ ma​ bil​ ity promises to offer the same out​ come for net​ works as the in​ dus​ trial rev​ ol​ u​ tion did for goods. and yielded more con​ sis​ tency in the con​ fig​ ured re​ sults. ACME's net​ work en​ gi​ neer​ ing and de​ sign teams can ben​ e​ fit from the quick time to pro​ vi​ sion large con​ fig​ ur​ a​ tions.

.

how​ ever the meth​ ods by which these goals may be re​ al​ ized have been more dif​ fi​ cult to grasp. at a va​ ri​ ety of lev​ els that will cater to the level of com​ fort the user has with pro​ gram​ ming. Struc​ tured data that may not be vi​ su​ ally ap​ peal​ ing can be rapidly parsed. ACI also pro​ vides a num​ ber of ac​ cess meth​ ods to read and ma​ nip​ u​ late this data. and also can eas​ ily rep​ re​ sent the full de​ tail that a com​ pre​ hen​ sive ob​ ject-ori​ ented con​ fig​ u​ ra​ tion model may rep​ re​ sent. Tra​ di​ tional net​ work​ ing de​ vices pro​ vide out​ put that is meant for vi​ sual con​ sump​ tion by peo​ ple. In ad​ di​ tion to pro​ vid​ ing this in​ ter​ face into the ob​ ject model. and con​ fig​ u​ ra​ tions are dri​ ven using text input that is sim​ pler for a per​ son to type. Reference to Object Model Representation of the top levels of the Object Model While a com​ pre​ hen​ sive overview of the Ob​ ject Model is out​ side of this book. Ma​ chines are able to more eas​ ily process data that is pro​ vided in some struc​ tured form. from a pro​ gram​ ma​ bil​ ity per​ spec​ tive it is im​ por​ tant to note that every as​ pect of ACI func​ tion​ al​ - .Scripting 355 ACI and Scripting The goals for net​ work pro​ gram​ ma​ bil​ ity are clear. how​ ever these goals stand in con​ trast to an au​ toma​ tion-dri​ ven ap​ proach. all of which use open stan​ dards and open source. ACI uses an ad​ vanced ob​ ject model that rep​ re​ sents net​ work con​ fig​ u​ ra​ tion with ap​ pli​ ca​ tion-based se​ man​ tics which can be con​ sumed and posted against using a well doc​ u​ mented REST API.

CLI. and all other facets of the prod​ uct. The L4-7 de​ vice pack​ age in​ ter​ face al​ lows for ACI to apply pol​ icy to ex​ ist​ ing L4-7 de​ vices that do not have an im​ plicit knowl​ edge of ACI pol​ icy. This is a key as​ pect to the open​ ness of the ACI fab​ ric. The ac​ tual im​ ple​ men​ ta​ tion of de​ vice pack​ ages is done via Python scripts which run on the APIC within a con​ tained ex​ e​ cu​ tion en​ vi​ ron​ ment. and con​ sis​ tency rules that are en​ forced. ma​ nip​ u-​ lated. The north​ bound REST API is re​ spon​ si​ ble for ac​ cept​ ing con​ fig​ ur​ a​ tion. com​ pute in​ te​ gra​ tion. This means that all of the con​ fig​ ur​ a​ tion that can be made on the fab​ ric. As a . South​ bound in​ ter​ faces on APIC allow for the de​ clar​ at​ ive model of in​ tent to be ex​ tended be​ yond the fab​ ric. and also pro​ vides a touch point for au​ toma​ tion tools. and made to cater for the user's needs. which can reach the de​ vice through their na​ tive con​ fig​ ur​ a​ tion in​ ter​ faces. iden​ tity. These de​ vices can be from any ven​ dor. in that pol​ icy can be pro​ grammed once via APIC and then pushed out to hy​ per​ vi​ sors. ex​ ter​ nal net​ work​ ing. so long as the de​ vice has some form of in​ ter​ face which is ac​ ces​ si​ ble via IP. L4-7 de​ vices and po​ ten​ tially more in the fu​ ture. and as such is a crit​ ic ​al as​ pect of the ar​ chi​ tec​ ture for being able to pro​ vide a con​ sis​ tent pro​ gram​ matic ex​ pe​ ri​ ence. as well as ex​ tend​ ing that con​ fig​ ur​ a​ tion into sub​ or​ di​ nate com​ po​ nents. This data is stored within the Man​ age​ ment In​ for​ ma​ tion Tree. vir​ tu​ al​ iza​ tion in​ te​ gra​ tion. This in​ ter​ face is a cru​ cial com​ po​ nent for the GUI and CLI. be that REST. The REST API is a sin​ gu​ lar entry point to the fab​ ric for mak​ ing con​ fig​ ur​ a​ tion changes. with every piece of the model rep​ re​ sented as a pro​ gram​ matic ob​ ject with prop​ er​ ties. This south​ bound ex​ ten​ sion is re​ al​ ized through two meth​ ods: L4-7 De​ vice Pack​ ages and OpFlex. as well as pro​ vid​ ing ac​ cess to man​ age​ ment func​ tions for the con​ troller. into sub​ or​ di​ nate de​ vices. This en​ sures that the con​ fig​ ured state of the model will never get out of hand with stale nodes or en​ tries.356 Scripting ity is en​ com​ passed within the ob​ ject model. SOAP or oth​ ers. pro​ vi​ sion​ ing scripts and third party mon​ it​ or​ ing and man​ age​ ment tools. can be made pro​ gram​ mat​ ic ​ally using the REST API. Programmatic Interfaces APIC is very flex​ ib ​le in terms of how it can ac​ cept con​ fig​ ur​ a​ tion and pro​ vide ad​ min​ is​ tra​ tive and op​ er​ ab ​le states. with​ out the need to in​ di​ vid​ u​ ally con​ fig​ ure those de​ vices. This in​ cludes in​ ter​ nal fab​ ric net​ work​ ing. and every as​ pect can be in​ spected. There are two pri​ mary cat​ e​ gories of in​ ter​ faces that fa​ cil​ it​ ate these func​ tions: the north​ bound REST API and the south​ bound pro​ gram​ matic in​ ter​ faces.

Pay​ loads to and from the REST in​ ter​ face can be en​ cap​ su​ lated through ei​ ther XML or JSON en​ cod​ ing. Stan​ dard REST meth​ ods are sup​ ported on the API. the en​ cod​ ing op​ er​ at​ ion is sim​ ple: the el​ e​ ment tag is the name of the pack​ age and class. OpFlex it​ self does not dic​ tate the in​ for​ ma​ tion model. OpFlex is cur​ rently used to ex​ tend pol​ icy to the Ap​ pli​ ca​ tion Vir​ tual Switch as well as ex​ tend Group Based Pol​ icy into Open​ Stack. . mean​ ing that it can be called zero or more times with​ out mak​ ing any changes (or that it is a read-only op​ er​ at​ ion). so that when a change oc​ curs in the MIT.Scripting 357 user makes changes to ser​ vice graphs or EPG pol​ icy. so that when​ ever in​ for​ ma​ tion is dis​ played. and can be used with any tree-based ab​ stract model in which each node in the tree has a uni​ ver​ sal re​ source iden​ ti​ fier (URI) as​ so​ ci​ ated with it. The REST API is the in​ ter​ face into the MIT and al​ lows ma​ nip​ ul​ a​ tion of the ob​ ject model state. Con​ tain​ ment is de​ fined by cre​ at​ ing child el​ e​ ments. and any prop​ er​ ties of that ob​ ject are spec​ if​ ied as at​ trib​ utes of that el​ e​ ment. In the case of XML. and SDK. and when con​ fig​ ur​ a​ tion changes are made. it is read through the REST API. an event can be sent through a web socket. in​ clud​ ing sta​ tis​ tics. they are writ​ ten through the REST API. and it even pro​ vides a means of sub​ scrib​ ing to push-based event no​ ti​ fi​ ca​ tion. The REST API also pro​ vides an in​ ter​ face through which other in​ for​ ma​ tion can be re​ trieved. and audit events. You can use any pro​ gram​ ming lan​ guage to gen​ er​ ate the mes​ sages and the JSON or XML doc​ um ​ents that con​ tain the API meth​ ods or MO de​ scrip​ tions. The API ac​ cepts and re​ turns HTTP (not en​ abled by de​ fault) or HTTPS mes​ sages that con​ tain JavaScript Ob​ ject No​ ta​ tion (JSON) or Ex​ ten​ si​ ble Markup Lan​ guage (XML) doc​ um ​ents. REST The Cisco APIC REST API is a pro​ gram​ matic in​ ter​ face for Cisco APIC that uses REST ar​ chi​ tec​ ture. mean​ ing that there is no ad​ di​ tional ef​ fect if they are called more than once with the same input pa​ ra​ me​ ters. OpFlex is de​ signed to allow a data ex​ change of a set of man​ aged ob​ jects that is de​ fined as part of an in​ for​ ma​ tional model. and DELETE op​ er​ at​ ions through HTTP. The same REST in​ ter​ face is used by the Cisco APIC com​ mand-line in​ ter​ face (CLI). the de​ vice pack​ age will trans​ late the APIC pol​ icy into API calls on the L4-7 de​ vice. GET. The GET method is nul​ lipo​ tent. The pro​ to​ col is de​ signed to sup​ port XML and JSON (as well as the bi​ nary en​ cod​ ing used in some sce​ nar​ ios) and to use stan​ dard re​ mote pro​ ce​ dure call (RPC) mech​ an ​isms such as JSON-RPC over TCP. faults. GUI. In ACI. which in​ cludes POST. The POST and DELETE meth​ ods are idem​ po​ tent.

or delete op​ er​ a​ tions on the REST API. Next in the re​ quest URI is the lit​ eral string /api. Gen​ er​ ally. • The children key contains a list that defines all the child objects. in which the key is the name of the package and class. The children in this list are dictionaries containing any nested objects. read. The fol​ low​ ing di​ a​ gram shows the syn​ tax for a read op​ er​ a​ tion from the REST API. The first two sec​ tions of the re​ quest URI sim​ ply de​ fine the pro​ to​ col and ac​ cess de​ tails of Cisco APIC. read op​ er​ a​ tions are for an ob​ ject or class. how​ ever. as dis​ cussed ear​ lier. they can be used in cre​ ate. Read Operations After the ob​ ject pay​ loads are prop​ erly en​ coded as XML or JSON. so the next part of the URI spec​ i​ fies . and the value is another nested dictionary with two keys: attribute and children. in​ di​ cat​ ing that the API will be in​ voked.358 Scripting For JSON. up​ date. • The attribute key contains a further nested dictionary describing key-value pairs that define attributes on the object. the de​ f​ i​ n​ i​ tion is re​ peated at all lev​ els of the tree. defin​ ing the uni​ ver​ sal re​ source iden​ ti​ fier (URI) to ac​ cess a cer​ tain re​ source type is im​ por​ tant. • All objects are described as JSON dictionaries. which are defined as described here. REST syntax Be​ cause the REST API is HTTP based. so it is fairly sim​ ple to im​ ple​ ment after it is ini​ tially un​ der​ stood. en​ cod​ ing re​ quires de​ f​ i​ n​ i​ tion of cer​ tain en​ ti​ ties to re​ flect the tree-based hi​ er​ ar​ chy.

Scripting 359 whether the op​ er​ a​ tion will be for an MO or class. though very large con​ fig​ u​ ra​ tions may need to be bro​ ken into smaller pieces if they re​ sult in a dis​ trib​ uted trans​ ac​ tion. or the pack​ age and class name for class-based queries.xml or . This is the only method by which the pay​ load for​ mat is de​ fined (Cisco APIC ig​ nores Con​ tent-Type and other head​ ers). it will be cre​ ated. so that a com​ plete tree can be de​ fined in a sin​ gle com​ mand so long as all ob​ jects are within the same con​ text root and are under the 1MB limit for data pay​ loads for the REST API. The final manda​ tory part of the re​ quest URI is the en​ cod​ ing for​ mat: ei​ ther . it will be up​ dated to re​ flect any changes be​ tween its ex​ ist​ ing state and de​ sired state. so that if an ob​ ject does not al​ ready exist. and if it does al​ ready exist. REST Payload . The con​ text root helps de​ fine a method by which Cisco APIC dis​ trib​ utes in​ for​ ma​ tion to mul​ ti​ ple con​ trollers and helps en​ sure con​ sis​ tency. The next com​ po​ nent de​ fines ei​ ther the fully qual​ i​ fied Dn being queried for ob​ ject-based queries. the con​ fig​ u​ ra​ tion should be trans​ par​ ent to the user. This limit is in place to guar​ an​ tee per​ for​ mance and pro​ tect the sys​ tem under high load. Both cre​ ate and up​ date op​ er​ a​ tions can con​ tain com​ plex ob​ ject hi​ er​ ar​ chies. For the most part. Write Operations Cre​ ate and up​ date op​ er​ a​ tions in the REST API are both im​ ple​ mented using the POST method.json.

be​ cause you can​ not make changes to every ob​ ject of a spe​ cific class (nor would you want to). Filters The REST API sup​ ports a wide range of flex​ i​ ble fil​ ters. <aaaUser name='admin' pwd='in​ sieme'/>. ex​ cept that they al​ ways are tar​ geted at an ob​ ject level. . The cre​ ate or up​ date op​ er​ a​ tion should tar​ get a spe​ cific man​ aged ob​ ject. start​ ing with a ques​ tion mark (?) and con​ cate​ nated with an am​ per​ sand (&). fol​ lowed next by the ac​ tual Dn. Sub​ se​ quent op​ er​ a​ tions on the REST API can use this token value as a cookie named APIC-cookie to au​ then​ ti​ cate fu​ ture re​ quests. if you want to re​ trieve the re​ sults of your POST op​ er​ a​ tion in the re​ sponse. use​ ful for nar​ row​ ing the scope of your search to allow in​ for​ ma​ tion to be lo​ cated more quickly. Mul​ ti​ ple con​ di​ tions can be joined to​ gether to form com​ plex fil​ ters. and aaaRe​ fresh as the Dn tar​ gets of a POST op​ er​ a​ tion.and pass​ word-based au​ then​ ti​ ca​ tion uses a spe​ cial sub​ set of re​ quest URIs. The pay​ load of the POST op​ er​ a​ tion will con​ tain the XML or JSON en​ coded data rep​ re​ sent​ ing the man​ aged ob​ ject the de​ fines the Cisco API com​ mand body.360 Scripting Cre​ ate and up​ date op​ er​ a​ tions use the same syn​ tax as read op​ er​ a​ tions. in​ clud​ ing aaaLo​ gin. Fil​ ter strings can be ap​ plied to POST op​ er​ a​ tions. Their pay​ loads con​ tain a sim​ ple XML or JSON pay​ load con​ tain​ ing the MO rep​ re​ sen​ ta​ tion of an aaaUser ob​ ject with the at​ tribute name and pwd defin​ ing the user​ name and pass​ word: for ex​ am​ ple. The re​ sponse to the POST op​ er​ a​ tion will con​ tain an au​ then​ ti​ ca​ tion token as both a SetCookie header and an at​ tribute to the aaaLo​ gin ob​ ject in the re​ sponse named token. you can pass the rsp-sub​ tree=mod​ i​ fied query string to in​ di​ cate that you want the re​ sponse to in​ clude any ob​ jects that have been mod​ i​ fied by your POST op​ er​ a​ tion. aaaL​ o​ gout. The fil​ ters them​ selves are ap​ pended as query URI op​ tions. for ex​ am​ ple. for which the XPath is /im​ data/aaaLo​ gin/@to​ ken if the en​ cod​ ing is XML. Authentication REST API user​ name. so the lit​ eral string /mo in​ di​ cates that the Dn of the man​ aged ob​ ject will be pro​ vided.

Scripting 361 The fol​ low​ ing query fil​ ters are avail​ able: .

.

The API In​ spec​ tor fur​ ther sim​ pli​ fies the process of ex​ am​ in​ ing what is tak​ ing place on the REST in​ ter​ face as the GUI is nav​ i​ gated by dis​ play​ ing in real time the URIs and pay​ loads. API Inspector . time stamps will ap​ pear along with the REST method. and when in​ for​ ma​ tion is dis​ played on the GUI. and pay​ loads. To get started with the API In​ spec​ tor. the GET re​ quest is dis​ played. There may also be oc​ ca​ sional up​ dates in the list as the GUI re​ freshes sub​ scrip​ tions to data being shown on the screen. vis​ i​ ble at the top right of the Cisco APIC GUI.Scripting 363 API Inspector All op​ er​ a​ tions that are per​ formed in the GUI in​ voke REST calls to fetch and com​ mit the in​ for​ ma​ tion being ac​ cessed. it can be ac​ cessed from the Ac​ count menu. URIs. Click Wel​ come. When a new con​ fig​ u​ ra​ tion is com​ mit​ ted. <user​ name> and then choose the Show API In​ spec​ tor op​ tion After the API In​ spec​ tor is brought up. the API In​ spec​ tor dis​ plays the re​ sult​ ing POST re​ quests.

364 Scripting From the out​ put above it can see that the last logged item has a POST re​ quest with the JSON pay​ load con​ tain​ ing a ten​ ant named Cisco and some at​ trib​ utes de​ fined on that ob​ ject: url: http://​ 172.​ json { "fvTenant": { "attributes": { "name": "Cisco". "children": [ { "fvRsCtx": { "attributes": { "tnFvCtxName": "CiscoVrf". "children": [ { "fvBD": { "attributes": { "mac": "00:22:BD:F8:19:FF".modified" }. { "fvCtx": { "attributes": { "name": "CiscoVrf". "status": "created" .​ 16. "status": "created" }. "name": "CiscoBd". "children": [] } } ] } }.​ 176. "status": "created. "status": "created" }.​ 176/​ api/​ node/​ mo/​ uni/​ tn-Cisco.

"children": [] } } ] } } .Scripting 365 }.

.

Scripting 367 Development Techniques ACI has a num​ ber of meth​ ods for de​ vel​ op​ ing code that can be used by en​ gi​ neers who have vary​ ing lev​ els of com​ fort with pro​ gram​ ming or in​ ter​ act​ ing with pro​ gram​ matic in​ ter​ faces. EPGs and the as​ so​ ci​ ated con​ cepts to con​ nect those to phys​ ic ​al in​ fra​ struc​ ture. or by sav​ ing XML/JSON di​ rectly from the GUI. and using com​ mon freely avail​ able tools. to send this in​ for​ ma​ tion back to the REST API. com​ pre​ hen​ sive data val​ id ​a​ tion. to en​ able users to rapidly cre​ ate ten​ ants. ACI Toolkit is a util​ ity de​ vel​ oped in open-source that ex​ poses the most com​ mon ACI build​ ing blocks. ap​ pli​ ca​ tion pro​ files. The stream​ lined in​ ter​ face pro​ vided makes it very quick to adopt and al​ lows users to begin to quickly de​ velop their ap​ pli​ ca​ tions. The most basic and straight-for​ ward tech​ nique in​ volves sim​ ply tak​ ing in​ for​ ma​ tion gleaned by the API in​ spec​ tor. The most pow​ er​ ful of the de​ vel​ op​ ment tools avail​ able is the Cobra SDK. Vi​ sore. cou​ pling these with the power and flex​ ib ​il​ ity of the ACI pol​ icy lan​ guage and the pop​ ul​ ar Python pro​ gram​ ming lan​ guage to con​ fig​ ure ACI in a pro​ gram​ matic fash​ ion. A step up from this method en​ ables users to use com​ mon ter​ mi​ nol​ ogy and well un​ der​ stood net​ work​ ing con​ structs. such as POST​ man. With a com​ plete rep​ re​ sen​ ta​ tion of the ACI ob​ ject model avail​ able. . Cobra en​ sures that the com​ plete ACI ex​ pe​ ri​ ence is avail​ able to de​ vel​ op​ ers and users alike. and ex​ ten​ sive sup​ port for query​ ing and fil​ ter​ ing.

.

at​ trib​ utes changed in the cap​ tured data and then sent back to the REST API to make the mod​ if​ i​ ca​ tions. to both send and re​ ceive data which may rep​ re​ sent con​ fig​ ur​ a​ tion.Scripting 369 POSTman POST​ man is an open source ex​ ten​ sion for the Chrome web browser. . the col​ lec​ tion can be named “APIC”. the first step is to down​ load the plu​ gin for the Chrome web browser. For this ex​ am​ ple. which is avail​ able at www. In order to do this. For an in​ di​ vid​ ual un​ fa​ mil​ iar with the struc​ ture of REST.​ getpostman. Once the plu​ gin is in​ stalled. POST​ man can be used to in​ ter​ act with the APIC REST in​ ter​ face. the user can switch be​ tween the his​ tory of REST re​ quests sent by POST​ man. ac​ tions. it can be ac​ cessed using the Chrome App launcher. Installation To get started with POST​ man. at which point a popup will ap​ pear prompt​ ing the user to give a name to the col​ lec​ tion. Using the side​ bar. the user should first click into the Col​ lec​ tions tab in the side​ bar. Within the side​ bar. which should then be clicked. Collections A use​ ful post to cre​ ate in a col​ lec​ tion is a basic Login op​ er​ at​ ion. Ini​ tially the user will be pre​ sented with an in​ ter​ face that has two pri​ mary sec​ tions: the side​ bar on the left and the re​ quest con​ struc​ tor on the right. it is very sim​ ple to uti​ lize the API In​ spec​ tor to view what the un​ der​ ly​ ing calls being made to the GUI are for cer​ tain op​ er​ at​ ions.​ com. as well as Col​ lec​ tions of re​ quests that con​ tain com​ mon tasks. cap​ ture those. which pro​ vides REST client func​ tion​ al​ ity in an easy-to-use pack​ age. Fur​ ther​ more POST​ man al​ lows for the re​ quests to be mod​ if​ ied: GUI op​ er​ at​ ions can be made once. a small folder with a plus (+) sign will be​ come vis​ ib ​le. after which the Cre​ ate but​ ton should be clicked. and then use POST​ man to re​ play those op​ er​ at​ ions. pol​ icy and op​ er​ at​ ional state data.

370 Scripting

Build Login request
Now a new re​
quest can be built. In the re​
quest con​
struc​
tor, where “Enter re​
quest URL
here” is shown, the fol​
low​
ing re​
quest URI should be en​
tered, sub​
sti​
tut​
ing APIC​
IP
​AD​
DRESS with the IP of the APIC: https://<APIC-IP>/api/mo/aaaLogin.​
xml
This re​
quest URI will call the Login method in the REST API. Since a Login will re​
quire
post​
ing data, the HTTP method should be changed, which can be done by click​
ing the
drop​
down list to the right of the re​
quest URL. By de​
fault it will be a GET re​
quest, but
POST will need to be se​
lected from the drop down list.
With the POST method se​
lected, it is now pos​
si​
ble to pro​
vide the REST pay​
load. Given
that the data will be sent via REST, the “raw” Re​
quest body se​
lec​
tor should be picked.
Now the pay​
load for the re​
quest can be en​
tered, which will be the sim​
ple XML con​
tain​
ing the user​
name and pass​
word that will be used for au​
then​
ti​
ca​
tion. Note that the URL
is https, mean​
ing that it will be en​
crypted be​
tween the web browser and the APIC, so
no data is being trans​
mit​
ted in clear text. The fol​
low​
ing re​
quest body should be en​
tered, sub​
sti​
tut​
ing the cor​
rect user​
name and pass​
word in place of USER​
NAME and
PASS​
WORD: <aaaUser name='USERNAME' pwd='PASSWORD'/>
With this re​
quest built, it is now pos​
si​
ble to Send the re​
quest, but since this will be a
com​
monly used method, the re​
quest should be added to a col​
lec​
tion. This can be ac​
com​
plished by click​
ing the “Add to col​
lec​
tion” but​
ton be​
neath the re​
quest body. Se​
lect
the “APIC” col​
lec​
tion from the ex​
ist​
ing col​
lec​
tion list, and change the Re​
quest name to
“Login” and then click “Add to col​
lec​
tion”.
By adding the re​
quest to a col​
lec​
tion it can later be quickly ac​
cessed to es​
tab​
lish a login
ses​
sion with APIC as needed.
After com​
plet​
ing the above steps, the re​
quest is ready to be sent. Click the “Send” but​
ton in the re​
quest con​
struc​
tor, and the REST API will re​
turn the XML rep​
re​
sent​
ing a
login ses​
sion with the APIC. The fol​
low​
ing will be vis​
ib
​le in the POST​
man GUI:

Scripting 371

Login request in POSTman

Make Query to APIC
The next re​
quest that will be built is one that queries the APIC for a list of ten​
ants on
the sys​
tem. First click the “Reset” but​
ton in the re​
quest con​
struc​
tor, and pro​
ceed with
the same steps as above, ex​
cept that the re​
quest URL will be shown as https://<APICIP>/api/class/fvTenant.​
xml, and the re​
quest method will be changed to GET.
Click “Add to col​
lec​
tion” and place the re​
quest into the APIC col​
lec​
tion, and for the
name enter “Query APIC for ten​
ants”
Now upon click​
ing “Send”, this re​
quest will re​
turn an XML en​
coded list of ten​
ants in the
re​
sponse body sec​
tion of the con​
struc​
tor pane on the right.

Make Configuration Change in APIC
Mak​
ing a con​
fig​
u​
ra​
tion change will use a POST re​
quest sim​
i​
lar to log​
ging in, how​
ever
the re​
quest URL and body will con​
tain a dif​
fer​
ent set of in​
for​
ma​
tion.

372 Scripting

For this ex​
am​
ple, a new ten​
ant will be cre​
ated in the fab​
ric. Click the “Reset” but​
ton in
the re​
quest con​
struc​
tor to clear out all ex​
ist​
ing re​
quest fields, and use URL
https://<APIC-IP>/api/mo/uni.​
xml and change the method to POST.
In the re​
quest pay​
load, enter the fol​
low​
ing data:

<fvTenant name="Cisco"/>

The re​
quest URL spec​
if​
ies that the tar​
get for this query will be the pol​
icy uni​
verse,
which is where ten​
ants live. With this tar​
get prop​
erly scoped, the data rep​
re​
sent​
ing the
ten​
ant can be pro​
vided in the pay​
load, in this case cre​
at​
ing a ten​
ant named Cisco.

Use API Inspector for Query Guidance
As dis​
cussed in the In​
tro​
duc​
tion to Script​
ing sec​
tion, API in​
spec​
tor can be used as a
guide​
line for build​
ing cus​
tom REST re​
quests. Fur​
ther​
ing on the ex​
am​
ple in that sec​
tion,
where the re​
quest URL is https://<APIC-IP>/api/node/mo/uni/tn-Cisco.​
json and
the pay​
load is the fol​
low​
ing com​
pacted ver​
sion of JSON:
{"fvTenant": {"attributes": {"name": "Cisco", "status": "created"}, "children":
[{"fvBD": {"attributes": {"mac": "00:22:BD:F8:19:FF", "name": "CiscoBd", "status":
"created"}, "children": [{"fvRsCtx": {"attributes": {"tnFvCtxName": "CiscoVrf",
"status": "created,modified"}, "children": [] } } ] } }, {"fvCtx": {"attributes":
{"name": "CiscoVrf", "status": "created"}, "children": [] } } ] } }

It is pos​
si​
ble to mod​
ify the re​
quest URI and pay​
load and sub​
sti​
tute the ten​
ant name
“Cisco” with an​
other ten​
ant name, to cre​
ate an en​
tirely new ten​
ant, with the same con​
fig​
ur​
a​
tion. The new re​
quest URL and JSON would be: https://<APICIP>/api/node/mo/uni/tn-Acme.​
json
{"fvTenant": {"attributes": {"name": "Acme", "status": "created"}, "children":
[{"fvBD": {"attributes": {"mac": "00:22:BD:F8:19:FF", "name": "AcmeBd", "status":
"created"}, "children": [{"fvRsCtx": {"attributes": {"tnFvCtxName": "AcmeVrf",
"status": "created,modified"}, "children": [] } } ] } }, {"fvCtx": {"attributes":
{"name": "AcmeVrf", "status": "created"}, "children": [] } } ] } }

Scripting 373

These val​
ues can be placed into a POST re​
quest in POST​
man, and after es​
tab​
lish​
ing a
Login ses​
sion using the saved Login re​
quest, the new ten​
ant “Acme” can be cre​
ated,
iden​
ti​
cal to the pre​
vi​
ously cre​
ated Cisco ten​
ant, with​
out need​
ing to man​
ua
​lly click
through the GUI or use other man​
ual meth​
ods.

Scripting 375

Cobra SDK and Arya
The com​
plete Cisco ACI Python SDK is named Cobra. It is a pure Python im​
ple​
men​
ta​
tion of the API that pro​
vides na​
tive bind​
ings for all the REST func​
tions and also has a
com​
plete copy of the ob​
ject model so that data in​
tegrity can be en​
sured, as well as sup​
port​
ing the com​
plete set of fea​
tures and func​
tions avail​
able in ACI. Cobra pro​
vides
meth​
ods for per​
form​
ing lookups and queries and ob​
ject cre​
ation, mod​
if​
i​
ca​
tion, and
dele​
tion that match the REST meth​
ods used by the GUI and those that can be found
using API In​
spec​
tor. As a re​
sult, pol​
icy cre​
ated in the GUI can be used as a pro​
gram​
ming tem​
plate for rapid de​
vel​
op​
ment.
The in​
stal​
la​
tion process for Cobra is straight​
for​
ward, and you can use stan​
dard Python
dis​
tri​
bu
​t​
ion util​
it​
ies. Cobra is dis​
trib​
uted on the APIC as an .egg file and can be in​
stalled
using easy_in​
stall, and is also avail​
able on github at http://​
github.​
com/​
datacenter/​
cobra.
The com​
plete doc​
um
​en​
ta​
tion for the Cobra SDK is avail​
able at http://​
cobra.​
readthedocs.​
org/​
en/​
latest/​

Establish Session
The first step in any code that uses Cobra is es​
tab​
lish​
ing a login ses​
sion. Cobra cur​
rently sup​
ports user​
name- and pass​
word-based au​
then​
ti​
ca​
tion, as well as cer​
tifi​
catebased au​
then​
ti​
ca​
tion. The ex​
am​
ple here uses user​
name- and pass​
word-based au​
then​
ti​
ca​
tion.
import cobra.mit.access
import cobra.mit.session
apicUri = 'https://10.0.0.2'
apicUser = 'username'
apicPassword = 'password'
ls = cobra.mit.session.LoginSession(apicUri, apicUser, apicPassword)
md = cobra.mit.access.MoDirectory(ls)
md.login()

376 Scripting

This ex​
am​
ple pro​
vides an MoDi​
rec​
tory ob​
ject named md, which is logged into and au​
then​
ti​
cated for Cisco APIC. If for some rea​
son au​
then​
ti​
ca​
tion fails, Cobra will dis​
play a
cobra.​
mit.​
request.​
CommitEr​
ror ex​
cep​
tion mes​
sage. With the ses​
sion logged in, you are
ready to pro​
ceed.

Work with Objects
Use of the Cobra SDK to ma​
nip​
ul​
ate the MIT gen​
er​
ally fol​
lows this work​
flow:
1

Identify the object to be manipulated.

2

Build a request to change attributes or add or remove children.

3

Commit the changes made to the object.

For ex​
am​
ple, if you want to cre​
ate a new ten​
ant, you must first iden​
tify where the ten​
ant will be placed in the MIT, where in this case it will be a child of the pol​
icy Uni​
verse man​
aged ob​
ject (pol​
Un
​iMo):
import cobra.model.pol
polUniMo = cobra.model.pol.Uni('')

With the pol​
Un
​iMo ob​
ject de​
fined, you can cre​
ate a ten​
ant ob​
ject as a child of pol​
U-​
niMo:
import cobra.model.fv
tenantMo = cobra.model.fv.Tenant(polUniMo, 'cisco')

All these op​
er​
at​
ions have re​
sulted only in the cre​
ation of Python ob​
jects. To apply the
con​
fig​
ur​
a​
tion, you must com​
mit it. You can do this using an ob​
ject called a Con​
fi​
gRe​
quest. Con​
fi​
gRe​
quest acts as a con​
tainer for MO-based classes that fall into a sin​
gle
con​
text, and they can all be com​
mit​
ted in a sin​
gle atomic POST op​
er​
at​
ion.
import cobra.mit.request
config = cobra.mit.request.ConfigRequest()
config.addMo(tenantMo)
md.commit(config)

Scripting 377

The Con​
fi​
gRe​
quest ob​
ject is cre​
ated, then the ten​
antMo ob​
ject is added to the re​
quest,
and then you com​
mit the con​
fig​
ur​
a​
tion through the MoDi​
rec​
tory ob​
ject.
For the pre​
ced​
ing ex​
am​
ple, the first step builds a local copy of the pol​
Uni ob​
ject. Be​
cause it does not have any nam​
ing prop​
er​
ties (re​
flected by the empty dou​
ble sin​
gle
quo​
ta​
tion marks), you don’t need to look it up in the MIT to fig​
ure out what the full Dn
for the ob​
ject is; it is al​
ways known as uni.
If you wanted to post some​
thing deeper in the MIT, where the ob​
ject has nam​
ing prop​
er​
ties, you would need to per​
form a lookup for that ob​
ject. For ex​
am​
ple, if you wanted
to post a con​
fig​
ur​
a​
tion to an ex​
ist​
ing ten​
ant, you could query for that ten​
ant and cre​
ate
ob​
jects be​
neath it.
tenantMo = md.lookupByClass('fvTenant', propFilter='eq(fvTenant.name, "cisco")')
tenantMo = tenantMo[0] if tenantMo else None

The re​
sult​
ing ten​
antMo ob​
ject will be of class cobra.​
model.​
fv.​
Tenant and will con​
tain
prop​
er​
ties such as .dn, .sta​
tus, and .name, all de​
scrib​
ing the ob​
ject it​
self. The lookup​
By​
Class() entry re​
turns an array, be​
cause it can re​
turn more than one ob​
ject. In this
case, the com​
mand is spe​
cific and is fil​
ter​
ing on an fv​
Tenant ob​
ject with a par​
tic​
ul​
ar
name. For a ten​
ant, the name at​
tribute is a spe​
cial type of at​
tribute called a nam​
ing at​
tribute. The nam​
ing at​
tribute is used to build the rel​
at​
ive name, which must be unique
in its local name​
space. As a re​
sult, you can be as​
sured that lookup​
By​
Class on an fv​
Tenant ob​
ject with a fil​
ter on the name al​
ways re​
turns ei​
ther an array of length 1 or
None, mean​
ing that noth​
ing was found.
To en​
tirely avoid a lookup, you can build a Dn ob​
ject and make an ob​
ject a child of that
Dn. This method works only in cases in which the par​
ent ob​
ject al​
ready ex​
ists.
topDn = cobra.mit.naming.Dn.fromString('uni/tn-cisco')
fvAp = cobra.model.fv.Ap(topMo, name='AppProfile')

These fun​
da​
men​
tal meth​
ods for in​
ter​
act​
ing with Cobra pro​
vide the build​
ing blocks
nec​
es​
sary to cre​
ate more com​
plex work​
flows that can help au​
to​
mate net​
work con​
fig​
u-​
ra​
tion, per​
form trou​
bleshoot​
ing, and man​
age the net​
work.

378 Scripting

Cisco APIC REST to Python Adapter
The process of build​
ing a re​
quest can be time con​
sum​
ing, be​
cause you must rep​
re​
sent
the ob​
ject data pay​
load as Python code re​
flect​
ing the ob​
ject changes that you want to
make. Be​
cause the Cobra SDK is di​
rectly mod​
eled on the Cisco ACI ob​
ject model, you
should be able to gen​
er​
ate code di​
rectly from what re​
sides in the ob​
ject model. As ex​
pected, you can do this using a tool de​
vel​
oped by Cisco Ad​
vanced Ser​
vices. The tool is
the Cisco APIC REST to Python Adapter, known as Arya.

Sample REST to Python Adapter
The above fig​
ure clearly shows how the input that might come from the API In​
spec​
tor,
Vi​
sore, or even the out​
put of a REST query and can then be quickly con​
verted into
Cobra SDK code, to​
k​
enized, and reused in more ad​
vanced ways.
In​
stal​
la​
tion of Arya is rel​
a​
tively sim​
ple, and the tool has few ex​
ter​
nal de​
pen​
den​
cies. To
in​
stall Arya, you must have Python 2.7.5 and git in​
stalled. Use the fol​
low​
ing quick in​
stal​
la​
tion steps to in​
stall it and place it in your sys​
tem Python.
git clone https://github.com/datacenter/ACI.git
cd ACI/arya
sudo python setup.py install

addMo(topMo) md.model.model. Some placeholders will ' + 'need to be changed') # list of packages that should be imported for this code to work import cobra. enter: arya.xml The entry will yield the fol​ low​ ing Python code: #!/usr/bin/env python ''' Autogenerated code using arya.fv import cobra.1.mit.fv.1'.commit(c) .MoDirectory(ls) md.request import cobra. 'admin'.Uni('') # build the request using cobra syntax fvTenant = cobra. For ex​ am​ ple.model. you can take XML or JSON rep​ re​ sent​ ing Cisco ACI mod​ eled ob​ jects and con​ vert it to Python code quickly.pol from cobra. name='bob') # commit the generated code to APIC print toXMLStr(topMo) c = cobra.xmlcodec import toXMLStr # log into an APIC and create a directory object ls = cobra.1.access. 'password') md = cobra.py -f /home/palesiak/simpletenant.codec.ConfigRequest() c.mit.Tenant(topMo.session.internal.access import cobra.mit.mit.mit.session import cobra.LoginSession('https://1.model.pol.mit.Scripting 379 After Arya has been in​ stalled.request.login() # the top level object on which operations will be made topMo = cobra.py Original Object Document Input: <fvTenant name='bob'/> ''' raise RuntimeError('Please review the auto generated code before ' + 'executing the output.

You can find this Dn by query​ ing for the ob​ ject in Vi​ sore or in​ spect​ ing the re​ quest URI for the ob​ ject shown in the API In​ spec​ tor.​ 1. it is pur​ posely put in place to help en​ sure that any other to​ ke ​nized val​ ues that must be up​ dated are cor​ rected. Arya may not be able to iden​ tify it through heuris​ tics. which you will need to re​ place with the cor​ rect Dn.​ 1. For ex​ am​ ple. . In this case. the Cisco APIC IP ad​ dress. should be up​ dated to re​ flect the ac​ tual Cisco APIC IP ad​ dress. which de​ faults to 1.380 Scripting The place​ holder rais​ ing a run​ time error must first be re​ moved be​ fore this code can be ex​ e​ cuted. a place​ holder will be pop​ ul​ ated with the text RE​ PLACEME. The same ap​ plies to the cre​ den​ tials and any other place​ hold​ ers.​ 1. Note that if you pro​ vide input XML or JSON that does not have a fully qual​ if​ ied hi​ er​ ar​ chy.

• aci-endpoint-tracker. a num​ ber of ap​ pli​ ca​ tions have been built on top of ACI toolkit. In​ stalling MySQL is out​ side the scope of this book. A sample is shown below: .py—This script creates a web interface that provides a way to present the contents of the database to the operator. which may be daunt​ ing for a user being first in​ tro​ duced to net​ work pro​ gram​ ma​ bil​ ity.​ github.py—This script creates the subscription to the endpoint class and populates the MySQL database • aci-endpoint-tracker-gui. The ac​ it​ oolkit makes avail​ able a sim​ pli​ fied sub​ set of the model that can act as an in​ tro​ duc​ tion to the con​ cepts in ACI. The end​ point tracker ap​ pli​ ca​ tion has two pri​ mary com​ po​ nents that are both python scripts. so we will as​ sume you have ac​ cess to cre​ ate a new data​ base on a MySQL server. In ad​ di​ tion. and give users a way to quickly bring up com​ mon tasks and work​ flows. and other de​ vices). The com​ plete doc​ um ​en​ ta​ tion for ac​ it​ oolkit is avail​ able at http://​ datacenter.Scripting 381 ACI Toolkit The com​ plete ACI ob​ ject model con​ tains many en​ ti​ ties. fire​ walls.​ io/​ acitoolkit/​ ACI Toolkit Applications Endpoint Tracker The end​ point tracker ap​ pli​ ca​ tion cre​ ates a sub​ scrip​ tion to the end​ point class (fvCEp) and pop​ ul​ ates a MySQL data​ base with per​ ti​ nent de​ tails about each end​ point pre​ sent on the fab​ ric (for ex​ am​ ple servers. load bal​ ancers.

The second script enables the content to be viewed in an understandable web UI./aci-endpoint-tracker.py MySQL IP address: 127. The first script.​ 5. aciendpoint-tracker.0. and the table is fil​ tered ac​ cord​ ingly.382 Scripting To launch Endpoint Tracker run the following python scripts. Using the ACI End​ point Tracker is sim​ ply a mat​ ter of in​ putting an IP or MAC ad​ dress into the search field. user@linuxhost:~/acitoolkit/applications/endpointtracker$ . .py.​ 20 has been input into the search field. In the ex​ am​ ple below.0.0.py MySQL IP address: 127.1:5000/ * Restarting with reloader After run​ ning those python scripts you can now bring up a browser and go the Web UI. the IP ad​ dress 192.​ 168.0.1 MySQL login username: root MySQL Password: user@linuxhost::~/acitoolkit/applications/endpointtracker$ python aci-endpointtracker-gui. and the match​ ing re​ sults are dis​ played.0. will actually connect to the APIC and populate the database.1 MySQL login username: root MySQL Password: * Running on http://127.0.

and EPGs. These are rep​ re​ sen​ ta​ tions of where end points are within the ACI fab​ ric and how they re​ late to or de​ pend on other ob​ jects in the en​ vi​ ron​ ment.Scripting 383 One more interesting usage of the endpoint tracker applications is a series of visualizations that can represent how various endpoints are mapped to other fabric constructs including Tenants. Applications. Some sam​ ple screen​ shots are shown below. Pie chart view of endpoint distribution .

384 Scripting Tree view of endpoint relationships Force diagram of endpoint location ACI Lint .

Lint is a term that refers to iden​ ti​ fy​ ing dis​ crep​ an​ cies.checks for com​ mon con​ fig​ ur​ a​ tion er​ rors and re​ ports them to the user. Critical 001: EPG 'default' in tenant 'infra' app 'access' is not assigned security clearance Critical 001: EPG 'x' in tenant 'common' app 'default' is not assigned security clearance Warning 001: Tenant 'Cisco' has no Application Profile. In the sense that ACI pro​ vides in​ fra​ struc​ ture as code.. Warning 006: Contract 'WebServers' in Tenant 'Acme' is not provided at all. Warning 007: Contract 'WebServers' in Tenant 'Acme' is not consumed at all. Warning 002: Tenant 'Books' has no Context. Warning 002: Tenant '3tierapp' has no Context.Scripting 385 ACI Lint In com​ puter pro​ gram​ ming. Warning 004: Context 'oob' in Tenant 'mgmt' has no BridgeDomains./acilint. Warning 006: Contract 'default' in Tenant 'common' is not provided at all. ACI Toolkit pro​ vides just that. or sim​ ple debug tool for com​ mon er​ rors. Warning 005: BridgeDomain 'CiscoBd' in Tenant 'Cisco' has no EPGs. Warning 007: Contract 'outside-to-web' in Tenant 'roberbur' is not consumed at all. Warning 006: Contract 'External' in Tenant 'Acme' is not provided at all.. Processing configuration. Warning 001: Tenant 'Books' has no Application Profile. . A sam​ ple out​ put is pro​ vided here for ref​ er​ ence: user@linuxhost:~/acitoolkit/applications/lint$ . Warning 007: Contract 'default' in Tenant 'common' is not consumed at all.. and then runs a val​ id ​a​ tion that con​ tracts are not used to cross se​ cu​ rity bound​ aries.py Getting configuration from APIC. Warning 005: BridgeDomain 'inb' in Tenant 'mgmt' has no EPGs. Warning 001: Tenant '3tierapp' has no Application Profile. it is ap​ pro​ pri​ ate for ACI to also have a Lint ap​ pli​ ca​ tion.. Warning 001: Tenant 'mgmt' has no Application Profile..sup​ ports the abil​ ity to tag EPGs as ei​ ther se​ cure or in​ se​ cure.. Warning 007: Contract 'External' in Tenant 'Acme' is not consumed at all. Con​ fig​ u​ ra​ tion Is​ sues . ACI Lint is an ap​ pli​ ca​ tion that checks and no​ ti​ fies an op​ er​ at​ or of mis​ con​ fig​ ur​ a​ tion er​ rors in two pri​ mary ca​ pac​ it​ ies: Se​ cu​ rity Is​ sues .

the real value is in the abil​ ity to take these ex​ am​ ples as a start​ ing point. Give it a try! Be sure to share your work back with the com​ mu​ nity! .386 Scripting While the ACI Toolkit pro​ vides some use​ ful tools for an op​ er​ at​ or to im​ me​ di​ ately use. and mod​ ify or ex​ tend these sam​ ples to suit your par​ tic​ ul​ ar needs.

with a cen​ tral server main​ tain​ ing a com​ mon data​ base of source code. and Sub​ ver​ sion (SVN) were used to allow many de​ vel​ op​ ers to work to​ gether. data​ bases and even en​ tire op​ er​ at​ ing sys​ tems. there has been a slow mi​ gra​ tion away from those server-based tools to de​ cen​ tral​ ized util​ it​ ies. and built-in pro​ ject doc​ um ​en​ ta​ tion. or al​ ter​ nately move for​ ward to a newer ver​ sion. which pro​ vides both free and paid host​ ing ser​ vices. se​ cur​ ing ac​ cess to pro​ jects. dis​ trib​ uted ar​ chi​ tec​ ture. the au​ thor of the pop​ ul​ ar open-source op​ er​ at​ ing sys​ tem Linux. and con​ tribute their ef​ forts back into larger pro​ jects. how​ ever the git pro​ to​ col it​ self sup​ ports stor​ age and ver​ sion con​ trol of any file type. not lim​ ited to any spe​ cific lan​ guage. GitHub also pro​ vides tech​ niques for track​ ing is​ sues. While these tools have and con​ tinue to work well. One of the key as​ pects to the suc​ cess of open source is the abil​ ity for many de​ vel​ op​ ers around the globe to col​ lab​ or​ ate to​ gether on a sin​ gle pro​ ject. and more ef​ fi​ cient sup​ port for branches. and has been the mo​ ti​ va​ tion be​ hind many suc​ cess​ ful pro​ jects. with the fore​ most being Git. Git was cre​ ated by Linus Tor​ valds. The com​ bi​ na​ tion of all of these fea​ tures has made GitHub a very com​ mon place for mem​ bers of the com​ mu​ nity to share code with one an​ other. Git also main​ tains an audit of changes that have been made to files and even has ad​ vanced sup​ port for branch​ ing ver​ sions of files to allow mul​ ti​ ple con​ cur​ rent mod​ if​ i​ ca​ tions to a file to take place. in​ clud​ ing con​ sumer soft​ ware. Aside from being a wrap​ per around git. . What is stored on GitHub is usu​ ally source code. Git has a num​ ber of ad​ van​ tages over most other source con​ trol tools: com​ plete local repos​ it​ ory copies. and allow for them to be merged after work ef​ forts have com​ pleted.Scripting 387 GitHub Source Control Open source soft​ ware has been a pop​ ul​ ar move​ ment in IT. that allow for in​ di​ vid​ ua ​ls to col​ lab​ o​ rate with over eight-mil​ lion other GitHub users on pro​ jects to​ gether. Pre​ vi​ ous tools like Con​ cur​ rent Ver​ sion Con​ trol (CVS). The pri​ mary ad​ van​ tage is that the ver​ sion con​ trol pro​ vided by git al​ lows a user to re​ vert a file back to any pre​ vi​ ously stored ver​ sion. so it’s not un​ com​ mon for users to store doc​ um ​en​ ta​ tion or other con​ stantly chang​ ing files in git. GitHub GitHub is a host​ ing plat​ form based around git. build on each other's work. web servers.

how​ ever for those who have not had an in​ tro​ duc​ tion it can seem like a com​ plex topic.​ com and the Mac ver​ sions at http://​ mac.​ github. and for users fa​ mil​ iar with GitHub. and down​ load a graph​ ic ​al-based client to pro​ vide a sim​ pler in​ ter​ face to the com​ mand line-based git tool. or split off a pro​ ject that is avail​ able into their own pri​ vate repos​ it​ ory. GitHub is ac​ tu​ ally a very sim​ ple tool to use. mod​ ify and con​ tribute to the pro​ ject. avail​ able at http://​ sourcetreeapp. which es​ sen​ tially means that the user is propos​ ing their ef​ forts should be pulled back into the orig​ in ​al pro​ ject.​ github. Other com​ mon source con​ trol tools in​ clude Source​ Tree from At​ lass​ ian. so if a new user ac​ cesses the front page of a pro​ ject on Git. . they will typ​ ic ​ally be able to find in​ struc​ tions on how to down​ load and in​ stall the pro​ ject. and the sim​ plest way to begin to take ad​ van​ tage of the in​ for​ ma​ tion stored on GitHub is to sim​ ply ac​ cess a pro​ jects main page and look for the “Down​ load ZIP” but​ ton at the bot​ tom right of any pro​ ject's main page. The process can be that sim​ ple. What a user does with these files will greatly de​ pend on what the con​ tents are. GitHub it​ self has a graph​ ic ​al client with the Win​ dows ver​ sion avail​ able at http://​ windows. If those changes work. this is an in​ vi​ ta​ tion to down​ load. “It’s on github”. how​ ever one of the most highly en​ cour​ aged be​ hav​ iors on GitHub is to pro​ vide clear and ob​ vi​ ous doc​ um ​en​ ta​ tion for a pro​ ject.​ com. right on the first page they see. Once a user has an ac​ count and a github client. they can “Fork”. The re​ sult​ ing down​ loaded file will con​ tain the lat​ est ver​ sion of the files in the pro​ ject. For users look​ ing to con​ tribute back to a pro​ ject. though many more ad​ vanced pro​ jects have stan​ dards and rules for con​ tribut​ ing to those pro​ jects that put in place re​ quire​ ments around how work is com​ mit​ ted back into the pro​ jects. it is pos​ si​ ble to sub​ mit a “Pull” re​ quest. and the user wishes to con​ tribute them back into the orig​ in ​al pro​ ject.388 Scripting "It's on github" A com​ mon phrase in mod​ ern IT jar​ gon is. the next step would be to sign up for an ac​ count on GitHub. which may re​ quire some read​ ing be​ fore at​ tempt​ ing to con​ tribute.​ com. make changes and com​ mit those back to their pri​ vate branch.

389 Hardware Expansion and Replacement .

.

Hardware Expansion and Replacement 391 Section Content • Expanding and Shrinking the Fabric ​Switches Add Connected Switch Pre-Provision Switch Before Connection Decommission Existing Switch APICs Add New APIC Decommission Existing APIC • Hardware Diagnostics and Replacement ​Identify Hardware Failure Resolve Leaf Hardware Failure Resolve APIC Hardware Failure Diagnose Equipment Failures .

.

cable it to all the leaf switches. All devices should connect to the leaf switches. there may be times when you need to re​ place failed hard​ ware. a best-practice ACI fabric is connected in a full mesh topology with every leaf cabled to every spine. and leaf switches are added for more ac​ cess ports. spine switches are added for more through​ put. Both meth​ ods have the same out​ come: an ex​ panded fab​ ric in the mat​ ter of min​ utes. 3 Click on Fabric Membership in the left navigation pane. and pos​ si​ bly APICs. APICs are added as the num​ ber of poli​ cies and end​ points in​ crease. some switches or APICs may need to be de​ com​ mis​ sioned. Ideally. or by prepro​ vi​ sion​ ing the switches by adding their se​ ri​ al num​ bers and later con​ nect​ ing them phys​ ic ​ally to the fab​ ric when the switches ar​ rive. Add Connected Switch To add a switch that has al​ ready been at​ tached to the fab​ ric go through the fol​ low​ ing steps in the APIC GUI: 1 In the case of a leaf switch. Ad​ di​ tion​ ally. 2 On the APIC click on Fabric at the top of the screen. This sec​ tion will also cover de​ com​ mis​ sion​ ing switches. Gen​ er​ ally. and spines should never connect to other spines. This sec​ tion will walk through the op​ er​ at​ ions of adding and re​ mov​ ing switches and APICs in your ex​ ist​ ing ACI fab​ ric. . leaves should never connect to other leaves. In the case of a spine switch. Also. which is dis​ cussed in the Hard​ ware Re​ place​ ments chap​ ter. cable it to all of the spine switches. This is done the same way for both spine and leaf switches.Hardware Expansion and Replacement 393 Expanding and Shrinking the Fabric ACME may de​ cide to ex​ pand their ACI fab​ ric as their data cen​ ter grows. Switches There are two ways switches can be added to ACME's ex​ ist​ ing fab​ ric: by dis​ cov​ er​ ing the switches au​ to​ mat​ ic ​ally in the APIC after they have been ca​ bled to the fab​ ric. which means adding new leaf and spine switches. Adding APICs will also be cov​ ered.

This is commonly used to identify the physical location of the switch in the data center. choose Actions > Create Add Fabric Node Member. refer to the fol​ low​ ing white paper: . 6 Click Submit. You will need to know the se​ ri​ al num​ ber of the switch you will re​ ceive to pre-pro​ vi​ sion. number leaf nodes starting with 101. 7 Repeat this process for all new switches connected to the fabric. Lower numbers are reserved for APICs. you can also pre-pro​ vi​ sion fab​ ric poli​ cies. you'll see a node with a serial number but no Node ID or Node Name configured. As a best practice.394 Hardware Expansion and Replacement 4 When the new switch appears. 2 In the Navigation pane. and spine nodes with 201. Assign a Node ID and a Switch Name. 5 Click Submit. but the switch will im​ me​ di​ ately be​ come a mem​ ber of the fab​ ric once it ar​ rives and is ca​ bled. choose Fabric. b. choose Fabric Membership. Double click the switch and assign a Node ID and a Node Name. The fol​ low​ ing steps walk you through switch pre-pro​ vi​ sion​ ing for both leaves and spines: 1 On the menu bar. The new entry in the Fab​ ric Mem​ ber​ ship win​ dow will show Un​ sup​ ported in the Role col​ umn until the switch is ac​ tu​ ally con​ nected to the fab​ ric. Lower numbers are reserved for APICs. Note: Repeat this process for all switches you wish to pre-provision. and spine nodes with 201. enter the serial number of the switch that will be arriving. Pre-Provision Switch Before Connection Pre-pro​ vi​ sion​ ing a switch is a handy op​ er​ at​ ionally proac​ tive step to get the switch reg​ is​ tered be​ fore it even ar​ rives to your data cen​ ter. To be proac​ tive. 4 In the Create Add Fabric Node Member dialog box. 3 In the Work pane. perform the following actions: a. 5 Optionally. As a best practice. In the pop-up window. add a Rack Name name. number leaf nodes starting with 101. Fab​ ric poli​ cies are cov​ ered in the Fab​ ric Con​ nec​ tiv​ ity chap​ ter. For more in​ for​ ma​ tion on pre-pro​ vi​ sion​ ing poli​ cies.

3 Click the switch to decommission in the Navigation pane. There are two types of switch de​ com​ mis​ sion​ ing: Reg​ ul​ ar. The Re​ move from Con​ troller op​ tion will com​ pletely re​ move the switch from the ACI fab​ ric and all APICs. APICs Add New APIC Be​ fore mak​ ing any changes to an APIC clus​ ter. Reg​ u​ lar de​ com​ mis​ sion​ ing can be used for main​ te​ nance and es​ sen​ tially si​ lences the switch from re​ port​ ing faults and send​ ing SNMP in​ for​ ma​ tion as a tem​ po​ rary so​ lu​ tion. Chose the Actions > Decommission. choose Fabric. The switch will no longer show up in the fab​ ric mem​ ber​ ship as a reg​ is​ tered node and the in​ fra​ struc​ ture VTEP IP ad​ dresses it was as​ signed will be re​ moved. The switch will show up under the Dis​ abled In​ ter​ faces and De​ com​ mis​ sioned Switches folder in the left nav​ i​ ga​ tion pane. and Re​ move from Con​ troller. as the switch will not for​ ward traf​ fic after de​ com​ mis​ sion​ ing. 2 In the Navigation pane. en​ sure each APIC in the clus​ ter is fully fit and change the clus​ ter size to re​ flect the new con​ troller you are adding to the clus​ ter. c. Click the General tab. Per​ form the fol​ low​ ing steps to ver​ ify clus​ ter health: .Hardware Expansion and Replacement 395 http://​ www.​ cisco. b.​ html#_​ Toc405844675 Decommission Existing Switch De​ com​ mis​ sion​ ing a switch can ei​ ther re​ move it from the fab​ ric en​ tirely. a.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c11-731960. En​ sure you do not have any de​ vices con​ nected. choose Inventory > Pod 1. In the pop-up. Per​ form the fol​ low​ ing steps from the APIC GUI to de​ com​ mis​ sion a switch from the ACI fab​ ric: 1 On the menu bar. 4 Click Submit. choose either Regular or Remove from Controller. while keep​ ing the switch's node ID and fab​ ric mem​ ber​ ship. or re​ move a switch tem​ porar​ ily to per​ form main​ te​ nance.

Click the Cluster folder.396 Hardware Expansion and Replacement 1 2 On the menu bar. Verify that the APIC controllers are in operational state. 2 In the Navigation pane. choose Controllers. In the Navigation pane. The APIC controllers are added one by one and displayed in the sequential order starting with N + 1 and continuing until the target cluster size is achieved. a. choose System Controllers.​ cisco.​ cisco. choose System > Controllers. b. Verify every controller shows Fully Fit under the Heath State column. choose Actions > Change Cluster Size. c. 2 In the Navigation pane. If any of the APICs are not fully fit. . choose System > Controllers. Note: A cluster size of two is not permitted as that does not allow for quorum amongst APICs.​ html 1 On the menu bar. choose Controllers > APIC_Name > Cluster. a. choose Controllers > APIC_Name > Cluster. Expand the first APIC in the folder. a.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ troubleshooting/​ b_​ APIC_​ Troubleshooting. Per​ form the fol​ low​ ing steps to add a new APIC to the clus​ ter: 1 Install and stage the APIC by connecting it to the fabric by following the hardware installation guide: http://​ www. b. 3 In the Work pane.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ application-policy-infrastructure-controller-apic/​ products-installationguides-list.​ html Per​ form the fol​ low​ ing steps to change the APIC clus​ ter size: 1 On the menu bar. Change the Target Cluster Administrative Size to reflect the new APIC(s) being added. refer to the fol​ low​ ing trou​ bleshoot​ ing guide: http://​ www. and the health state of each controller is Fully Fit from the Cluster folder under the new controller. 4 Click Submit.

Verify the APIC no longer appears in the Cluster folder under any of the remaining APICs. click the APIC to be decommissioned.Hardware Expansion and Replacement 397 Note: It will take sev​ eral min​ utes for the APICs to syn​ chro​ nize and join the new APIC to the clus​ ter. choose System > Controllers. 4 Click Submit. Per​ form the fol​ low​ ing steps to de​ com​ mis​ sion an APIC that needs to be re​ moved from the fab​ ric: 1 On the menu bar. a. 6 In the Work pane. 5 In the Navigation pane. Decommission Existing APIC When de​ com​ mis​ sion​ ing APICs. choose Controllers > APIC_Name > Cluster. You can​ not de​ com​ mis​ sion a pow​ ered on fully fit APIC. For ex​ am​ ple. be​ fore mak​ ing any changes to an APIC clus​ ter. APIC5 must be de​ com​ mis​ sioned be​ fore APIC4. choose Actions > Actions > Decommission. 3 In the Work pane. a. Note: A cluster size of two is not permitted as that does not allow for quorum ​ amongst APICs. 2 In the Navigation pane. Change the Target Cluster Administrative Size to reflect the new APIC(s) being added. Fab​ ric op​ er​ at​ ion will con​ tinue nor​ mally. choose Controllers > APIC_Name > Cluster. Note: Select an APIC that is NOT being decommissioned. they must be de​ com​ mis​ sioned se​ quen​ tially in re​ verse order. . choose Actions > Change Cluster Size. Again. Note: In the main pane. en​ sure each APIC in the clus​ ter is fully fit with the ex​ cep​ tion of the faulty APIC being de​ com​ mis​ sioned.

.

the hot-swap​ pable com​ po​ nents en​ able them to re​ place failed hard​ ware quickly and non-dis​ rup​ tively. If ACME ever ex​ pe​ ri​ ences some sort of power surge or sees a com​ po​ nent of their switches go bad. Identify Hardware Failure When a hard​ ware fail​ ure oc​ curs in the fab​ ric. there are sev​ eral hot-swap​ pable com​ po​ nents on both the leaf and spine switches in ad​ di​ tion to a few com​ po​ nents that are fixed on the chas​ sis. Re​ gard​ ing hard​ ware. Ex​ am​ ples of hot-swap​ pable com​ po​ nents on both the leaves and spines in​ clude: • Power supplies • Fan trays Ex​ am​ ples of hot-swap​ pable com​ po​ nents on the spines in​ clude: • Power supplies • Fan trays • Supervisors • System Controller cards • Linecard modules De​ spite sig​ nif​ ic ​ant ad​ vances in the above com​ po​ nents that re​ duce the MTBF. In such an event. or a com​ bi​ na​ tion of the two that ne​ ces​ si​ tates a leaf re​ place​ ment. there is al​ ways the pos​ si​ bil​ ity of a fail​ ure on a leaf switch ei​ ther in switch hard​ ware or soft​ ware.Hardware Expansion and Replacement 399 Hardware Diagnostics and Replacement The Cisco Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture (ACI) fab​ ric em​ ploys a com​ bi​ na​ tion of key soft​ ware and hard​ ware fea​ tures that are specif​ ic ​ally de​ signed to re​ duce the mean time be​ tween fail​ ures (MTBF) and the mean time to re​ pair (MTTR). the state​ less na​ ture of the ACI fab​ ric pro​ vides sig​ nif​ ic ​ant ad​ van​ tages to ad​ min​ is​ tra​ tors from an op​ er​ at​ ions stand​ point. faults are raised in the sys​ tem dash​ board and are pre​ sented to the ad​ min​ is​ tra​ tor. For cases where there is a com​ po​ nent level .

you want to re​ move the failed switch and pro​ vi​ sion a new switch as part of the fab​ ric. choose Pod 1. sys​ log mes​ sages and SNMP traps are gen​ er​ ated. such as Splunk. such as ZenOSS. How​ ever. The first step in re​ plac​ ing the failed switch .400 Hardware Expansion and Replacement fail​ ure with re​ dun​ dant com​ po​ nents pre​ sent in the sys​ tem. Resolve Leaf Hardware Failure As an ex​ am​ ple of a leaf fail​ ure. there is a pos​ si​ bil​ ity that the hard​ ware has failed. The leaf and spine switches in the ACI fab​ ric also sup​ port tra​ di​ tional meth​ ods of de​ tect​ ing fail​ ures. If re​ sponses are not re​ ceived from the switch in a cer​ tain time​ frame. Note: The pod health displays in the Work pane and is zero. or SNMP mes​ sages can be sent to NMS sys​ tems. choose Fabric > Inventory. a Nexus 9396 leaf switch that is a part of the fab​ ric is un​ reach​ able. Ex​ am​ ples of hard​ ware events that gen​ er​ ate sys​ log mes​ sages and SNMP traps in​ clude: • Linecard failure on a spine switch • Supervisor failure on a spine switch • System controller failure on a spine switch • Power supply or fan failures on a leaf or a spine switch While Cisco Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) is a cen​ tral point of man​ age​ ment for the en​ tire fab​ ric. the APICs them​ selves do not have the abil​ ity to gen​ er​ ate alerts using SNMP or sys​ log. op​ er​ at​ ions teams can lever​ age their ex​ ist​ ing NMS tools. while the leaf and spine switches re​ port SNMP and Sys​ log mes​ sages for com​ po​ nent level fail​ ures. Log​ ging mes​ sages can be sent to sys​ log servers. To view the node health score: 1 On the menu bar. 2 In the Navigation pane. You can use the GUI to de​ ter​ mine the node health to con​ firm that the leaf has failed. For ex​ am​ ple a power sup​ ply fail​ ure on the APIC will not gen​ er​ ate an SNMP or sys​ log mes​ sage and must be mon​ it​ ored and re​ me​ di​ ated using the APIC dash​ board. such as SNMP polling at a set in​ ter​ val. per​ haps due to a hard​ ware fail​ ure on the up​ link mod​ ules. After con​ firm​ ing that the leaf node has failed. to pro​ vide alert​ ing.

2 In the Navigation pane. and syslog servers and the ACLs that are associated with each of them. 4 Stage the device with the right configuration file and eliminate any errors. To de​ com​ mis​ sion and recom​ mis​ sion a switch: 1 On the menu bar. 6 Bring up links one by one and verify if data traffic is flowing correctly.Hardware Expansion and Replacement 401 is to get the failed switch's unique ID (node ID). choose Fabric > Inventory. . Each node is as​ signed an ID in the fab​ ric. 2 Load the correct version of code. 3 Attempt to obtain the latest version of configurations from a configuration repository server. In an ACI fab​ ric. 5 Copy the old configuration over to the switch. expand Pod 1."0") In the case of a tra​ di​ tional op​ er​ at​ ions model where each switch was man​ aged as an in​ de​ pen​ dent en​ tity. For example. NTP. 3 Right click the failed node and choose Decommission. as shown in the fol​ low​ ing ex​ am​ ple: {{protocol}}://{{apic}}/api/class/topSystem. you can take ad​ van​ tage of the state​ less na​ ture of the hard​ ware to in​ stan​ ti​ ate the log​ ic ​al con​ fig​ ur​ a​ tion pro​ files.cur. choose Fabric > Inventory. the fol​ low​ ing high-level pro​ ce​ dure re​ places the switch: 1 Stand up the replacement switch. You can also use a sin​ gle REST API call to pe​ ri​ od​ ic ​ally poll for a full list of nodes that are at or below a cer​ tain health level. which is the ref​ er​ ence ob​ ject that al​ lows a re​ place​ ment switch with a new se​ ri​ al num​ ber to in​ herit the same state​ less con​ fig​ ur​ a​ tion that was as​ signed to the old node. 2 In the Navigation pane.xml?rsp-subtree-include=health&rspsubtree-filter=le(healthInst. choose Fabric Membership. Re​ plac​ ing the node is as sim​ ple as de​ com​ mis​ sion​ ing the switch and recom​ mis​ sion​ ing it. update the AAA. To view the fab​ ric node IDs using the GUI: 1 On the menu bar.

choose Unreachable Nodes. 7 The new leaf appears with a node ID of 0 and an IP address of 0. 9 Choose Actions > Commission Switch. enter the old node's ID. In most cases. 10 When prompted for the node ID. 3 Find the new switch and record its name and node ID. 8 In the Work pane.0. 6 In the Navigation pane. you can also reuse the same leaf name. which is automatically set to the latest firmware version. To view which ver​ sion of the firmware that the APIC will load: 1 On the menu bar. When prompted for the name and node ID.402 Hardware Expansion and Replacement 4 Replace the failed leaf switch with the new leaf switch. To view the un​ reach​ able nodes: 1 On the menu bar. You can get the name and ID by view​ ing the un​ reach​ able nodes. 2 In the Navigation pane. 5 On the menu bar. starting with step 5. 2 In the Navigation pane. Note: In the Work pane. click on the new leaf.0. When the new leaf switch is com​ mis​ sioned suc​ cess​ fully. 11 Click Update. choose Fabric Membership. .0. enter the information that you recorded in this procedure. choose Fabric > Inventory. choose Fabric > Inventory. the APIC au​ to​ mat​ ic ​ally loads the cor​ rect ver​ sion of the firmware into the leaf. choose Admin > Firmware. you can see the target firmware version. If the new switch is not op​ er​ at​ ional. choose Fabric Node Firmware > Firmware Groups > All. 4 Repeat the "To decommission and recommission a switch" procedure. the new switch's name and node ID are dif​ fer​ ent from the name and ID that you en​ tered.

bridge do​ mains. ensure that images are copied to the standby supervisor in case of a full chassis replacement by using the command: # copy bootflash:aci_image bootflash://sup-standby/ 6 Configure the switch not to boot from Cisco NX-OS. by lever​ ag​ ing the state​ less ob​ ject mod​ el​ ing that re​ places the tra​ di​ tional run​ ning con​ fig​ u​ ra​ tion on a de​ vice. 2 Set the IP address on the mgmt0 interface to allow connectivity between the switch and the APIC. SNMP. In the event that the re​ place​ ment switch runs stand​ alone NX-OS soft​ ware in​ stead of ACI switch soft​ ware. ACLs. such as AAA. NTP. switch(config)# no boot nxos 7 Save the configuration.Hardware Expansion and Replacement 403 In ad​ di​ tion. you might need to copy the ACI switch soft​ ware image to the switch in ques​ tion. switch(config)# copy running-config startup-config . sys​ log. and EPGs. APIC au​ to​ mat​ i​ cally loads the cor​ rect run​ ning con​ fig​ u​ ra​ tion onto the de​ vice. 3 Enable SCP services: # feature scp-server 4 Copy the firmware image from APIC to the switch: # scp -r /firmware/fwrepos/fwrepo/switch_image_name admin@switch_ip_address:switch_image_name 5 For dual supervisor systems. To copy the ACI switch soft​ ware image to the switch: 1 Connect to the switch console.

switch(config)# boot aci bootflash:aci-image-name 9 Verify the integrity of the file by displaying the MD5 checksum. admin@apic1:aci> openssl asn1parse /securedata/ssl/server. switch(config)# reload 11 Log in to the switch as an administrator. . From the GUI of an op​ er​ at​ ional APIC. Login: admin 12 Verify whether you must install certificates for your device. the correct certificates are installed. Resolve APIC Hardware Failure In this ex​ am​ ple.crt 13 Look for PRINTABLESTRING in the command output. switch(config)# show file bootflash:aci-image-name md5sum 10 Reload the switch.404 Hardware Expansion and Replacement 8 Boot the active and standby supervisor modules with the ACI image. If "Cisco Manufacturing CA" is listed. the switch should ap​ pear as an un​ man​ aged fab​ ric node when con​ nected to the fab​ ric. Once you have con​ firmed the that cer​ tifi​ cate is in​ stalled and the switch is in ACI mode. 1 On the menu bar choose System > Controllers. you must iden​ tify and re​ me​ di​ ate a hard​ ware fail​ ure on one of the APICs in your APIC clus​ ter. If something else is listed. contact TAC to generate and install the correct certificates for your device.

9 In the Work pane. click the new APIC to select it. 7 Proceed through the setup script and enter the values of the failed APIC that you recorded in step 3. choose Controllers > apic_name > Cluster.Hardware Expansion and Replacement 405 2 In the Navigation pane. 3 Record the fabric name. . you should see the failed APIC in the "Unavailable" operational state. Note: In the Work pane. and the TEP address space. 4 In the Work pane. Failure to configure the APIC with the same settings could result in the fabric entering a partially diverged state. The new APIC should boot to the initial setup script. click the failed APIC to select it. The APIC changes to an "Out of Service" admin state. 8 Once the new APIC finishes booting. in the Navigation pane. You can choose any APIC. node ID of the failed APIC. Decommissioning a Failed APIC 6 Remove the failed APIC from your rack and install the replacement. This information is also available through the acidiag avread command on APIC’s CLI. choose Controllers > apic_name > Cluster. target size. 5 Choose Actions > Decommission.

Diagnose Equipment Failures The ACI fab​ ric pro​ vides bootup. . Recommissioning an APIC 11 The new APIC will receive an IP address. Waiting for Cluster Convergence 12 On the command line of the new APIC.406 Hardware Expansion and Replacement 10 Choose Actions > Commission. It might take 5 to 10 minutes for this to occur. you can verify that it has joined the fabric by logging in using the credentials that are configured for the rest of the fabric. run​ time and on-de​ mand di​ ag​ nos​ tics to help as​ sess the hard​ ware health of sev​ eral sub-sys​ tems on each leaf and spine switch. which will be reflected in the APIC GUI. The new APIC might also cycle between the Available and Unavailable operational states before becoming Fully Fit.

To look at the de​ fault di​ ag​ nos​ tic poli​ cies. card boots up. Comes with default set of tests that can be modified. Can only run non-disruptive tests. These are typically ONLY disruptive tests. By de​ fault. 2 Health (aka On-going) tests run periodically. Deployed via selectors.Hardware Expansion and Replacement 407 1 Boot-up tests run when switch. click fab​ ric > fab​ ric poli​ cies > Mon​ i​ tor​ ing poli​ cies > de​ fault > di​ ag​ nos​ tics pol​ icy In the work pane se​ lect the fab​ ric el​ e​ ment that you would like to view the di​ ag​ nos​ tic mon​ i​ tor​ ing pol​ icy for. and they can be disruptive. Comes with default set of tests that can be modified and are deployed via selectors 3 On-Demand Tests are to be run on specific ports or cards for troubleshooting. there are no defaults. Viewing Diagnostic Monitoring Policies in the GUI . tests are log​ i​ cally grouped into col​ lec​ tions.

in the work pane se​ lect the Trou​ bleshoot​ ing tab to view GOLD di​ ag​ nos​ tic re​ sults for the su​ per​ vi​ sor. Create new on-demand diagnostic test As a part of op​ er​ a​ tional pro​ ce​ dures. mod​ ules.408 Hardware Expansion and Replacement Test re​ sults are view​ able by click​ ing on Fab​ ric > in​ ven​ tory > Pod-1 > Leaf-xx or Spine-xx > Chas​ sis > Su​ per​ vi​ sor mod​ ules > Slot-1 AND Fab​ ric > in​ ven​ tory > Pod-1 > Leaf-xx or Spine-xx > Chas​ sis > Line mod​ ules > Slot-1 Once there. it may be nec​ es​ sary to val​ i​ date a sys​ tem is healthy by run​ ning non-dis​ rup​ tive tests on the sys​ tem. Viewing GOLD Diagnostics Information in the GUI In a mod​ u​ lar chas​ sis-based sys​ tem such as the Cisco Nexus 9500 se​ ries switch. . sys​ tem con​ troller and the fab​ ric mod​ ules in the sys​ tem. di​ ag​ nos​ tic re​ sults are avail​ able for all the su​ per​ vi​ sor.

right click on the test or test set you would like to run on de​ mand. and click on Cre​ ate. the op​ er​ a​ tor se​ lects the op​ tions for a non-dis​ rup​ tive test to be done on leaf-2. Enter a name for the test. fab​ ric-port 1/97. se​ lect a test con​ fig (de​ fault is "no tests") and se​ lect the fab​ ric ports that the test would need to be run on. Creating a On-Demand Diag Policy in the GUI Once you click cre​ ate. nav​ i​ gate to Fab​ ric > Fab​ ric Poli​ cies > Trou​ bleshoot Poli​ cies > On de​ mand di​ ag​ nos​ tics Once there.Hardware Expansion and Replacement 409 In order to do this. the APIC GUI al​ lows cre​ ation of an On-de​ mand di​ ag​ nos​ tic test. To do this. en​ sure you check the box against "in​ clude dis​ rup​ tive tests" if this is the in​ tent. an op​ er​ a​ tor is cre​ at​ ing a fab​ ric-port on-de​ mand diag pol​ icy that al​ lows the abil​ ity to test the leaf up​ link ports going to the spine. In this case. In the ex​ am​ ple below. se​ lect the admin state. under the di​ ag​ nos​ tic tests. .

click "SUB​ MIT" to sub​ mit the pol​ icy.410 Hardware Expansion and Replacement Creating a Fabric Port On-Demand Diag Policy in the GUI After ver​ i​ fy​ ing the pa​ ra​ me​ ters above. Note that APIC dis​ plays a warn​ ing mes​ sage in cases where a non-dis​ rup​ tive tests are se​ lected to the ef​ fect of below. Once sub​ mit​ ted. . you can kick off the di​ ag​ nos​ tic test by click​ ing on the test it​ self and click​ ing sub​ mit.

the di​ ag​ nos​ tic re​ sults can be ob​ tained from the same lo​ ca​ tion as the lo​ ca​ tion for di​ ag​ nos​ tics that are run at bootup or are on​ go​ ing. Fab​ ric > In​ ven​ tory > Pod-1 > Leaf-2 > Line-mod​ ules > Slot-1 > Fab​ ric ports > 1/97 Viewing On-Demand Diag Policy Test Results in the GUI Note the on-de​ mand re​ sults in the right hand cor​ ner after the test has com​ pleted its run.Hardware Expansion and Replacement 411 GUI Warning for Executing a Disruptive Diag Policy Once you con​ firm the test run. .

.

413 Appendix .

.

The list below is a sub​ set of the full list of the avail​ able classes. and/or sta​ tis​ tics. The HTML body of the re​ sponse mes​ sage con​ tains a JSON or XML struc​ ture that con​ tains re​ quested data. health. All the phys​ ic ​al and log​ ic ​al com​ po​ nents that com​ prise the Ap​ pli​ ca​ tion Cen​ tric In​ fra​ struc​ ture fab​ ric are rep​ re​ sented in a hi​ er​ ar​ chi​ cal man​ age​ ment in​ for​ ma​ tion tree (MIT).1 or HTTPS POST. or DELETE mes​ sage to the APIC. You can use any pro​ gram​ ming lan​ guage to gen​ er​ ate the mes​ sages.Appendix 415 Classes The Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller (APIC) classes are cru​ cial from an op​ er​ at​ ional per​ spec​ tive to un​ der​ stand how sys​ tem events and faults re​ late to ob​ jects within the ob​ ject model. The APIC REST API is a pro​ gram​ matic in​ ter​ face to the APIC that uses a REST ar​ chi​ tec​ ture. or error in​ for​ ma​ tion. The API ac​ cepts and re​ turns HTTP or HTTPS mes​ sages that con​ tain JSON or XML doc​ um ​ents. GET. Each event and/or fault in the sys​ tem is a unique ob​ ject that can be ac​ cessed for con​ fig​ ur​ a​ tion. Each node in the tree rep​ re​ sents a man​ aged ob​ ject (MO) or group of ob​ jects that con​ tains its ad​ min​ is​ tra​ tive state and its op​ er​ at​ ional state. You can in​ voke an API func​ tion by send​ ing an HTTP/1. The HTML body of the POST mes​ sage con​ tains a JSON or XML data struc​ ture that de​ scribes an MO or an API method. To ac​ cess the com​ plete list of classes. point to the APIC and ref​ er​ ence the doc/html di​ rec​ tory at the end of the URL: https://apic_ip_address/doc/html/ . con​ fir​ ma​ tion of a re​ quested ac​ tion. fault. and the JSON or XML doc​ um ​ents that con​ tain the API meth​ ods or man​ aged ob​ ject (MO) de​ scrip​ tions. The fol​ low​ ing sec​ tion is a rep​ re​ sen​ ta​ tion of use​ ful classes for es​ tab​ lish​ ing a foun​ da​ tion for mon​ it​ or​ ing and man​ age​ ment of the fab​ ric.

2/api/node/class/topSystem. One in​ stance ob​ ject is cre​ ated for each fault con​ di​ tion of the par​ ent ob​ ject. Usage: The top​ Sys​ tem class can be used to de​ rive ob​ ject prop​ er​ ties in​ clud​ ing inb/oob man​ age​ ment de​ tails. node model num​ bers and de​ vice roles.416 Appendix Fabric Monitoring topSystem Name: top:Sys​ tem De​ scrip​ tion: Pro​ vides a list of all the de​ vices within the fab​ ric. ten​ ant or in​ di​ vid​ ual man​ aged ob​ jects within the APIC.16.2/api/node/class/fabricNode. . fabricNode REST :: https://172. cur​ rent time. Usage: The fault​ Inst class can be used to de​ rive all faults as​ so​ ci​ ated with the fab​ ric. leafs and spines.json faultInst Name: fault:Inst De​ scrip​ tion: Con​ tains de​ tailed in​ for​ ma​ tion of the fault. sys​ tem up​ time and cur​ rent state. A fault in​ stance ob​ ject is iden​ ti​ fied by a fault code. in​ clud​ ing con​ trollers.96.json fabricNode Name: fab​ ric:Node De​ scrip​ tion: Pro​ vides a list of all the nodes that are part of the fab​ ric.96. as​ signed node ids. This ob​ ject is at​ tached as a child of the ob​ ject on which the fault con​ di​ tion oc​ curred. leafs and spines. in​ clud​ ing con​ trollers. topSystem REST :: https://172.16. Usage: The fab​ ric​ N​ ode class can be used to de​ rive ob​ ject prop​ er​ ties in​ clud​ ing node se​ ri​ al num​ bers.

96.Appendix 417 faultInst REST :: https://172.16.json fvRsCEpToPathEp Name: De​ scrip​ tion: fv:RsCEp​ ToPathEp This is an in​ ter​ nal ob​ ject that pro​ vides a re​ la​ tion to a path end​ point.2/api/node/class/fvRsCEpToPathEp.2/api/node/class/fabricHealthTotal. The fab​ ricHealth​ To​ tal class can be used to de​ rive the over​ all sys​ tem fabricHealthTotal REST :: https://172. fvRsCEpToPathEp REST :: https://172.96.2/api/node/class/fvCEp. ap​ pli​ ca​ tion pro​ file and end point group.16. fvCEp REST :: https://172.json fvCEp Name: De​ scrip​ tion: fv:CEp A client end​ point at​ tach​ ing to the net​ work.96.16.2/api/node/class/faultInst.16.json fabricHealthTotal Name: De​ scrip​ tion: Usage: health. fab​ ric:Health​ To​ tal The fab​ ric total health score in​ stance.json eqptFabP . Usage: The fvCEp class can be used to de​ rive a list of end points at​ tached to the fab​ ric and the as​ so​ ci​ ated ip/mac ad​ dress and en​ cap​ su​ la​ tion for each ob​ ject.96. Usage: The fvRsCEp​ ToPathEp class can be used to de​ rive path fab​ ric de​ tails such as the node and port as well as the ten​ ant de​ tails such as the ten​ ant name.

16. eqptFabP REST :: https://172. eqptLeafP REST :: https://172.96. eqptCh REST :: https://172.96.16.2/api/node/class/eqptCh. the fab​ ric fac​ ing ex​ ter​ nal IO port. se​ ri​ al num​ ber and model num​ ber.json eqptCh Name: De​ scrip​ tion: eqpt:ChA The hard​ ware chas​ sis con​ tainer. Usage: The eqpt​ FabP class can be used to de​ rive a list of fab​ ric port and the as​ so​ ci​ ated de​ tails such as the line card and chas​ sis place​ ment. the non-fab​ ric fac​ ing ex​ ter​ nal leaf IO port. Usage: The eqpt​ FabP class can be used to de​ rive a list of non-fab​ ric port and the as​ so​ ci​ ated de​ tails such as the line card and chas​ sis place​ ment.418 Appendix eqptFabP Name: De​ scrip​ tion: eqpt:FabP Fab​ ric port.96. Usage: The eqptCh class can be used to de​ rive a chas​ sis list and the as​ so​ ci​ ated de​ tails such as the op​ er​ at​ ional state.2/api/node/class/eqptFabP.2/api/node/class/eqptLeafP.json eqptLC Name: eqpt:LCA .16.json eqptLeafP Name: De​ scrip​ tion: eqpt:LeafP Fab​ ric port.

eqptFt REST :: https://172. model. se​ ri​ al num​ ber.16. . eqptPsu REST :: https://172.2/api/node/class/eqptFt.json eqptPsu Name: De​ scrip​ tion: eqpt:Psu The power sup​ ply unit. The eqptLC class can be used to de​ rive a list of line cards de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the re​ dun​ dancy state. Usage: The eqptFt class can be used to de​ rive a list of fan trays and the as​ so​ ci​ ated de​ tails such as the op​ er​ a​ tional sta​ tus. con​ tain​ ing IO ports.16.json eqptSupC Name: De​ scrip​ tion: eqpt:SupC The su​ per​ vi​ sor card. Usage: The eqptFt class can be used to de​ rive a list of power sup​ plies within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the model num​ ber. op​ er​ at​ ional sta​ tus. se​ r​ ial num​ ber and hard​ ware ver​ sion.2/api/node/class/eqptPsu.Appendix 419 De​ scrip​ tion: Usage: The line card (IO card). which con​ tains the CPU run​ ning con​ trol plane. and the volt​ age source. model num​ ber. eqptLC REST :: https://172.96.16.json eqptFt Name: De​ scrip​ tion: eqpt:Ft The in​ ven​ to​ ried fan tray. se​ ri​ al num​ bers and the num​ ber of ports.2/api/node/class/eqptLC.96.96.

op​ er​ at​ ional sta​ tus and re​ dun​ dancy state. Usage: The db​ gAc​ Trail class can be used to de​ rive a list of the atomic coun​ ters de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts.16.16.16.96. eqptSupC REST :: https://172.96. se​ ri​ al num​ ber.2/api/node/class/ethpmPhysIf. op​ er​ at​ ional sta​ tus.json dbgAcTrail Name: De​ scrip​ tion: dbg:Ac​ Trail The atomic counter trail. du​ plex. and usage state. ethpmPhysIf REST :: https://172.2/api/node/class/dbgAcTrail.420 Appendix Usage: The eqptFt class can be used to de​ rive a list of su​ per​ vi​ sor cards de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the model num​ ber.json ethpmPhysIf Name: De​ scrip​ tion: ethpm:PhysIf The phys​ ic ​al in​ ter​ face in​ for​ ma​ tion holder.json dbgEpgToEpgRslt Name: De​ scrip​ tion: entry. dbgAcTrail REST :: https://172. dbg:Epg​ ToEp​ gRslt The end​ point group to end​ point group atomic counter. . Usage: The eth​ pm​ PhysIf class can be used to de​ rive a list of phys​ ic ​al in​ ter​ faces in the fab​ ric and the as​ so​ ci​ ated de​ tails such as a the speed. on-de​ mand.96.2/api/node/class/eqptSupC.

. On-de​ mand.16. Usage: The com​ pVm class can be used to de​ rive a list of vir​ tual ma​ chines de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the name and state.2/api/node/class/compVm. and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts.96.json VMM Monitoring compVm Name: De​ scrip​ tion: comp:Vm The Vir​ tual ma​ chine ob​ ject. compVm REST :: https://172.16.json compHv Name: De​ scrip​ tion: comp:Hv An ob​ ject rep​ re​ sent​ ing the com​ pute hy​ per​ vi​ sor.Appendix 421 Usage: The dbgEpg​ ToEp​ gR​ sIt class can be used to de​ rive a list of the EPG to EPG atomic coun​ ters de​ ployed within the fab​ ric. Entry.2/api/node/class/dbgEpgToEpgRslt. dbgEpToEpTsIt REST :: https://172.96.96.json dbgEpToEpRslt Name: De​ scrip​ tion: Usage: dbg:Ep​ ToEpRslt The end​ point to end​ point atomic counter. dbgEpgToEpgRsIt REST :: https://172.16. The dbgEp​ ToEpT​ sIt class can be used to de​ rive a list of the end​ point to end​ point atomic coun​ ters de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as dropped packet sta​ tis​ tics and packet counts.2/api/node/class/dbgEpToEpRslt.

the VMM con​ troller pro​ file could be a pol​ icy to con​ nect a VMware vCen​ ter that is part a VMM do​ main. compHv REST :: https://172.96.16.json fvRsHyper Name: De​ scrip​ tion: fv:RsHy​ per A re​ la​ tion to the hy​ per​ vi​ sor that con​ trols and mon​ it​ ors the APIC VMs.2/api/node/class/fvRsVm. which spec​ if​ ies how to con​ nect to a sin​ gle VM man​ age​ ment con​ troller that is part of con​ tain​ ing pol​ icy en​ force​ ment do​ main.json vmmCtrlrP Name: vmm:CtrlrP De​ scrip​ tion: The VMM con​ troller pro​ file.422 Appendix Usage: The com​ pVm class can be used to de​ rive a list of com​ pute hy​ per​ vi​ sor de​ ployed within the fab​ ric and the as​ so​ ci​ ated de​ tails such as the name and sta​ tus. This is an in​ ter​ nal ob​ ject. fv:RsVm A re​ la​ tion to a vir​ tual ma​ chine con​ nected to a hy​ per​ vi​ sor. fvRsHyper REST :: https://172. Usage: The fvR​ sHy​ per class can be used to de​ rive the re​ la​ tion​ ship of the hy​ per​ vi​ sor that con​ trols and mon​ it​ ors the APIC VMs. vRsVm REST :: https://172.2/api/node/class/fvRsHyper.16. .json fvRsVm Name: De​ scrip​ tion: ter​ nal ob​ ject. This is an in​ - Usage: The fvRsVm class can be used to de​ rive the re​ la​ tion​ ship of the vir​ tual ma​ chines con​ nected to the hy​ per​ vi​ sor.16. For ex​ am​ ple.2/api/node/class/compHv.96.96.

2/api/node/class/vnsAbsGraph. The con​ fig​ ur​ a​ tion is pushed down to the VIP ad​ dress. Usage: The class vns​ Ab​ s​ Graph can be used to de​ rive a list of ser​ vice graph tem​ - plates con​ fig​ ured on the APIC. vmmCtrlrP REST :: https://172. which is rep​ re​ sented by a sin​ gle vir​ tual IP (VIP).json vnsLDevVip Name: vnsLDevVip De​ scrip​ tion: An L4-L7 de​ vice clus​ ter.16.96.96.2/api/node/class/vnsLDevVip.96.16. vnsAbsGraph REST :: https://172. Usage: The class vnsLDevVip can be used to de​ rive all the VIPs con​ fig​ ured for the log​ ic ​al de​ vice clus​ ters in the fab​ ric vnsLDevVip REST :: https://172.json .json Layer 4 to Layer 7 Monitoring vnsAbsGraph Name: vns​ Ab​ s​ Graph De​ scrip​ tion: The ab​ stract graph is made up of ab​ stract nodes and used to de​ fine the traf​ fic flow through a ser​ vice func​ tion such as load bal​ anc​ ing. or fire​ wall. and con​ nec​ tions.Appendix 423 Usage: The vmm​ C​ trlrP class can be used to de​ rive the ip ad​ dress and the dat​ a-​ cen​ ter name of the con​ nected VM do​ main.16. ab​ stract term nodes (the nodes that are con​ nected to end​ point groups).2/api/node/class/vmmCtrlrP. SSL of​ fload. along with their prop​ er​ ties. Ab​ stract nodes are com​ prised of ser​ vice nodes such as a ser​ vice node bal​ ancer (SLB) or fire​ wall (FW).

Usage: The class vnsLif can be used to de​ rive the con​ nec​ tion be​ tween a ser​ vice graph and de​ vice in​ ter​ faces.2/api/node/class/vnsLIf.424 Appendix vnsCDev Name: vn​ sCDev De​ scrip​ tion: The in​ di​ vid​ ual ser​ vice de​ vice.2/api/node/class/vnsLDevCtx. To spec​ ify a wild card.json vnsRsLDevCtxToLDev . which is used to de​ fine a con​ crete l4-l7 ser​ vice de​ vice. and func​ tion label or names.json vnsLif Name: vnsLif De​ scrip​ tion: The log​ ic ​al in​ ter​ face.2/api/node/class/vnsCDev. which is as​ so​ ci​ ated with a set of con​ crete in​ ter​ - faces from the L4-L7 de​ vice clus​ ter. set the name to Any. sub​ ject.16.json vnsLDevCtx Name: vnsLDe​ vCtx De​ scrip​ tion: A de​ vice clus​ ter con​ text.96.16. which points to the de​ vice clus​ ter used to pick a spe​ cific de​ vice based on con​ tract. Usage: The class vn​ sCDev can be used to de​ rive a list of con​ crete de​ vices con​ fig​ ured as part of the L4-7 ser​ vice in​ te​ gra​ tion vnsCDev REST :: https://172. Usage: The class vnsLDe​ vCtx can be used to de​ rive the node and con​ tract name.96. nsLDevCtx REST :: https://172.96. vnsLif REST :: https://172.16.

compHostStats1h REST :: https://172.96.16. Usage: The com​ pRcvdEr​ rP​ kt​ s​ 1h class can be used to de​ rive the most cur​ rent sta​ tis​ tics for re​ ceived error pack​ ets.2/api/node/class/compHostStats1h.2/api/node/class/vnsRsLDevCtxToLDev.16.json .16.Appendix 425 vnsRsLDevCtxToLDev Name: De​ scrip​ tion: vn​ sRsLDe​ vC​ tx​ ToLDev A source re​ la​ tion to the ab​ strac​ tion of a ser​ vice de​ vice clus​ ter or of a proxy ob​ ject for a log​ ic ​al de​ vice clus​ ter in the ten​ ant. Usage: The class vn​ sRsLDe​ vC​ tx​ ToLDev can be used to de​ rive the re​ la​ tion​ ship be​ tween vnsLDe​ vCtx and vnsLDev. Usage: The com​ pHost​ Stat​ s1h class can be used to de​ rive the sta​ tis​ tics as​ so​ ci​ ated with the com​ pute hy​ per​ vi​ sor.json compRcvdErrPkts1h Name: comp:RcvdEr​ rP​ kt​ s​ 1h De​ scrip​ tion: A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for re​ ceived error pack​ ets in a 1 hour sam​ pling in​ ter​ val.json Statistics compHostStats1h Name: comp:Host​ Stat​ s1h De​ scrip​ tion: A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for host in a 1 hour sam​ pling in​ ter​ val. This class up​ dates every 15 min​ utes.96. vnsRsLDevCtxToLDev REST :: https://172. compRcvdErrPkts1h REST :: https://172. This class up​ dates every 15 min​ utes.96.2/api/node/class/compRcvdErrPkts1h.

16.2/api/node/class/aaaUser. Authorization. Usage: The aaaUser class can be used to de​ rive a list of user ac​ counts de​ ployed within the fab​ ric. A log record is au​ to​ mat​ ic ​ally gen​ er​ ated when​ ever a user mod​ if​ ies an ob​ ject.96.96. compTrnsmtdErrPkts1h REST :: https://172. aaaUser REST :: https://172. and Accounting aaaModLR Name: aaa:ModLR De​ scrip​ tion: The AAA audit log record.2/api/node/class/aaaModLR.json Authentication.16. aaaModLR REST :: https://172.16.96.2/api/node/class/compTrnsmtdErrPkts1h. Usage: The aaaModLR class can be used to de​ rive a fab​ ric based audit log for all changes and events.json . Usage: The compTrnsmt​ dEr​ rP​ kt​ s​ 1h class can be used to de​ rive the most cur​ rent sta​ tis​ tics for trans​ mit​ ted error pack​ ets. This class up​ dates every 15 min​ utes.426 Appendix compTrnsmtdErrPkts1h Name: De​ scrip​ tion: comp:Trnsmt​ dEr​ rP​ kt​ s​ 1h A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for trans​ mit​ ted error pack​ ets in a 1 hour sam​ pling in​ ter​ val.json aaaUser Name: De​ scrip​ tion: aaa:User A lo​ cally-au​ then​ ti​ cated user ac​ count.

16.2/api/class/eqptcapacityPolEntry5min. A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for pol​ icy entry in a 5 minute sam​ pling in​ ter​ val. aaaRemoteUser REST :: https://172.2/api/node/class/aaaRemoteUser.96. Usage: The eqpt​ ca​ pac​ it​ y​ Po​ lEn​ try5min class can be used to de​ rive the cur​ rent value as​ so​ ci​ ated with the Pol​ icy TCAM usage.16. This class up​ dates every 10 sec​ onds. Usage: The aaaUser class can be used to de​ rive a list of re​ mote user ac​ counts de​ ployed within the fab​ ric. Usage: The eqpt​ ca​ pac​ ityL3En​ try5min class can be used to de​ rive the cur​ rent value as​ so​ ci​ ated with the Pre​ fix TCAM usage. . A class that rep​ re​ sents the most cur​ rent sta​ tis​ tics for lay​ er3 entry in a 5 minute sam​ pling in​ ter​ val.96.json Fabric Capacity Policy TCAM Name: eqpt​ ca​ pac​ it​ y​ Po​ lEn​ try5min De​ scrip​ tion: Pol​ icy CAM entry sta​ tis​ tics. eqptcapacityPolEntry5min REST :: http://172.Appendix 427 aaaRemoteUser Name: aaa:Re​ mo​ teUser De​ scrip​ tion: A re​ mote user login ac​ count.json Prefix TCAM Name: eqpt​ ca​ pac​ ityL3En​ try5min De​ scrip​ tion: Lay​ er3 entry sta​ tis​ tics. This class up​ dates every 10 sec​ onds.

2/api/class/eqptcapacityL3Entry5min.16. syslogRemoteDest REST :: https://172.json SNMP/SYSLOG SNMP Trap Destination Name: sn​ mpTrapDest A des​ ti​ na​ tion to which traps and in​ forms are sent.​ html .16.16.json Prefix TCAM Name: sys​ lo​ gRe​ mot​ eDest De​ scrip​ tion: The sys​ log re​ mote des​ ti​ na​ tion host en​ ables you to spec​ ify sys​ log servers to which mes​ sages from the APIC and fab​ ric nodes should be for​ warded. De​ scrip​ tion: Usage: The sn​ mpTrapDest class can be used to de​ rive the cur​ rent list of snmp trap des​ ti​ na​ tions im​ ple​ mented within the fab​ ric.96. The Cisco APIC Com​ mand-Line In​ ter​ face User Guide may also be help​ ful for un​ der​ stand​ ing the fol​ low​ ing sec​ tions .96.2/api/node/class/syslogRemoteDest.96. snmpTrapDest REST :: https://172.​ cisco.json Use Cases The class fault​ Inst used in Use Case #1 and Use Case #2 below can be re​ placed with any of the man​ aged ob​ ject classses dis​ cussed above or spec​ if​ ied within the APIC doc​ u-​ men​ ta​ tion.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ cli/​ b_​ APIC_​ CLI_​ User_​ Guide.428 Appendix eqptcapacityL3Entry5min REST :: https://172.2/api/node/class/snmpTrapDest.see: http://​ www. Usage: The sys​ lo​ gRe​ mot​ eDest class can be used to de​ rive the cur​ rent list of sys​ log re​ mote des​ ti​ na​ tions im​ ple​ mented within the fab​ ric.

The user has the op​ tion of col​ lect​ ing the re​ sults via CLI. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions.: Class or DN :: faultInst Property :: n/a Op :: n/a Value :: n/a From a POST​ MAN per​ spec​ tive.mit. Vi​ sore.query(classQuery): print fault.name Sam​ ple Cobra script to cap​ ture faults within the fab​ ric: #!/usr/bin/env python import cobra.xml From a Cobra per​ spec​ tive.access import cobra.mit.session import LoginSession .session from cobra.Appendix 429 Case 1: Creating an application script to retrieve the current list of faults in the fabric. POST​ MAN and/or Cobra. user the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/class/faultInst. use the fol​ low​ ing class query: # Class Query classQuery= ClassQuery('faultInst') for fault in md.mit. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -c faultInst From a Vi​ sore per​ spec​ tive. This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to ob​ tain the list of cur​ rent faults in the fab​ ric. From a CLI per​ spec​ tive. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query.

access. <password>.faultInst. secure=False) md = cobra.cause.MoDirectory(ls) md. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -c faultInst -f 'fv.session."config-failure")) .mit.LoginSession('https://'<your apic ip address>.cause=="config-failure"' From a Vi​ sore per​ spec​ tive. <username>.430 Appendix from cobra. From a CLI per​ spec​ tive. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions.mit. This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cob​ tain the list of cur​ rent faults in the fab​ ric.mit. use the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/class/faultInst.login() # Class Query classQuery= ClassQuery('faultInst') for fault in md.xml?query-targetfilter=and(eq(faultInst. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query: Class or DN :: faultInst Property :: cause Op :: == Value :: config-failure From a POST​ MAN per​ spec​ tive.name Case 2: Creating an application script to retrieve the current list of faults in the fabric that have been caused by a failed configuration. The user has an op​ tion of col​ lect​ ing the re​ sults via CLI.request import ClassQuery ls = cobra. POST​ MAN and/or Cobra. Vi​ sore.query(classQuery): print fault.

<username>.mit.access.propFilter = 'wcard(faultInst.mit.session import LoginSession from cobra.mit. use the fol​ low​ ing com​ mand to per​ form the query: admin@apic1:~> moquery -d uni/tn-common .mit.LoginSession('https://'<your apic ip address>. <password>.mit.name Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures. use the fol​ low​ ing class query: # Class Query classQuery= ClassQuery('faultInst') classQuery. Vi​ sore.session from cobra. "{0}")'. POST​ MAN and/or Cobra.login() # Class Query classQuery= ClassQuery('faultInst') for fault in md. Please refer to the sec​ tion above for ap​ pli​ ca​ tion spe​ cific ac​ cess and ex​ plan​ tions.mit. DN This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to ob​ tain the prop​ er​ ties of the ten​ ant name Com​ mon.request import ClassQuery ls = cobra. secure=False) md = cobra.query(classQuery): print fault. The user has an op​ tion of col​ lect​ ing the re​ sults via CLI.MoDirectory(ls) md.name Case 3: Creating an application script to retrieve the properties for a specific managed object.query(classQuery): print fault.format('config-failure') for fault in md. #!/usr/bin/env python import cobra.Appendix 431 From a Cobra per​ spec​ tive.access import cobra. From a CLI per​ spec​ tive.cause.session.

query(dnQuery): print results. secure=False) md = cobra.session.login() # DN Query dnQuery= DnQuery('uni/tn-common') for results in md.access.dn Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures.mit.xml?query-target=self From a Cobra per​ spec​ tive.dn . <username>.mit.request import DnQuery ls = cobra. use the fol​ low​ ing REST GET to per​ form the query: GET http://<your apic ip address>/api/node/mo/uni/tn-common.mit.session import LoginSession from cobra. #!/usr/bin/env python import cobra.session from cobra. use the fol​ low​ ing class query: # DN Query dnQuery= DnQuery('uni/tn-common') for results in md.mit. use the fol​ low​ ing pa​ ra​ me​ ters to per​ form the query: Class or DN :: uni/tn-common Property :: n/a Op :: n/a Value :: n/a From a POST​ MAN per​ spec​ tive.mit.mit.432 Appendix From a Vi​ sore per​ spec​ tive.MoDirectory(ls) md.access import cobra. <password>.query(dnQuery): print results.LoginSession('https://'<your apic ip address>.

subtree = 'children' q.mit.mit. #!/usr/bin/env python from cobra.session import LoginSession from cobra.mit.session. <password>.request import ClassQuery lls = cobra.LoginSession('https://'<your apic ip address>.mit.subtreeClassFilter = 'fvRsCEpToPathEp' mos = md.login() q = ClassQuery('fvCEp') q. secure=False) md = MoDirectory(ls) md.Appendix 433 Case 4: Creating an application script to retrieve the current list of endpoints (mac-addresses) attached to the fabric This use case may be typ​ ic ​al for en​ vi​ ron​ ments where an ACI ad​ min​ is​ tra​ tor wishes to cre​ ate an ap​ pli​ ca​ tion script to cap​ ture the list of cur​ rent end​ points at​ tached to the fab​ ric along with the node de​ tails per​ tain​ ing to each end​ point. Cobra Script to cap​ ture faults ca​ sued by con​ fig​ ur​ a​ tion fail​ ures.access import MoDirectory from cobra.dn . <username>.query(q) for mo in mos: for child in mo.rscEpToPathEp: print child.

.

Appendix 435 Package Decoder There are sev​ eral ab​ bre​ vi​ at​ ions used in the names of classes in the ACI ob​ ject model. which may help when de​ ci​ pher​ ing what class ob​ jects are when using them with REST calls. Package Decoder Aaa: au​ then​ ti​ ca​ tion. ac​ count​ ing ac: atomic coun​ ters actrl: ac​ cess con​ trol ac​ trl​ cap: ac​ cess con​ trol ca​ pa​ bil​ ity adcom: ap​ pli​ ance di​ rec​ tor com​ mu​ ni​ ca​ tion aib: ad​ ja​ cency in​ for​ ma​ tion base arp: ad​ dress res​ o​ lu​ tion pro​ to​ col bgp: bor​ der gate​ way pro​ to​ col call​ home: Cisco smart call home ser​ vices cap: ca​ pa​ bil​ ity cdp: Cisco dis​ cov​ ery pro​ to​ col cnw: node clus​ ter comm: com​ mu​ ni​ ca​ tion pol​ icy comp: com​ pute . au​ tho​ riza​ tion. Here are some de​ scrip​ tions of com​ monly used ab​ bre​ vi​ at​ ions.

436 Appendix com​ pat: com​ pat​ ib ​il​ ity con​ di​ tion: health pol​ icy con​ fig: con​ fig​ ur​ a​ tion pol​ icy coop: Coun​ cil of Or​ ac ​les pro​ to​ col copp: con​ trol plane polic​ ing pol​ icy: con​ tains set of rules de​ scrib​ ing po​ licer rates ctrlr: con​ troller ctx: con​ text date​ time: date/time pol​ icy dbg: debug dbgac: debug atomic coun​ ters dbg​ exp: debug ex​ port pol​ icy dhcp: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col dhcptlv: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col type length value dhcptlvpol: dy​ namic host con​ fig​ ur​ a​ tion pro​ to​ col type length value pol​ icy dns: do​ main name ser​ vice draw: graph vi​ su​ al​ iza​ tion for GUI epm: end​ point man​ ager eqpt: equip​ ment eqpt​ cap: equip​ ment ca​ pa​ bil​ ity eqpt​ ca​ pac​ ity: equip​ ment ca​ pac​ ity .

Appendix 437 eqpt​ diag: equip​ ment di​ ag​ nos​ tics eqpt​ di​ agp: equip​ ment di​ ag​ nos​ tics pol​ icy ethpm: eth​ er​ net pol​ icy man​ ager event: event pol​ icy extnw: ex​ ter​ nal net​ work fab​ ric: fab​ ric fault: fault pol​ icy. coun​ ters file: file path. con​ fig im​ port/ex​ port pol​ icy firmware: firmware fm​ cast: fab​ ric mul​ ti​ cast fsm: fi​ nite state ma​ chine fv: fab​ ric vir​ tu​ al​ iza​ tion fvns: fab​ ric vir​ tu​ al​ iza​ tion name​ space fv​ topo: fab​ ric vir​ tu​ al​ iza​ tion topol​ ogy geo: ge​ olo​ ca​ tion glean: glean ad​ ja​ cency ha: high avail​ abil​ ity health: health score hvs: hy​ per​ vi​ sors vir​ tual switch icmp: in​ ter​ net con​ trol pro​ to​ col .

438 Appendix icm​ pv4: in​ ter​ net con​ trol pro​ to​ col ver​ sion 4 icm​ pv6: in​ ter​ net con​ trol pro​ to​ col ver​ sion 6 ident: iden​ tity igmp: in​ ter​ net group man​ age​ ment pro​ to​ col igmp​ snoop: in​ ter​ net group man​ age​ ment pro​ to​ col snoop​ ing im: in​ ter​ face man​ ager mod​ ule im​ gin​ stall: image in​ stall infra: in​ fra​ struc​ ture ip: in​ ter​ net pro​ to​ col ipv4: in​ ter​ net pro​ to​ col ver​ sion 4 ipv6: in​ ter​ net pro​ to​ col ver​ sion 6 isis: in​ ter​ me​ di​ ate sys​ tem to in​ ter​ me​ di​ ate sys​ tem isistlv: in​ ter​ me​ di​ ate sys​ tem to in​ ter​ me​ di​ ate sys​ tem type length value l1: layer 1 l1cap: layer 1 ca​ pa​ bil​ ity l2: layer 2 l2cap: layer 2 ca​ pa​ bil​ ity l2ext: layer 2 ex​ ter​ nal l3: layer 3 l3cap: layer 3 ca​ pa​ bil​ ity .

Appendix 439 l3ext: layer 3 ex​ ter​ nal l3vm: Layer 3 Vir​ tual Ma​ chine lacp: link ag​ gre​ ga​ tion pro​ to​ col lbp: load bal​ anc​ ing pol​ icy leqpt: loose equip​ ment (un​ man​ aged nodes. not in the fab​ ric) lldp: link layer dis​ cov​ ery pro​ to​ col lldptlv: link layer dis​ cov​ ery pro​ to​ col type length value lldptlvpol: link layer dis​ cov​ ery pro​ to​ col type length value pol​ icy maint: main​ te​ nance mcast: mul​ ti​ cast mcp: mas​ ter con​ trol proces​ sor mem​ ory: mem​ ory sta​ tis​ tics mgmt: man​ age​ ment mo: man​ aged ob​ ject mock: mock (ob​ jects used on the sim​ ul​ a​ tor mostly for show​ ing stats/faults/etc) mon: mon​ it​ or​ ing mon​ i​ tor: mon​ it​ or (SPAN) nam​ ing: ab​ stract for ob​ jects with names nd: neigh​ bor dis​ cov​ ery nw: net​ work .

and mem​ ory uti​ liza​ tion sta​ tis​ tics psu: power sup​ ply unit pol​ icy qos: qual​ ity of ser​ vice pol​ icy qosm: qos sta​ tis​ tics qosp: qos/ 802.1p rbqm: de​ bug​ ging . logs/his​ tory opflex: OpFlex os: op​ er​ at​ ing sys​ tem ospf: open short​ est path first pc: port chan​ nel pcons: **gen​ er​ ated and used by in​ ter​ nal processes** phys: phys​ ic ​al do​ main pro​ file ping: ping ex​ e​ cu​ tion and re​ sults pki: pub​ lic key in​ fra​ struc​ ture pol: pol​ icy de​ f​ in ​i​ t​ ion po​ licer: traf​ fic polic​ ing (rate lim​ it​ ing) pool: ob​ ject pool pres: **gen​ er​ ated and used by in​ ter​ nal processes** proc: sys​ tem load. state. health. ad​ min​ is​ tra​ tions and man​ age​ ment ob​ server: ob​ server for sta​ tis​ tics.440 Appendix oam: eth​ er​ net op​ er​ at​ ions. fault. cpu.

Appendix 441 regress: re​ gres​ sion reln: **gen​ er​ ated and used by in​ ter​ nal processes** repl: **gen​ er​ ated and used by in​ ter​ nal processes** res: **gen​ er​ ated and used by in​ ter​ nal processes** rib: rout​ ing in​ for​ ma​ tion base rmon: re​ mote net​ work mon​ it​ or​ ing/ in​ ter​ face stats/coun​ ters rpm: route pol​ icy map rtcom: route con​ trol com​ mu​ nity list rtc​ trl: route con​ trol rtextcom: router ex​ tended com​ mu​ nity rtflt: route fil​ ter rtleak: route leak rtmap: RPM route map rtpfx: route pre​ fix list rtreg​ com: route reg​ ul​ ar com​ mu​ nity list rtsum: route sum​ ma​ riza​ tion ad​ dress/pol​ icy satm: satel​ lite man​ ager snmp: sim​ ple net​ work man​ age​ ment pro​ to​ col span: switched port an​ al​ yzer stats: sta​ tis​ tics col​ lec​ tion poli​ cies .

length. sub​ ject. etc) sys​ m​ grp: con​ tainer for cores pol​ icy & ab​ stract class for all qos pol​ icy de​ f​ in ​i​ t​ ions tag: alias (use de​ scrip​ tive name for dn). and re​ sult testin​ fralab: test in​ fra​ struc​ ture tlv: type. tags (group mul​ ti​ ple ob​ jects by a de​ scrip​ tive name) task: task ex​ e​ cu​ tion. sys​ tem states. value sys​ tem struc​ tures top: sys​ tem task man​ ager for proces​ sor ac​ tiv​ ity . su​ per​ vi​ sor. and re​ sult test: ab​ stract class for test rule. in​ stance.442 Appendix stat​ store: sta​ tis​ tics data hold​ ers storm​ c​ trl: storm con​ trol (traf​ fic sup​ pres​ sion) pol​ icy stp: span​ ning tree pro​ to​ col de​ f​ in ​i​ t​ ions and pol​ icy sts: Ser​ vice Tag Switch​ ing (used for ser​ vices in​ ser​ tion) svc​ core: core pol​ icy svi: switched vir​ tual in​ ter​ face/ routed VLAN in​ ter​ face syn​ thetic: syn​ thetic ob​ jects (for test​ ing) sys​ de​ bug: sys​ tem debug sys​ file: sys​ tem files syshist: sys​ tem cards reset records/his​ tory sys​ log: sys​ log pol​ icy sys​ mgr: sys​ tem man​ ager (firmware.

Appendix 443 topoc​ trl: topol​ ogy con​ trol pol​ icy (shard​ ing. fab​ ric VxLan. fab​ ric LB. Con​ tracts Model Naming schemes Rs: Re​ la​ tion​ ship source Rt: Re​ la​ tion​ ship tar​ get Ag: Ag​ gre​ gated stats BrCP: Bi​ nary Con​ tract Pro​ file .e. etc) tracer​ oute: tracer​ oute ex​ e​ cu​ tion and re​ sults tracer​ outep: tracer​ oute end points trig: trig​ ger​ ing pol​ icy tun​ nel: tun​ nel​ ing uribv4: ipv4 uni​ cast rout​ ing in​ for​ ma​ tion base en​ tity vlan: vlan in​ stances vlan​ mgr: vlan man​ ager con​ trol plane vmm: vir​ tual macine man​ ager (con​ troller. vmm pol​ icy and de​ f​ in ​i​ t​ ions) vns: vir​ tual net​ work ser​ vice (L4-L7 pol​ icy and de​ f​ in ​i​ t​ ions) vpc: vir​ tual port chan​ nel (vpc pol​ icy and de​ f​ in ​i​ t​ ions) vsvc: ser​ vice la​ bels (provider/con​ sumer) vtap: trans​ lated ad​ dress of ex​ ter​ nal node (NATed IP of ser​ vice node) vxlan: Vir​ tu​ ally ex​ ten​ si​ ble LAN de​ f​ in ​i​ t​ ions vz: vir​ tual zones (for​ mer name of the pol​ icy con​ trols) i.

.

an ASIC on a leaf switch. . AEPs are also the mech​ an ​ism that ties the phys​ ic ​al port to the do​ main (phys​ ic ​al or vir​ tual) to a switch pol​ icy.Appendix 445 Acronyms and Definitions Overview This sec​ tion is de​ signed to pro​ vide a high level de​ scrip​ tion of terms and con​ cepts that get brought up in this book. ACI Ex​ ter​ nal Con​ nec​ tiv​ ity: Any con​ nec​ tiv​ ity to and from the fab​ ric that uses an ex​ ter​ nal routed or switched in​ ter​ me​ di​ ary sys​ tem. A AAA: acronym for Au​ then​ ti​ ca​ tion. ACID trans​ ac​ tions: ACID is an acronym for Atom​ ic​ ity. This means that if one part of a trans​ ac​ tion fails the en​ tire trans​ ac​ tion fails. AEP: At​ tach​ able En​ tity Pro​ file – this is a con​ fig​ ur​ a​ tion pro​ file of the in​ ter​ face that gets ap​ plied when an en​ tity at​ taches to the fab​ ric. ALE: Ap​ pli​ ca​ tion Leaf En​ gine. there are some new terms and con​ cepts em​ ployed. As​ so​ ci​ ated new acronyms are also pro​ vided. and Ac​ count​ ing. This is not meant to be an ex​ haus​ tive list nor a com​ pletely de​ tailed dic​ tio​ nary of all of the terms and con​ cepts. only the key ones that may not be a part of the com​ mon ver​ nac​ ul​ ar. to en​ sure that data​ base con​ sis​ tency is main​ tained. Dura​ bil​ ity – prop​ er​ ties of trans​ ac​ tions that en​ sure con​ sis​ tency in data​ base trans​ ac​ tions. Con​ sis​ tency. and un​ der​ stand​ ing those new terms and con​ cepts will help those work​ ing on ACI com​ mu​ ni​ cate with one an​ other about the con​ structs used in ACI to trans​ mit those bits. Trans​ ac​ tions to APIC de​ vices in an ACI clus​ ter are con​ sid​ ered ACID. Iso​ la​ tion. While ACI does not change how pack​ ets are trans​ mit​ ted on a wire. where end​ points fall out​ side of the man​ aged scope of the fab​ ric. An AEP rep​ re​ sents a group of ex​ ter​ nal en​ ti​ ties with sim​ il​ ar in​ fra​ struc​ ture pol​ icy re​ quire​ ments. or which would be rel​ e​ vant to the trou​ bleshoot​ ing ex​ er​ cises that were cov​ ered in the trou​ bleshoot​ ing sce​ nar​ ios dis​ cussed. Au​ tho​ riza​ tion.

BDs can em​ u​ late the be​ hav​ ior of a tra​ di​ tional VLAN but are not con​ strained by for​ ward​ ing scale lim​ i​ ta​ tions. Bridge Do​ mains are also a con​ tainer for IP sub​ nets and are where fab​ ric Layer 3 gate​ way func​ tion​ al​ ity is con​ fig​ ured. an ASIC on a Spine switch. C CLOS fab​ ric: A multi-tier non​ block​ ing leaf-spine ar​ chi​ tec​ ture net​ work. a BD is a child of a Pri​ vate Layer 3 or con​ text. ACI works on a white list pol​ icy model. the de​ fault for​ ward​ ing pol​ icy is to not allow any com​ mu​ ni​ ca​ tion be​ tween EPGs. With​ out a con​ tract.) for each unique Layer 2 for​ ward​ ing do​ main. on the ACI fab​ ric Multi-Pro​ to​ col BGP is used to dis​ trib​ ute reach​ ab ​il​ ity in​ for​ ma​ tion within the fab​ ric. Ap​ pli​ ca​ tion Pro​ file: Term used to ref​ er​ ence an ap​ pli​ ca​ tion pro​ file-man​ aged ob​ ject ref​ er​ ence that mod​ els the log​ ic ​al com​ po​ nents of an ap​ pli​ ca​ tion and how those com​ po​ nents com​ mu​ ni​ cate.446 Appendix APIC: Ap​ pli​ ca​ tion Pol​ icy In​ fra​ struc​ ture Con​ troller is a cen​ tral​ ized pol​ icy man​ age​ ment con​ troller clus​ ter. ASE: Ap​ pli​ ca​ tion Spine En​ gine. etc. In the ACI ob​ ject model. API: Ap​ pli​ ca​ tion Pro​ gram​ ming In​ ter​ face used for pro​ gram​ ma​ ble ex​ ten​ si​ bil​ ity. ARP flood​ ing. Bridge Do​ main: An ACI con​ struct that de​ fines Layer 2 for​ ward​ ing be​ hav​ iors (Broad​ cast. and In​ ter​ nal BGP is used to peer the fab​ ric with ex​ ter​ nal Layer 3 de​ vices. The AP is the key ob​ ject used to rep​ re​ sent an ap​ pli​ ca​ tion and is also the an​ chor point for the au​ to​ mated in​ fra​ struc​ ture man​ age​ ment in an ACI fab​ ric. but com​ mu​ ni​ ca​ tion within an EPG is al​ lowed. The APIC con​ fig​ ures the in​ tended state of the pol​ icy to the fab​ ric. Clus​ ter: Set of de​ vices that work to​ gether as a sin​ gle sys​ tem to pro​ vide an iden​ ti​ cal or sim​ il​ ar set of func​ tions. . Con​ tracts: A log​ ic ​al con​ tainer for the sub​ jects which re​ late to the fil​ ters that gov​ ern the rules for com​ mu​ ni​ ca​ tion be​ tween end​ point groups. B BGP: Bor​ der Gate​ way Pro​ to​ col.

dMIT: dis​ trib​ uted Man​ age​ ment In​ for​ ma​ tion Tree. F Fault: When a fail​ ure oc​ curs or an alarm is raised. equiv​ al​ ent to a VRF. As an ex​ am​ ple. The tree con​ tains all as​ pects of the ob​ ject model that rep​ re​ sent an ACI fab​ ric. E EP: End​ point . and in ACI ver​ nac​ ul​ ar a Pri​ vate Layer 3. It is made up of a con​ cate​ na​ tion of all of the rel​ at​ ive names from it​ self back to the root of the tree. the dn would be ex​ pressed as uni/tn-Prod/ap-com​ merce​ work​ space. a rep​ re​ sen​ ta​ tion of the ACI ob​ ject model with the root of the tree at the top and the leaves of the tree at the bot​ tom. which is used to iden​ tify the end​ point.Any log​ ic ​al or phys​ ic ​al de​ vice con​ nected di​ rectly or in​ di​ rectly to a port on a leaf switch that is not a fab​ ric fac​ ing port. End​ point groups can be dy​ namic or sta​ tic. Dn: Dis​ tin​ guished name – a fully qual​ if​ ied name that rep​ re​ sents a spe​ cific ob​ ject within the ACI man​ age​ ment in​ for​ ma​ tion tree as well as the spe​ cific lo​ ca​ tion in​ for​ ma​ tion in the tree. Ex​ am​ ples in​ clude vir​ tual-ma​ chines. servers. stor​ age de​ vices. A col​ lec​ tion of end​ points that can be grouped based on com​ mon re​ quire​ ments for a com​ mon pol​ icy. named com​ merce work​ space within a Ten​ ant named Prod. in​ for​ ma​ tion about the op​ er​ at​ ional state of the af​ fected ob​ ject and po​ ten​ tial res​ ol​ u​ tions for the prob​ lem.Appendix 447 Con​ text: A Layer 3 for​ ward​ ing do​ main. DME: Data Man​ age​ ment En​ gine. . if pol​ icy ob​ ject of type Ap​ pli​ ca​ tion Pro​ file is cre​ ated. or po​ ten​ tially some other at​ tribute. etc. lo​ ca​ tion. End​ points have spe​ cific prop​ er​ ties like an ad​ dress. A fault con​ tains the con​ di​ tions. D DLB: Dy​ namic Load Bal​ anc​ ing – a net​ work traf​ fic load-bal​ anc​ ing mech​ an ​ism in the ACI fab​ ric based on flowlet switch​ ing. the sys​ tem cre​ ates a fault-man​ aged ob​ ject for the fault. EPG: End Point Group. a ser​ vice that runs on the APIC that man​ ages data for the data model.

Spine and Vir​ tual Switches plus APICs) fines net​ work man​ age​ ment tasks. FCAPS is n acronym for FCAPS: The ISO model de​ fault.448 Appendix Fab​ ric: The col​ lec​ tive end​ points as​ so​ ci​ ated with an ACI so​ lu​ tion (Leaf. load-bal​ anc​ ing method​ ol​ ogy based on re​ search from MIT in 2004. . Flowlet switch​ ing: An op​ ti​ mized. H HTML: Hy​ per​ Text Markup Lan​ guage. Used for com​ mu​ ni​ ca​ tion be​ tween dif​ fer​ ent de​ vices on the ACI fab​ ric. I IFM: In​ tra-Fab​ ric Mes​ sages. per​ for​ mance. ac​ count​ ing. Flowlet Switch​ ing is a way to use TCP’s own bursty na​ ture to more ef​ fi​ ciently for​ ward TCP flows by dy​ nam​ ic ​ally split​ ting flows into flowlets. the man​ age​ ment cat​ e​ gories Fil​ ters: Fil​ ters de​ fine the rules out​ lin​ ing the Layer 2 to layer 4 fields that will be matched by a con​ tract. Hy​ per​ vi​ sor in​ te​ gra​ tion: Ex​ ten​ sion of ACI Fab​ ric con​ nec​ tiv​ ity to a vir​ tual ma​ chine man​ ager to pro​ vide the APIC with a mech​ an ​ism for vir​ tual ma​ chine vis​ ib ​il​ ity and pol​ icy en​ force​ ment. a markup lan​ guage that fo​ cuses on the for​ mat​ ting of web pages. G GUI: Graph​ ic ​al User In​ ter​ face. se​ cu​ rity. mul​ ti​ path. and split​ ting traf​ fic across mul​ ti​ ple par​ al​ lel paths with​ out re​ quir​ ing packet re​ order​ ing. con​ fig​ ur​ a​ tion. Hy​ per​ vi​ sor: Soft​ ware that ab​ stracts the hard​ ware on a host ma​ chine and al​ lows the host ma​ chine to run mul​ ti​ ple vir​ tual ma​ chines.

. M MO: Man​ aged Ob​ ject – every con​ fig​ urable com​ po​ nent of the ACI pol​ icy model man​ aged in the MIT is called a MO. Ser​ vice nodes op​ er​ ate be​ tween Lay​ ers 4 and Layer 7 of the OSI model.) into the flow of traf​ fic.Appendix 449 In​ band Man​ age​ ment (INB): In​ band Man​ age​ ment. Loop​ back and VTEP ad​ dresses are in​ ter​ nally ad​ ver​ tised over IS-IS. This uses a front panel (data plane) port of a leaf switch for ex​ ter​ nal man​ age​ ment con​ nec​ tiv​ ity for the fab​ ric and APICs. J JSON: JavaScript Ob​ ject No​ ta​ tion. Leafs con​ nect only to hosts and spines. DLP. Con​ nec​ tiv​ ity using an in​ band man​ age​ ment con​ fig​ ur​ a​ tion. Leafs never con​ nect to each other. IDS/IPS. Load Bal​ ancers. Layer 3 Out (l3out): Layer 3 con​ nec​ tiv​ ity to an ex​ ter​ nal net​ work that ex​ ists out​ side of the ACI fab​ ric. L4-L7 Ser​ vice In​ ser​ tion: The in​ ser​ tion and stitch​ ing of VLANs/Layer 3 con​ structs of vir​ tual or phys​ ic ​al ser​ vice ap​ pli​ ances (Fire​ wall.e. the fab​ ric) op​ er​ ate at lay​ ers 1-3).. . etc. La​ bels: Used for clas​ si​ fy​ ing which ob​ jects can and can​ not com​ mu​ ni​ cate with each other.. a data en​ cap​ su​ la​ tion for​ mat that uses human read​ able text to en​ cap​ su​ late data ob​ jects in at​ tribute and value pairs. IS-IS: Link local rout​ ing pro​ to​ col lever​ aged by the fab​ ric for in​ fra​ struc​ ture topol​ ogy. L Layer 2 Out (l2out): Layer 2 con​ nec​ tiv​ ity to an ex​ ter​ nal net​ work that ex​ ists out​ side of the ACI fab​ ric. Leaf: Net​ work node in fab​ ric pro​ vid​ ing host and bor​ der con​ nec​ tiv​ ity. where as net​ work​ ing el​ e​ ments (i. IS-IS an​ nounces the cre​ ation of tun​ nels from leaf nodes to all other nodes in fab​ ric.

O Ob​ ject Model: A col​ lec​ tion of ob​ jects and classes are used to ex​ am​ ine and ma​ nip​ ul​ ate the con​ fig​ ur​ a​ tion and run​ ning state of the sys​ tem that is ex​ pos​ ing that ob​ ject model. Rep​ re​ sen​ ta​ tional State Trans​ fer (REST): a state​ less pro​ to​ col usu​ ally run over HTTP that al​ lows a client to ac​ cess a server-side or cloud-based API with​ out hav​ ing to write a local client for the host ac​ cess​ ing the API. In ACI the ob​ ject model is rep​ re​ sented as a tree known as the dis​ trib​ uted man​ age​ ment in​ for​ ma​ tion tree (dMIT).450 Appendix Model: A model is a con​ cept which rep​ re​ sents en​ ti​ ties and the re​ la​ tion​ ships that exist be​ tween them. R RBAC: Role Based Ac​ cess Con​ trol. REST​ ful: An API that uses REST. ap​ pli​ ca​ tion logic. ob​ jects and priv​ il​ ege lev​ els. Data is usu​ ally ac​ cessed and re​ turned in ei​ ther XML or JSON for​ mat. Out-of-Band man​ age​ ment (OOB man​ age​ ment): Ex​ ter​ nal con​ nec​ tiv​ ity using a spe​ cific out-of-band man​ age​ ment in​ ter​ face on every switch and APIC. and data​ base man​ age​ ment func​ tions re​ quire phys​ ic ​al or log​ ic ​al sep​ ar​ a​ tion and re​ quire net​ work​ ing func​ tions to com​ mu​ ni​ cate with the other tiers for ap​ pli​ ca​ tion func​ tion​ al​ ity. The lo​ ca​ tion that the client ac​ cesses usu​ ally de​ fines the data the client is try​ ing to ac​ cess from the ser​ vice. then using those roles in the process of grant​ ing or deny​ ing ac​ cess to de​ vices. or Rep​ re​ sen​ ta​ tional State Trans​ fer. which is a method of man​ ag​ ing se​ cure ac​ cess to in​ fra​ struc​ ture by as​ sign​ ing roles to users. . Multi-tier Ap​ pli​ ca​ tion: Client–server ar​ chi​ tec​ ture in which pre​ sen​ ta​ tion. P Port Chan​ nel: A Port link ag​ gre​ ga​ tion tech​ nol​ ogy that binds mul​ ti​ ple phys​ ic ​al in​ ter​ faces into a sin​ gle log​ ic ​al in​ ter​ face and pro​ vides more ag​ gre​ gate band​ width and link fail​ ure re​ cov​ ery with​ out a topol​ ogy change.

S Ser​ vice graph: A mech​ an ​ism within ACI that au​ to​ mates redi​ rec​ tion of traf​ fic and VLAN stitch​ ing based on de​ fined pa​ ra​ me​ ters. Spine/Leaf topol​ ogy: A clos-based fab​ ric topol​ ogy in which spine nodes con​ nect to leaf nodes. Spine: Net​ work node in fab​ ric car​ ry​ ing ag​ gre​ gate host traf​ fic from leafs. a name of a spe​ cific ob​ ject within the ACI man​ age​ ment in​ for​ ma​ tion tree that is not fully qual​ if​ ied. . A Rn would need to be con​ cate​ nated with all the rel​ at​ ive names from it​ self back up to the root to make a dis​ tin​ guished name. leaf nodes con​ nect to hosts and ex​ ter​ nal net​ works. a sub​ net de​ fines the IP ad​ dress range that can be used within the bridge do​ main. which then be​ comes use​ ful for nav​ ig ​a​ tion. Sub​ nets: Con​ tained by a bridge do​ main or an EPG.Appendix 451 Rn: Rel​ at​ ive name. T Ten​ ants: The log​ ic ​al con​ tainer to group all poli​ cies for ap​ pli​ ca​ tion poli​ cies. Su​ per​ vi​ sor: Switch mod​ ule that pro​ vides the con​ trol plane for the 95xx switches. Any ser​ vices that are re​ quired are treated as a ser​ vice graph that is in​ stan​ ti​ ated on the ACI fab​ ric from the APIC. if an Ap​ pli​ ca​ tion Pro​ file ob​ ject is cre​ ated named “com​ merce​ work​ space”. As an ex​ am​ ple. Ser​ vice graphs iden​ tify the set of net​ work or ser​ vice func​ tions that are needed by the ap​ pli​ ca​ tion. but with​ out con​ text. the Rn would be “ap-com​ merce​ work​ space” be​ cause Ap​ pli​ ca​ tion Pro​ file rel​ at​ ive names are all pref​ aced with the let​ ters “ap-”. Sub​ jects: Con​ tained by con​ tracts and cre​ ate the re​ la​ tion​ ship be​ tween fil​ ters and con​ tracts. A Rn is sig​ nif​ ic ​ant to the in​ di​ vid​ ual ob​ ject. it’s not very use​ ful in nav​ ig ​a​ tion. See also the Dn de​ f​ in ​i​ t​ ion. and rep​ re​ sent each func​ tion as a node. con​ nected only to leafs in the fab​ ric and no other de​ vice types.

vPC: vir​ tual Port Chan​ nel. . Each seg​ ment rep​ re​ sents a unique Layer 2 broad​ cast do​ main.452 Appendix V Vir​ tu​ al​ iza​ tion: Tech​ nol​ ogy used to ab​ stract hard​ ware re​ sources into vir​ tual rep​ re​ sen​ ta​ tions and al​ low​ ing soft​ ware con​ fig​ ura​ bil​ ity. in which a port chan​ nel is cre​ ated for link ag​ gre​ ga​ tion. An ACI VXLAN header is used to iden​ tify the pol​ icy at​ trib​ utes if the ap​ pli​ ca​ tion end​ point within the fab​ ric. X XML: eX​ ten​ si​ ble Markup Lan​ guage. VRF: Vir​ tual Rout​ ing and For​ ward​ ing . and every packet car​ ries these pol​ icy at​ trib​ utes. but is spread across no more or less than 2 phys​ ic ​al switches. a markup lan​ guage that fo​ cuses on en​ cod​ ing data for doc​ um ​ents rather than the for​ mat​ ting of the data for those doc​ um ​ents.A Layer 3 name​ space iso​ la​ tion method​ ol​ ogy to allow for mul​ ti​ ple con​ texts to be de​ ployed on a sin​ gle de​ vice or in​ fra​ struc​ ture. A 24bit VXLAN seg​ ment ID (SID) or VXLAN net​ work iden​ ti​ fier (VNID) is in​ cluded in the en​ cap​ su​ la​ tion to pro​ vide up to 16 mil​ lion VXLAN seg​ ments for traf​ fic iso​ la​ tion or seg​ men​ ta​ tion. VXLAN: VXLAN is a Layer 2 over​ lay scheme trans​ ported across a Layer 3 net​ work.

​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-installation-guides-list. ACI Install and Upgrade Guides http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ white-paper-listing.​ cisco.​ cisco.​ html .​ html ACI Get​ ting Started .​ io/​ aci-troubleshooting-book/​ ACI White Pa​ pers http://​ www.Appendix 453 Reference Material Top​ ics that are out​ side of the scope of this op​ er​ at​ ions guide may be doc​ um ​ented in other places.​ html https://​ datacenter.​ html ACI De​ sign Guide http://​ www.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-troubleshooting-guides-list.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-installation-and-configurationguides-list.Fab​ ric Ini​ tial​ iza​ tion http://​ www.​ cisco.​ cisco.​ html#_​ Toc405844675 ACI Trou​ bleshoot​ ing Guides http://​ www. This sec​ tion in​ cludes links to other help​ ful ref​ er​ ence doc​ um ​en​ ta​ tion for fur​ ther read​ ing and view​ ing.​ cisco.​ github.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c11-731960.

​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ solution-overview-c22-732445. Pre​ sen​ ta​ tions and Train​ ings www.​ html ACI So​ lu​ tions Overview http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ application-centricinfrastructure/​ customer-case-study-listing.​ html .​ cisco.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ solution-overview-listing.​ com/​ web/​ techdoc/​ aci/​ acimatrix/​ matrix.​ html ACI Demos.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ application-centricinfrastructure/​ sales-resources-list.​ io/​ acitoolkit/​ ACI Com​ pata​ bil​ ity Tool http://​ www.​ cisco.​ com/​ c/​ en/​ us/​ solutions/​ data-center-virtualization/​ applicationcentric-infrastructure/​ presentations-listings.​ cisco.​ html ACI Ecosys​ tem Com​ pata​ bil​ ity List http://​ www.​ cisco.​ cisco.​ github.454 Appendix ACI Case Stud​ ies www.​ cisco.​ com/​ c/​ en/​ us/​ support/​ switches/​ application-virtual-switch/​ products-installation-and-configuration-guides-list.​ html AVS Con​ fig​ u​ ra​ tion and Scal​ a​ bil​ ity Guides http://​ www.​ html ACI Toolkit http://​ datacenter.​ html ACI Part​ ners and Cus​ tomers Pre​ sen​ ta​ tions http://​ www.

​ cisco.​ html APIC Com​ mand-Line In​ ter​ face User Guide http://​ www.​ cisco.​ cisco.​ org/​ en/​ latest/​ Cobra GitHub http://​ github.​ html .​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-command-reference-list.Appendix 455 AVS Topolo​ gies and So​ lu​ tion Guide http://​ www.​ com/​ c/​ en/​ us/​ solutions/​ collateral/​ data-center-virtualization/​ application-centric-infrastructure/​ white-paper-c07-732033.​ cisco.​ cisco.​ readthedocs.​ com/​ c/​ en/​ us/​ support/​ cloud-systems-management/​ applicationpolicy-infrastructure-controller-apic/​ products-configuration-examples-list.​ com/​ c/​ en/​ us/​ support/​ switches/​ application-virtual-switch/​ products-technical-reference-list.​ youtube.​ com/​ datacenter/​ cobra Con​ nect​ ing ACI to Out​ side Layer 2 and 3 Net​ works http://​ www.​ html Fab​ ric Con​ nec​ tiv​ ity Video https://​ www.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ L4-L7_​ Services_​ Deployment/​ guide/​ b_​ L4L7_​ Deploy.​ html APIC Layer 4 to Layer 7 Ser​ vices De​ ploy​ ment Guide http://​ www.​ html Cobra Docs http://​ cobra.​ com/​ watch?​ v=_​ iQvoC9zQ_​ A Nexus CLI to Cisco APIC Map​ ping http://​ www.

​ cisco.​ com/​ c/​ en/​ us/​ td/​ docs/​ switches/​ datacenter/​ aci/​ apic/​ sw/​ 1-x/​ mib/​ list/​ mib-support.​ html .456 Appendix POST​ man getpostman.​ com http://www.​ Sup​ ported SNMP MIB List http://​ www.