a Cisco SD-WAN primer

The underlying concept of software-defined WAN is that a router’s management and control planes are abstracted from the hardware and reside in the cloud. That router abstraction allows a company to join multiple physical locations into a single network over redundant, low-cost, telecommunication links. The idea is that a router chassis is drawn apart and distributed across a distance — the I/O modules remain physically on-premises while the switch fabric is stretched over long distance media (via TCP/IP, MPLS, 4G/LTE, etc.) and the backplane is virtualized on cloud servers.

The software of a router can be viewed as having three component systems: 

1) The Data Plane refers to all the functions and processes that forward packets and frames from one interface to another — routed data. In SD-WAN, the data plane is tunneled between on-premises edge-routers. Cisco calls them vEdges (named for Viptella, an SD-WAN company that Cisco bought) or cEdges. 

2) The Control Plane refers to all the functions and processes that determine which path to use — routing data. in SD-WAN, the control plane is shared between edge-routers via cloud-based controllers. Cisco’s controllers are called vSmart, and Cisco SD-WAN also requires another cloud-based server called vBond for network discovery. 

3) The Management Plane refers to the configuration used to control and monitor devices. In SD-WAN, the management plane also resides in the cloud. Cisco’s management server is called vManage.

by stephen

Updating VMware vCenter Server Appliance

Terminology

Baseline
A predefined or custom set of patches, extensions, or upgrades used to update an ESXi host or VM.

Dynamic vs Fixed
Dynamic baselines have selectable criteria for which patches are included.

Fixed baselines allow you to choose specific patches to include.

Attaching / Detaching
Baselines (or baseline groups) are attached to hosts (or clusters of hosts), and then you scan. What you’re attaching is actually just the metadata of the baseline.

Baseline Group
A bunch of baselines, or an aggregate of individual patch and upgrade baselines. An upgrade involves a release number change (e.g. vSphere 6.5 to 6.7). An smaller change uses a patch or update (e.g. vSphere 6.0 Update 1).

Scanning
The process of checking if the host(s) or VMs need any of the patches (etc.) in the baseline (or group).

Staging
The process of downloading the actual data of the patches ahead of remediating.

Remediating
The process of applying the patch, extension, or upgrade to an ESXi host or VM. The data will be downloaded if needed — if you didn’t stage the patches. So you need to attach, then scan, stage if you want, and then remediate.

Patch Metadata
Index files stored in the patch repository that list all known patches available for use by Update Manager.

Patch Repository
The directory on the Update Manager Server that holds the Patch Metadata as well as the patches.

Host Extension
Additional software to apply to a host.

VAMI
vCenter Server Appliance Management Interface.
vCenter Server Appliance is a VM that runs in ESXi and manages the entire vSphere environment, and you usually access it via its web UI. But it also has a separate web ui for managing itself, which is accessed at its URL on port 5480.
vsphere.company.com:5480

To update vCenter

(For example from build 6.7.0.10000 to 6.7.0.20000.)

Go to https://my.vmware.com/group/vmware/patch#search

Choose vc and download the latest patch.

In this case:

  • Release name:
  • VC-6.7.0-update01.Appliance-Patch
  • release date
  • 10.16.2018
  • File name:
  • VMware-vCenter-Server-Appliance-6.7.20000-10244745-patch-FP.iso
Then attach the iso to the Vcenter Appliance VM.
  • Actions > Edit Settings > CD/DVD drive > attach datastore or client device file.
Make a snapshot

just in case.

Check for Updates
  • Log into the VAMI as root:
  • https://:5480
  • Navigate to Update > Check Updates > Check CD-ROM + URL
  • Twirl down the selected update and click RUN PRE-UPDATE CHECKS
  • Click Stage only
  • Click INSTALL
  • You’ll need to make a small backup.

To make the backup

You can back up to a server via SCP, HTTPS, HTTPS, FTP, or FTPS (but not SFTP).
Enter the server address like so:
– protocol://:/path
– scp://backup.server.info:22/~
You’ll enter the user name and password separately.

vcenter backup

If you get this ERROR:

vcenter_backup_error

Log into vCenter via SSH as root.

$ ssh root@vsphere.company.com

VMware vCenter Server Appliance 6.7.0.10000

Type: vCenter Server with an embedded Platform Services Controller

Password:
Connected to service

* List APIs: "help api list"
* List Plugins: "help pi list"
* Launch BASH: "shell"

Command>

Enable a BASH shell:

Command> shell

Shell access is granted to root

root@vcenter [ ~ ]#

Check the status of vCenter services with:

  • service-control --status
root@vcenter [ ~ ]# service-control --status

Stopped:
vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-postgres-archiver vmware-rbd-watchdog vmware-vcha vsan-dps
Running:
applmgmt lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-analytics vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-perfcharts vmware-pod vmware-rhttpproxy vmware-sca vmware-sps vmware-statsmonitor vmware-sts-idmd vmware-stsd vmware-updatemgr vmware-vapi-endpoint vmware-vmon vmware-vpostgres vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsphere-client vsphere-ui
root@vsphere [ ~ ]#

Start the service with

  • service-control --start
root@vcenter [ ~ ]# service-control --start vmware-postgres-archiver
Operation not cancellable. Please wait for it to finish...
Performing start operation on service vmware-postgres-archiver...
Successfully started service vmware-postgres-archiver
root@vcenter [ ~ ]#
by stephen

Add a DMARC record to your mail server.

DMARC is Domain-based Message Authentication Reporting and Conformance
DMARC uses SPF and DKIM to tell receiving email servers what to do (accept, reject, or quarantine) with messages that purport to be from your domain. It also reports to you.
DMARC is defined by a DNS record (TXT) and specifies:
  • An action policy for messages failing both SPF and DKIM. DMARC passes when either SPF or DKIM passes.
  • In Microsoft’s implementation of DMARC for incoming mail (which we don’t control), both SPF and DKIM must pass.
  • Allows for partial checking, e.g. 5 percent of mail.
To implement DMARC, you must first set up SPF and DKIM.
In Office 365, DMARC for incoming mail is done for you. But you must set up DMARC for your domain yourself.

SPF – Sender Policy Framework

SPF is a DNS record (TXT) that specifies which IP addresses are allowed to send mail on your domain’s behalf.
You need a single spf record for your domain, which will contain all the IP addresses that are allowed to send mail on your behalf.
Example record:

v=spf1 ip4:192.0.2.0/24 ip4:198.51.100.123 a -all

Example for Office 365:

v=spf1 include:spf.protection.outlook.com -all

Even if your domain doesn’t send mail, it is still vulnerable to spoofing and should have this SPF record:

v=spf1 -all

DKIM – DomainKeys Identified Mail

DKIM is a DNS record (TXT) that lets a domain associate its name with an email message by affixing a digital signature (using PKI) to it.
A valid signature guarantees that some parts of the email (possibly including attachments) have not been modified since the signature was affixed.
DKIM provides for two distinct operations, signing and verifying.
DKIM specification allows signers to choose which header fields they sign, but the From: field must always be signed.
You might need multiple DKIM records for your domain if you have different hosts, domains, or subdomains that you want to allow to send mail for you. Each host needs its own key specified in a separate TXT record, and each host is identified with a unique selector name.
Generate the keys with:

openssl genrsa -out private.key 2048
openssl rsa -in private.key -pubout -out public.key

Install the private key on the MTA/Email sending systems.
Publish the public key in the DKIM record with your registrar:
Example DKIM record:
  • type:  TXT
  • host:  selector1-dzsolutions-com._domainkey
  • Points to address or value:  v=DKIM1; k=rsa; p=<public_key>; n=1024,1453276987,1

DMARC

You want to ease into your DMARC policy to make sure it’s working correctly.
  • First, make a no-action policy (p=none;) that just collects reports. You’ll see who is sending mail on your behalf, and be able to correct for any senders you left out.
  • Second, switch to a low percentage quarantine policy (p=quarantine; pct=5;). It tells receiving servers to check only 5 percent of the mail, and if DMARC fails, send it to the spam folder. That way, most of the mail will go through no matter what, and your reports will show how well it’s working for the 5 percent. You can make adjustments until it works right.
  • Third, ease into rejecting (p=reject; pct=5;). Check your reports, adjust things, and increment up to 100%
no-action DMARC record:
  • Type:  TXT
  • Host: 
  • Value:  v=DMARC1; p=none; rua=mailto:admin@your_domain.com; ruf=mailto:admin@your_domain.com; sp=reject; ri:84600;
Where:
  • v is the version.
  • p is policy for handling pass/fail messages: none | quarantine | reject.
  • rua specifies where to send aggregate reports.
  • ruf specifies where to send forensic reports.
  • sp is the subdomain policy.
    If you have no subdomains for mail, it is best to reject.
  • ri is the time period for sending reports; default is 84600 seconds (24 hours).
Check your DMARC record at:
View the reports using https://dmarcian.com/ or https://mxtoolbox.com/domain/; they have validation tools for all DNS records.

by stephen

Create your own What’s My IP service.

Here’s an easy way to make your own IP echoing service on your own website if you’re hosting that website with Apache on a Debian-based Linux distro.

  • 1. Add a new site to the apache configuration. The file should be named as the URL with `.conf` at the end:

nano /etc/apache2/sites-available/myip.stephenfranklin.info.conf

And it should look like this:

ServerAdmin stephen@stephenfranklin.info
ServerName myip.stephenfranklin.info
DocumentRoot /var/www/myip.stephenfranklin.info/htdocs
ErrorLog /var/www/myip.stephenfranklin.info/logs/error.log
CustomLog /var/www/myip.stephenfranklin.info/logs/access.log combined
  • 2. Enable the site and reload apache:

root@www:sites-available$ a2ensite myip.stephenfranklin.info.conf
Enabling site myip.stephenfranklin.info.
To activate the new configuration, you need to run:
service apache2 reload
root@www:sites-available$ service apache2 reload
* Reloading web server apache2 *
root@www:sites-available$
  • 3. Then create the appropriate directories for that site in `/var/www/`:

mkdir -p /var/www/myip.stephenfranklin.info/htdocs
mkdir /var/www/myip.stephenfranklin.info/logs/

Create an index.php script in the htdocs directory:

nano /var/www/myip.stephenfranklin.info/htdocs/index.php
And put this in there:


< ?php echo $_SERVER['REMOTE_ADDR']."\n"; ?>
  • 4. Test it from another machine with `curl`:

stephen@mac:~$ curl -s http://myip.stephenfranklin.info/
191.145.122.55
stephen@mac:~$

That’s it! Try it out from a browser:
1_myip.stephenfranklin.info

by stephen

/dev/xvda1 should be checked for errors

Amazon EC2 Ubuntu instances sometimes display this message to the terminal when you ssh in:

**** /dev/xvda1 should be checked for errors ****

/dev/xvda1 refers to the first partition of the hard drive (which is actually a virtualized hard drive running under Xen). This partition (likely, the only partition) contains your machine’s file system. To run a file system check, we like to use the fsck command. But this can only be used when the file system is not in use, so we’ll tell the system to run it upon reboot. And to do that we need to temporarily change two files: /etc/fstab (the file system table), and /etc/default/rcS (a small configuration file).

About fstab:

LABEL=cloudimg-rootfs / ext4 defaults,discard 0 0
/var/swap.1 swap swap defaults 0 0

The file system table (/etc/fstab) contains information that allows the machine to automatically mount disk partitions. That is, it prepares them for access by assigning them a location (mount point) on the file system tree. For the partition that contains the file system, the table assigns it to the root directory, which is designated by a forward slash / in the second field.

The first field in an fstab entry contains the device node, in this case by it’s file system label. If you look in /dev, you’ll see many devices, including the disk (“xvda”) and its partitions (“xvda1”). You’ll also see the \dev\disk directory, and if you look in there you’ll see that you can list devices by-label or by-UUID.

Enter
ls -l /dev/disk/by-label
and you’ll see something like:

lrwxrwxrwx 1 root root 11 Jul 25 21:15 cloudimg-rootfs -> ../../xvda1

which shows that the label cloudimg-rootfs is just a link to the device /dev/xvda1.

The last field (in the fstab entry, not the ‘ls -l output) is what we’re interested in changing here. It’s the “pass number” and indicates the order in which file system checks are done. “0” tells it not to run fsck, “1” tells it to run fsck on the root partition, and “2” tell it to check other partitions. We’re going to change it to “1”.

About rcS:

The /etc/default/rcS file contains six variables that change the behavior of various boot scripts. (The “rc” term, as in .bashrc and rc.local, has some history behind it and may stand for “run commands” or “run configuration,” or something like that.) The variable we’re interested in is FSCKFIX, which, when enabled with a “yes”, will tell fsck to always repair the file systems without asking for further permission.

The repair:

  1. Edit fstab: 
    sudo vim /etc/fstab

    Look for the line describing the root (/) file system, and change the last ‘0’ to a ‘1’.

    LABEL=cloudimg-rootfs   /    ext4   defaults,discard    0 1
/var/swap.1 swap swap defaults 0 0

    To do that with vim:
    a. Enter insert mode by hitting ‘i’.
    b. Exit insert mode with the ‘esc’ key.
    c. Save and exit by typing :wq.

  2. Edit rcS: 
    sudo vim /etc/default/rcS

    Look for the line #FSCKFIX=no and under it type:

    FSCKFIX=yes
  3. Add in the /forcefsck trigger. 
    sudo touch /forcefsck

    In the script /etc/init/mountall.conf, there’s a line which looks for the /forcefsck file, and upon finding it, will direct fsck to perform a full file system check.

  4. Reboot. 
    sudo reboot
  5. Delete the line FSCKFIX=yes in the rcS file.
  6. Change the line in /etc/fstab back to a ‘0’.
    (The /forcefsck file was deleted for you by mountall.conf.)
by stephen

Revert to old version of Google Chrome (OS X)

The latest version of Chrome (39) introduced some annoying behavior when revealing and hiding the top bar, so annoying that I wrote up a bug report, and decided to keep using the previous version (37). But I kept getting this warning when starting the application:

Your profile can not be used because it is from a newer version of Google Chrome. Some features may be unavailable. Please specify a different profile directory or use a newer version of Chrome.

And my search settings (for which I have a lot of nifty shortcuts) were vaporized every time.

Here’s how I solved that problem:

cd ~/Library/Application Support/Google/Chrome/Default
mv Web\ Data Web\ Data.old

The warning is gone, and upon signing in to Chrome, my search settings were restored.

by stephen

Apple EarPods pin-out

Today I soldered a new jack onto my Apple EarPods. I have the EarPods with remote and mic. They’d been stretched and yanked too many times and were randomly pausing or advancing my podcasts. I bought a 50 peso knock-off pair (which worked but sounded like 50 peso earphones), and I cut off the jack. There are tear-downs online but they don’t mention the pin-out, so I had to work it out for myself. For future reference, here it is:

Pinout for the Apple EarPods with Remote and Mic
Tip – Left – Green
Ring1 – Right – Red
Ring2 – Ground – Green/Gold & Red/Gold & Red/Green*
Ring3 – Mic – Gold*

  • There’s a wire that has red and green strands wrapped around gold strands. Separate them. The red/green is ground. The gold is for the microphone.

(By gold, I mean the color of the insulation, not the metal.)

by stephen

A word prediction web app!

As the final project of the Coursera / Johns Hopkins data science series that I’ve just completed, I made this neat app.

See my code here.

It’s similar to the predictive keyboards you can buy for your smart phone, but I added in a fun little babbling tool.

The Coursera data science specialization is centered around the R statistical programming language and consists of ten classes that cover programming in R, statistics and probability, processing and analyzing data, machine learning, etc. I jumped into it in its first iteration, so I was somewhat of a guinea pig, but the series was very well-organized and entirely fascinating, except for the last class, which was essentially untaught and a bit overwhelming. It involved learning several topics from a huge field (Natural Language Processing) of which most of the students knew nothing, and then learning how to process a large amount of data (just over 100 million words), which my macbook pro (with 8 GB of RAM) proved incapable of doing (I ended up renting an Amazon EC2 server with 16 GB of RAM), and then writing a prediction algorithm and web application. Anyway, I did it! It’s been done before, but now I’ve done it too.

by stephen