# CPE Chef and Server Management



# Getting set up to work on this repo

Prereqs:

```
Install rbenv: https://github.com/rbenv/rbenv#installation
Install ruby 2.5.1 with rbenv
```

```
git clone git@git-aws.internal.justin.tv:CPE-Ops/cpe-core.git

cd cpe-core
rbenv shell 2.5.1 && gem install bundler
bundle install
teleport-bastion login
bundle exec sloth env sync

# After sloth runs, cookbooks will be set up at -
cpe-core/cookbooks/*

```

# Using Test Kitchen to test changes
```
# Get credentials set up from Isengard for twitch-cpe+infra-dev@amazon.com - 432112285195


# Create your ec2 instance to test chef
cd cpe-core/cookbooks/<cookbook you're working on>
kitchen list
kitchen create <name of test>

# Wait until you see something like...
    Waited 460/900s for instance <i-0a7b60e5cf2bdbdea> to become ready.
    Waited 0/900s for instance <i-0a7b60e5cf2bdbdea> to fetch windows admin password.
    Retrieved Windows password for instance <i-0a7b60e5cf2bdbdea>.
    EC2 instance <i-0a7b60e5cf2bdbdea> ready (hostname: ec2-54-183-180-89.us-west-1.compute.amazonaws.com).

# Then hit Ctrl+C to exit out (you won't be able to connect)

# Edit .kitchen/<test-name>.yml
# Change hostname to 127.0.0.1
hostname: 127.0.0.1

# In SEPARATE terminal run
ssh -L 5985:<DNS name for instance that was created>:5985 bastion01a-pdx.cpe.wtf

# Back in the first terminal 
kitchen converge
# kitchen will fail because chef registers reboots as failures so wait for the reboot to finish and...
kitchen converge
# And you might need to run again
kitchen converge
```


# Troubleshooting

Problem while running `kitchen converge`:

```
>>>>>>     Failed to complete #converge action: [execution expired] on desktop-agent-windows-2016-vs2017
```

Solution: Get new refreshed credentials out of Isengard


# Creating your user account on the Chef Server

1. Connect to TwitchVPN.
2. Make sure you are running sloth >= 1.0.2 using `sloth --version`. If you are not, run `git pull && bundle install` within the `cpe-core` checkout directory.
3. Make sure you have the `id_rsa_litany` key saved as `~/.ssh/id_rsa_litany`. You can find the `id_rsa_litany` key in 1Password.
4. Connect to wpa2 or Amazon VPN.
5. Disable teleport bastion.
6. In your `cpe-core` checkout directory, run `sloth auth --name <your full name> --email <your work email>`.
7. Verify that you now have a valid key in `cpe-core/.chef/private.pem`.

`sloth auth` will run the remote commands on the chef server to create your user and generate a unique private key for you to be able to upload and apply cookbooks.


# Uploading your cookbook

To use the helper scripts in cpe-core for uploading the cookbooks, you’ll want to have in your `~/.bash_profile` or `~/.zshrc`:
```for f in ~/workspace/cpe-core/env.d/*.sh; do source $f; done```

Make sure you've got `cpe-core/.chef/private.pem` set up correctly.

Once you've merged your changes into `master`:

```
cd cpe-core
git checkout master
git pull
cd cookbooks/<cookbook to release>/
bump_metadata_version
git add -A .
git commit -m "Bumping version"
git tag <new version number>
git push && git push --tags

# this is only for app_ cookbooks
upload_and_apply

# for other cookbooks:
cd <whatever is using the cookbook>
berks update <name_of_the_cookbook_you_just_updated>
# repeat the above steps to tag and publish new versions of any related app_ cookbooks to actually deploy the changes
```


# Working with encrypted data bags and chef vault

Data bags and Chef Vault should only be used for non-AWS hosts, so primarily just MacOS systems. We have to set up the data bag for Test Kitchen use before adding the secrets to the Chef Server to prevent conflicts.

The `test_kitchen_databag_secret` is available via 1Password in the **Mac Agent Cookbook Dev** vault.

Note: Before you can admin/edit existing Chef Vault items, you will need an existing admin to add you the secret you need to add or update.

## Test Kitchen

Note: Run these commands from your cookbook directory.

- Create a new data bag
```
knife data bag create <bag> <item> --secret-file kitchen/test_kitchen_databag_secret

# example:
# knife data bag create desktop macos_prod_keys --secret-file kitchen/test_kitchen_databag_secret
```

- Export and store the data bag in the cookbook
```
knife data bag show <bag> <item> -Fj > cookbooks/<cookbook>/data_bags/<bag>/<item>.json

# example:
# knife data bag show desktop macos_prod_keys -Fj > cookbooks/app_teamcity_agent_macos/data_bags/desktop/macos_prod_keys.json
```

## Chef Vault

Assuming you've run the above commands, run this from your cookbook directory.

- Save the unencrypted JSON output from your data bag to a file temporarily. You will need this when importing the data to Chef Vault.
```
knife data bag show <bag> <item> -Fj --secret-file ../../kitchen/test_kitchen_databag_secret > do_not_commit_this_secret.json
```

- Create the Chef Vault item
```
knife vault create <bag> <item> -J do_not_commit_this_secret.json -S "<pattern for allowed client node names>" -M client -A <list of admin users>

# example:
# knife vault create desktop macos_prod_keys -J do_not_commit_this_secret.json -S "name:tcagent-*-sna*.cpe.wtf" -M client -A tbriggs,thohanso,kaelten,richduar
```

- Delete the temporary JSON file
```
rm do_not_commit_this_secret.json
```

---


# The following is DEPRECATED: YMMV
Get [rbenv](https://github.com/rbenv/rbenv)  
Get ~/.ssh/cpedev_litany from 1Password  

```bash
xcode-select --install
bundle install
sloth auth -n "First Last" -e 'you@justin.tv'
sloth env sync
```

You need to create a 'test-kitchen' aws credentials profile with admin access to 432112285195 (twitch-cpe+infra-dev@amazon.com):
```bash
aws --profile test-kitchen configure
```

If you do new cookbook or container development, the generate commands now authenticate with github via a netrc file.  It is recommenad to use a Personal Access Token with the repo namespace permissions.

At ~/.netrc you'll need a section that is similar to this

```
machine git-aws.internal.justin.tv
	login <username>
	password <access token>
```

You're going to want these helpers:

#### ZSH
```bash
echo 'for f in ~/Development/cpe-core/env.d/*.sh; do source $f; done' >> ~/.zshrc
```

#### Bash
```bash
echo 'for f in ~/Development/cpe-core/env.d/*.sh; do source $f; done' >> ~/.bashrc
```
