## Grid Maintenance

Need to perform maintenance on Grid? Consider this your guide.

##### How to reboot a box:
1. Go to Amazon EC2 Console for [twitch-web-dev](https://twitch-web-dev.signin.aws.amazon.com/console).
2. Search `qa/smoca`
3. Find box correlating to the one high on CPU Utilization. Right click, Instance State, Reboot.

##### Cycling Nodes

Sometimes you want to perform maintenance to a node without disrupting tests from running.

Used nodes are indicated when the WebDriver icon is grayed out.

![Grid Node Details](/resources/docs/images/grid_node_details.png)

There is currently no way to shut a node down whenever it is becomes unused. You must manually wait for the node to be released.

1. Identify which node to restart and grab its IP from the AWS Console. This IP is also listed on [Grid](http://grid.us-west2.justin.tv/grid/console).
2. Optional: Bring up more nodes to reduce the likelihood of the node you want to restart being taken.
3. Watch test output. Wait for the tests to finish when the timing summary is displayed
4. Refresh Grid waiting for that box to become unused
5. Restart node when safe

#### Running commands across many boxes

Need to shut down all containers? Or bring up debug containers on all nodes?

See [PSSH Documentation](updating_container.nodes.md#via-pssh)
