# C7-go
Does for your configuration what SASS does for your CSS

# Loading a Config

The simple form of loading a config is:

```
import (
  "code.justin.tv/amzn/C7-go/c7"
  "code.justin.tv/amzn/C7-go/resolvers"
)

func Load() (*c7.C7, error) {
	region := os.Getenv("REGION")
	stage := os.Getenv("STAGE")
  set, err := resolvers.ResolveFile("/etc/hats/config/config.c7")
	if err != nil {
	    return nil, err
  }
  return c7.NewC7(region, stage), nil
}
```

However C7 configs aren't that useful by themselves. Read on for more information.

## Reading values
C7 takes an opinionated stance on on configuration. It forces you into a paradigm where you have to declare a schema for your configuration. You also don't get access to the raw config hashes or values. This prevents several common mistakes that are often made with configs at twitch:
- Configs which don't parse
- Configs which don't have the correct types
- Having a config singleton with access spider-webbed across the codebase
 
Instead C7 makes you pass it an annotated structure to fill out the configuration you need. You should do this very early in your application's lifecycle. If C7 cannot satisfy the values requested, it will return an error.

And resolve it from your app config like the following:

```
type SQSQueuePollerConfig struct {
  ARN         string `c7:"arn"`
  Topic       string `c7:"topic"`
  Concurrency in64 `c7:"concurrency"`
}

sqsConfig := SQSPollerConfig{}

cfg := Load()
err := config.FillWithNamespace("PurchaseSQS", sqsConfig)
if err != nil {
    log.Fatal(err)
}

// Use the config

fmt.Printf("Purchase SQS: ARN: %s, Topic: %s, Concurrency: %d\n", sqsConfig.ARN, sqsConfig.Topic, sqsConfig.Concurrency)
```

With the corresponding config:

```
*.*:
  PurchaseSQS.arn = "arn:other:arn";
  PurchaseSQS.topic = "other_topic";
  PurchaseSQS.concurrency = 34;
```

#Syntax

The general syntax of rules like the following:

```
[scopes]:namespace:key = NUMBER|BOOL|STRING|DURATION;
```

Where:
* Number is an int64
* Bool is a boolean
* String is a quoted string
* Is a golang duration (https://golang.org/pkg/time/#ParseDuration) with types of ms, s, m, h.

Scopes are optional and if you omit them the C7 will use the scope defined on the previous rule.

## Scopes
A series of strings separated by "." of the form [a-Z][a-Z0-9-_]*

Scopes are arbitrary in C7. However they are usually specific to the domain in which the app config is used, which is usually where most users will see it. For instance for defining service or client configuration values in twitch we have 2 keys: Region and Stage. 

In config files we can use * to match any of the scopes. However when resolving an application config, we specify all scopes in the
```
    return set.NewC7("us-west-2", "production"), nil
```
call. This takes the generic bag of properties and gets the values that match these scopes. The matching rule is the app config value that is the most specific wins. For instance when asking for (us-west-2, prod) app config rules would be matched in descending priority order with left priority on scopes in ties:
```
us-west-2.prod:namespace:key = 1;
us-west-2.*:namespace:key = 1;
*.prod:namespace:key = 1;
*.*:namespace:key = 1;
```
In general scopes don't have hierarchical precedence but because we have to pick a winner we pick from left to right when there are ties. Eg if we had:

```
*.prod:namespace:key = 2;
us-west-2.*:namespace:key = 1;
c7.NewC7("us-west-2", "prod")
```
This would resolve to 1, as left hand selectors have precedence in a tie.


### Namespaces
A single string of the format [a-Z][a-Z0-9-_]*
Namespaces should be unique across the use of the config and usually refer to a system. All separate groups of configs should use a different namespace in an application. Essentially, with shared configurations, this means that a namespace should be globally unique within the company in most cases. This is to keep from having a global namespace and having collisions across the dependency tree.

An example of this is the HATSClient would use the HATSClient namespace (which no one else would use), to vend it's endpoint, port, and other settings.
### Keys
One string of the form [a-Z][a-Z0-9-_]* seperated by ".". Keys define the specific config value you need. 

# Config Resolution

C7 Provides several ways to resolve configurations:
* Filesystem scanning
* Vendored compilation
* Brazil compilation

## Filesystem scanning

Often in dev mode you'll want to scan for config files and glom them together, do that like this:

```
import (
  "code.justin.tv/amzn/C7-go/C7"
  "code.justin.tv/amzn/C7-go/resolvers"
)

func Load() (*C7.C7, error) {
	region := os.Getenv("REGION")
	stage := os.Getenv("STAGE")
	
	set, err := resolvers.ResolveFilesWithExtension(".", ".c7")
	if err != nil {
		return nil, err
	}
  return c7.NewC7(region, stage), nil
}
```

This will scan your file system for any .c7 file below your working directory and load them into one config closure. This can be useful when you want to iterate on your development flow. However it is not the recommended way to load configurations. 

## Vendored Compilation
For vendored configuration we provide a tool which you point at your c7 directory and your vendor folder (or others). This tool will find all .c7 files, with vendor loading first and c7 loading last. Later config values override earlier ones, putting your application's configurations over vendored. The tool then outputs a .c7 file in your build artifacts.

When delivering your application you only need to package up this compiled c7 file, your go application, and any other artifacts (libraries, images, etc).

This means that when you vendor in a golang package that provides a config, you should make sure you vendor in the exported c7 files which they provided as well. At the very least don't strip them.

## Brazil Compilation
Brazil compilation is a specialized form of Vendored compilation. For brazil we provide a specialized build tool which uses some Brazil Path recipes (from C7BrazilPathRecipes) to find all of the C7 files included in your build closure. We then aggregate all of these configurations, and output them to one single .c7 file.

# Config Validation

## Basic validation rules
C7 will guarantee that if a value is present it is of the right type.

## Required Fields
C7 does not enforce required fields in this release. It was determined, looking at example configs, that it was very difficult
and possibly of little actual value to enforce required fields as they widely vary across stages, and often require complex 
chaining logic, such if c1 && c2 then c3 is required. We chose to instead hold off on this.

## Tuple based validation tool

C7 also provides tooling which allows you to, at compile time, verify that your config will at least be loadable: c7-config-validation-test-generator.

To use this tool:

* Make sure you have a config receiver object or objects declared in a package, we recommend c7s (C7Schema), such as:

```
package c7s

type Config struct {
	BasePrice int64 `c7:"basePrice"`
}
```

* Place a tuples.json file in the root of your project:

```
[
  {
    "type": "golang.a2z.com/FultonGolangTemplateLambda/c7s.Config",
    "selectors": [
      "us-west-2.production",
      "us-west-2.staging",
      "us-west-2.development"],
    "namespaces": ["FultonGolangTemplateLambda"]
  }
]
```

For every config receiver object you have that you wish to validate, you should list the service tuples and the namespaces you wish to validate them against.

* Have a .c7 file you wish to validate against. For obvious reasons, this should come AFTER config compilation.

* An optional test folder (usually goes into c7s).

The tool will output a unit test for every receiver/selector/namespace combination. These tests will fail if there are any errors loading the configs in this scenario.

# Why a new config tool?

Many services at twitch have highly redundant configurations in json/toml/yaml, for example:

# Question
Have you ever had configs in a directory that looked like this?

### development.json
```
{
    "StatsURL": "statsd.internal.justin.tv:8125",
    "StatsPrefix": "hats.dev",
    "StatsNoop": true,

    "SandstormRoleArn": "arn:aws:iam::734326455073:role/sandstorm/production/templated/role/hats-dev",
    "SandstormEnvironment": "development",
    
    "SiteDBMaxOpenConnections": 5,
    "SiteDBStatsPrefix": "hats.dev.sitedb",
    "SiteDBHost": "replica-sitedb.staging.us-west2.justin.tv",
}
```

### staging.json

```
{
    "StatsURL": "statsd.internal.justin.tv:8125",
    "StatsPrefix": "hats.staging",
    "StatsNoop": false,

    "SandstormRoleArn": "arn:aws:iam::734326455073:role/sandstorm/production/templated/role/hats-staging",
    "SandstormEnvironment": "staging",

    "SiteDBMaxOpenConnections": 5,
    "SiteDBStatsPrefix": "hats.staging.sitedb",
    "SiteDBHost": "replica-sitedb.staging.us-west2.justin.tv",
}
```

### production.json
```
{
    "StatsURL": "statsd.internal.justin.tv:8125",
    "StatsPrefix": "hats.prod",
    "StatsNoop": false,

    "SandstormRoleArn": "arn:aws:iam::734326455073:role/sandstorm/production/templated/role/hats-prod",
    "SandstormEnvironment": "production",

    "SiteDBMaxOpenConnections": 10,
    "SiteDBStatsPrefix": "hats.prod.sitedb",
    "SiteDBHost": "sitedb-replica-internal.prod.us-west2.justin.tv",
}
```

### all.c7

```
*.*:
    HATS:AWSRegion = "us-west-2";
    
*.development:
    Sandstorm:RoleArn = "arn:aws:iam::734326455073:role/sandstorm/production/templated/role/hats-dev";
    Sandstorm:Environment = "development";

*.staging:
    Sandstorm:RoleArn = "arn:aws:iam::734326455073:role/sandstorm/production/templated/role/hats-staging";
    Sandstorm:Environment = "staging";

*.production:
    Sandstorm:RoleArn = "arn:aws:iam::734326455073:role/sandstorm/production/templated/role/hats-prod";
    Sandstorm:Environment = "production";
    
*.*:
    SiteDB:MaxOpenConnections = 5;
    SiteDB:StatsPrefix = "hats.dev.sitedb";
    SiteDB:Host = "replica-sitedb.staging.us-west2.justin.tv";

*.development:
    SiteDB:StatsPrefix = "hats.dev.sitedb";

*.staging:
    SiteDB:StatsPrefix = "hats.staging.sitedb";

*.production:
    SiteDB:MaxOpenConnections = 10;

    SiteDB:StatsPrefix = "hats.prod.sitedb";
    SiteDB:Host = "sitedb-replica-internal.prod.us-west2.justin.tv";
    
*.*:
    Statsd:URL = "statsd.internal.justin.tv:8125";
    Statsd:Prefix = "hats.dev";

*.staging:
    Statsd:Prefix = "hats.staging";

*.production:
    Statsd:Prefix = "hats.prod";
```

# How does it work?

C7 uses a ANTLRv4 based parser with the schema defined in [C7.g4]. This is important in any language as it allows you to more easily build tooling and editors.
