# TwitchFeatureStoreClient
This is the implementation of the Golang client library as proposed in this [tech spec](https://docs.google.com/document/d/1Q_B_Qyttm7h2X_7XgdavULvauSmPDEVd6FZZRh2y75c/edit). It is a component of the [MLFS project](https://wiki.twitch.com/display/CP/Machine+Learning+Feature+Store+Project). 

This package is being built into `Twitch/live` version set through this [pipeline](https://pipelines.amazon.com/pipelines/TwitchMLFSVerification).

when making changes, make sure the [integration tests](https://code.amazon.com/packages/TwitchMLFSVerification/trees/mainline/--/cmd/TwitchMLFSVerification) in the TwitchMLFSVerification package won't fail.

## Feature Consumer Setup
If your service is a fulton service, you will need to add `TwitchFeatureStoreClient = 1.0;` to your `Config` file. If your service is not fulton, use this mirrored repo in GHE: https://git.xarth.tv/amzn/TwitchFeatureStoreClient

### Code Example
Firstly, you need to set up the `client`.

```Golang
	client := fsc.Client{
		Region:      "us-west-2",
		Environment: fsc.Beta,
		Config:      awsSession,
		Observer:    sampleReporter,
		ServiceName: "TwitchMLFSVerification",
		Logger:      logger,
		HttpClient:  httpClient,

		CircuitManager: circuitManager,
		CircuitConfig: circuit.Config{
			Execution: circuit.ExecutionConfig{
				Timeout:               200 * time.Millisecond,
				MaxConcurrentRequests: 1000,
			},
		},

		CacheConfig: cache.Config{
			CacheSize: 100000000,
		},
	}
```

Then you will have to call the `InitializeFeatures` function of the `client`
```Golang
	err = client.InitializeFeatures(context.Background(), 
    fsc.FeatureParams{
		FeatureKey: fsct.FeatureKey{
			FeatureID: "feature_0",
			Namespace: "test",
			Version:   0,
		},
		CacheFeature: true,
		CacheTTL: 1 * time.Hour,
		FeatureJitter: 20 * time.Minute,
	},
	fsc.FeatureParams{
		FeatureKey: fsct.FeatureKey{
			FeatureID: "feature_1",
			Namespace: "test",
			Version:   0,
		},
	},
    fsc.FeatureParams{
		FeatureKey: fsct.FeatureKey{
			FeatureID: "feature_2",
			Namespace: "test",
			Version:   0,
		}
	})
```
The `InitializeFeatures` should be called only once at service startup. It fetches the required feature metadata from Feature Registry. If the returned `err` is not nil, you should treat it as a critical issue and stop your service. `Initialize` will initialize the feature without caching any of them. It will be deprecated in the future.

Regarding how to use it to fetch feature data, refer to the `FeatureSourceBuilder` interface in this client. There is also an exmple [here](https://code.amazon.com/packages/TwitchMLFSVerification/blobs/46067dcf68d57d64a10293ee6e04450025ae6343/--/server/server.go#L215-L255).


### Some Important Notes
- `FeatureSourceBuilder` needs to be re-created for each request. 
- It's highly recommended to pass in a customized http client to FSC, instead of using the default one. You can refer to this [guide](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/custom-http.html) for configurations. E.g. setting `MaxIdleConnsPerHost` to a big number (~300) can help reduce CPU time spent in TLS handshakes to DDB.
- The `CircuitConfig` will be applied to all calls to OFS (features can live in multiple OFS). 
- Cache is not enabled by default. You need to have at least 1 feature that have `CacheFeature` set to `true` to enable caching. The feature value of each feature instance will be cached. Callers need to enable caching for each feature. The cache size is 4096 by default but it is recommended to have a larger cache size. `CacheTTL` specify the TTL of the feature in the Cache. `0` value means that the feature will not expire from the cache. The jitter has a default value of 10 minutes but with larger TTL, this value should be large to avoid a burst to the feature source.


## Developer Setup

### General setup
If you have not used Fulton or Amazon Builder tools before, you can follow this [guide](https://docs.fulton.twitch.a2z.com/docs/getting_started/setting_up_dev_env.html) to set up your local dev environment. 
If you want to learn more about what each tool does, the [fulton docs](https://docs.fulton.twitch.a2z.com/docs/getting_started/tech_stack.html) and [Amazon Builder tools](https://builderhub.corp.amazon.com/tools/) are very helpful.

### Checkout this repo
- Authenticate with `kinit` and `mwinit`.
- You would need to create a workspace:
`brazil ws create --root ~/workspaces/<project-name>"`.
- Then, you can cd into the workspace and checkout this package: `brazil ws use -p TwitchFeatureStoreClient`.
It is recommended to checkout `TwitchMLFSVerification` as well since it contains integration tests for FSC and OFS. 
- Before building the package locally, you need to run `brazil setup platform-support` inside the workspace. You only need to run this once.
- To build the project locally, cd into `~/workspaces/<project-name>/TwitchFeatureStoreClient/src/` and run `brazil-build`
- It is recommended to checkout this repo with `TwitchMLFSVerification` since all FSC integration tests are part of it.
```
brazil ws create --name TwitchFeatureStoreClient
cd TwitchFeatureStoreClient
brazil ws use --vs TwitchMLFSVerification/development -p TwitchFeatureStoreClient -p TwitchMLFSVerification
```

### Other notes
- Every change needs to get reviewed through the [CR](https://docs.fulton.twitch.a2z.com/docs/code_reviews.html) tool.
- Steps to add a dependency can be found [here](https://docs.fulton.twitch.a2z.com/docs/adding_dependencies.html).