# TwitchFeatureContext

This package contains the shared proto3 schema for FeatureContext, a message type for passing machine learning features between services (Ex. TwitchRecs is a consumer of this type), as well as library code for helping interact with those messages.

## Security Advisory

The FeatureContext message type may contain arbitrary feature data including user data. Treat it as such and avoid unnecessary logging / follow the appropriate controls.


## Memory Footprint

Because FeatureContext may contain arbitrary data, we also need to decide an arbitrary maximum memory size to adhere to. A single FeatureContext message should never exceed 4MB (a common document size limit for JSON parsers). In practice, it will be much much smaller.

Ex. An extreme case of 1000 flat features with 512 character keys and 1000 byte values will be `~1.5MB`.


## MLFSContext: Structuring a Struct

The primary message type contained in FeatureContext is MLFSContext, containing a map of string feature keys to arbitrary values.

If you are creating MLFSContext from scratch, consider using `structpb`'s built in function `structpb.NewStruct(map[string]interface{})`:

Read about how this handles specific types such as lists here: https://pkg.go.dev/google.golang.org/protobuf/types/known/structpb#NewValue

Ex:
```
// Build map
m := map[string]interface{}{
    "feature1": "value1",
    "age":      23,
}

m, err = structpb.NewStruct(m)
if err != nil {
    ...
}
m, err = util.MarshalMLFSContext(mlfs_context)
if err != nil {
    ...
}

// Set FeatureContext and go make some server calls
twitchrecs_request.FeatureContext = m
...
```

## Adding a New Message Type

You should consult with the recommendations product team if you plan on adding a new message. You should almost never have to do this!

To add a new message type to FeatureContext, add a proto schema for that message type under `schema/<message>.proto` and generate the go files as such:
```
protoc --go_out=schema schema/<message>.proto
```

For users of this library to be able to unmarshal a FeatureContext message into your new message type, it must be a part of the global type registry (aka. it should be referenced at least once in code, such as in a util helper function).


## Adding FeatureContext To Your Service's RPC

If you are using `dep` to install dependencies, you will need to make sure you are not pruning non-go files or else you won't have the `.proto` files available. Include the following in your `Gopkg.toml` if needed:

```
[[prune]]
  # regular prune rules #
  non-go = true

  # Disable prune for FeatureContext to pull .proto files
  [[prune.project]]
  name = "code.justin.tv/amzn/TwitchFeatureContext"
  non-go = false
```

Import the FeatureContext message type in your `.proto` file and add it to your relevant message type:

```
import "TwitchFeatureContext/schema/featurecontext.proto";

...

message MyServiceRequest {
    string game_id = 1;
    string request_tracking_id = 2;
    twitch.vx.featurecontext.FeatureContext feature_context = 3;
}
```

When generating go files, you will need to pass the path prefix of your import to `protoc` via the `--proto_path / -I` command. Ex:

```
protoc --go_out=schema -I=../vendor/code.justin.tv/amzn/ -I=. myservice.proto
```

If you are using `go generate`, you can include this in your `gen.go` file like so:

```
//go:generate protoc --twirp_out=. --go_out=. -I=../vendor/code.justin.tv/amzn -I=. service.proto
package mypackage
```

