# Twitch-Video-ServiceLog

Generate Cloudwatch logs and metrics based on struct values, to allow attaching these to units-of-work. Units-of-work are commonly things such as HTTP requests or jobs read from a queue, which are extremely useful to have associated log lines in order to diagnose internal app behavior.

## How to use

### Define Cloudwatch Log Group

Twitch-Video-ServiceLog sends logs to a log group provided by you. Use your preferred infrastructure-as-code tool (Terraform, CDK) to define this group and grant your application the following IAM permissions.

```
logs:CreateLogStream
logs:DescribeLogStreams
logs:PutLogEvents
```

### Import package

For Brazil apps, add `Twitch-Video-ServiceLog = 1.0;` to your dependencies in the `Config` file.

```go
import (
  servicelog "code.justin.tv/amzn/Twitch-Video-ServiceLog"
)
```

### Define your unit-of-work struct

Embed the `*servicelog.LogEntry` type into your struct to enable Cloudwatch metric creation.

```go
type MyLogEntry struct {
  *servicelog.LogEntry

  // any exported fields you define will be logged to your Cloudwatch log group
  MyField string
  // exported fields tagged with "servicelog" will be created as metrics, using the given Unit
  MyCounter int `servicelog:"Count"`
  // exported fields with "omitempty" option will not be counted as a metric if it contains the empty value
  MyDuration int64 `servicelog:"Milliseconds,omitempty"`

  // standard json tags work as expected, allowing per-request data to be referred to without being logged
  StartTime `json:"-"`
}
```

### Instantiate logger during application boot

Both production and local loggers are provided. For the production logger, a common pattern is to use the hostname of your instance as the log stream.

Logs are batched and sent to Cloudwatch in the background.

```go
import (
  "os"

  "github.com/aws/aws-sdk-go/aws"
  "github.com/aws/aws-sdk-go/aws/endpoints"
  "github.com/aws/aws-sdk-go/aws/session"

  servicelog "code.justin.tv/amzn/Twitch-Video-ServiceLog"
  identifier "code.justin.tv/amzn/TwitchProcessIdentifier"
)

func main() {
  sess := session.Must(session.NewSession(&aws.Config{
    Region:              aws.String(os.Getenv("AWS_REGION")),
    STSRegionalEndpoint: endpoints.RegionalSTSEndpoint,
  }))

  logGroup := os.Getenv("SERVICELOG_GROUP")

  // TwitchTelemetry users can get this from SampleReporter.SampleBuilder.ProcessIdentifier
  pid := identifier.ProcessIdentifier{}

  var serviceLogger servicelog.Logger
  if os.Getenv("DEBUG") == "1" {
    serviceLogger = servicelog.NewLocalLogger(pid)
  } else {
    logStream, err := os.Hostname()
    if err != nil {
      logStream = "unknown"
    }
    serviceLogger = servicelog.New(sess, logGroup, logStream, pid)
  }

  go serviceLogger.Run(context.Background())
}
```

### Set fields during your unit-of-work

For each unit-of-work, instantiate a new instance of your logging type (with embedded `*LogEntry`), set the values you want to log and then use the logger to send it to Cloudwatch.

```go
l := &MyLogEntry{
  // optionally set Operation name
  LogEntry: serviceLogger.NewLogEntry(""),
}

l.StartTime = time.Now()
l.MyField = "deadbeef"
l.MyCounter = 1
l.MyDuration = time.Since(l.StartTime).Milliseconds()

err := serviceLogger.Send(l)
```

Your metrics will be immediately available under the namespace defined from your `ProcessIdentifier.Service` setting, and your logs will be attached to your provided log group in the following format.

```json
{
  "_aws": {
    "Timestamp": 1614775550445,
    "CloudWatchMetrics": [
      {
        "Namespace": "MyServiceName",
        "Dimensions": [
          ["Region", "Service", "Stage"],
          ["Region", "Service", "Stage", "Substage"]
        ],
        "Metrics": [
          {
            "Name": "MyCounter",
            "Unit": "Count"
          },
          {
            "Name": "MyDuration",
            "Unit": "Milliseconds"
          }
        ]
      }
    ]
  },
  "Region": "us-west-2",
  "Service": "MyServiceName",
  "Stage": "beta",
  "Substage": "primary",
  "MyField": "deadbeef",
  "MyCounter": 1,
  "MyDuration": 50
}
```

### Context wrapper

To easily make your struct available throughout your code, consider adding a pointer to your struct into a `context.Context` which is already being passed around.

```go
l := &MyLogEntry{
  LogEntry: serviceLogger.NewLogEntry(""),
}

// add a pointer to your struct to the context at the beginning of your operation
ctx = servicelog.Context(ctx, l)

// pull it out of the context whenever you need to set a field
l = servicelog.FromContext(ctx).(*MyLogEntry)
l.MyCounter = 1
```

### Twirp hooks

Twirp hooks are provided to automatically create and send logs for each request. These hooks setup the context wrapper strategy for you.

To use the hooks, you need to provide three functions:

1. Initialize your struct with initial values
1. Finalize your struct, such as setting duration fields from an existing start time
1. Mark errors in the case that a Twirp error response is returned

```go
func main() {
  // register twirp middleware during app boot
  twirpMiddleware := twirp.ChainHooks(
    servicelog.ServerHooks(serviceLogger, createCallback, finalizeCallback, errorCallback),
  )
  handler := myservice.NewMyTwirpServiceServer(server, twirpMiddleware)
}

func createCallback(ctx context.Context, entry *servicelog.LogEntry) servicelog.MetricLogger {
  return &MyLogEntry{
    LogEntry:  entry,
    StartTime: time.Now(),
  }
}

func finalizeCallback(ctx context.Context) servicelog.MetricLogger {
  l = servicelog.FromContext(ctx).(*MyLogEntry)
  l.MyDuration = time.Since(l.StartTime).Milliseconds()
  return l
}

func errorCallback(ctx context.Context, err twirp.Error) context.Context {
  l = servicelog.FromContext(ctx).(*MyLogEntry)
  l.Counter = 1
  return ctx
}
```
