## How to contribute
The [Spade Schema Registry](https://wiki.xarth.tv/display/DI/Spade+Schema+Registry) (SSR) is presently an alternative to
Blueprint for defining and managing Spade event schemas. Therefore, schema definitions represent the primary type of 
contributions to this repository. 

SSR is just starting out and it is likely you will discover things that can be improved upon. Like all software projects,
this repository will also need refactoring and improvements with time. If you have suggestions or ideas, please 
contribute them for discussion in [#spade-schema-registry](https://twitch.slack.com/archives/C01GRJVPS02).

## Getting permissions to contribute to this repository
To contribute to this repository, you'll need to be a member of the 
[Spade GitHub organization](https://git.xarth.tv/spade/). If you already have permissions to create and edit events in 
[Blueprint](https://blueprint.di.xarth.tv/), you will have been automatically added to the Spade org.

If you are not yet a member of this org and wish to contribute, please follow the directions in the 
[Event Owner's Manual](https://wiki.xarth.tv/display/DI/Event+Owner%27s+Manual) to get access.

## Joining a team
Objects defined in SSR require an owning team to be assigned. Please reference our 
[published guidelines](https://wiki.xarth.tv/display/DI/Event+Ownership) on event ownership to learn more about this 
requirement.

If your [team already exists in the Spade org](https://git.xarth.tv/orgs/spade/teams), you will need to coordinate with
the team’s Maintainer to be added to the team. If your team does not yet exist, please reach out in 
[#spade-schema-registry](https://twitch.slack.com/archives/C01GRJVPS02) to request the team be created following the 
above guidelines for event-owning teams.

## Protecting sensitive data
The Data Platform team is responsible for ensuring
[privacy compliance for the Tahoe data lake](https://wiki.xarth.tv/display/DI/Tahoe+GDPR+Initiative). This requires 
sensitive data to be appropriately classified and labeled to safeguard users’ personal data and to protect Twitch from 
legal risk. It is extremely important that the sensitivity labels on fields are correct. Incorrect classification will 
prevent the data from being usable in Tahoe and, in some cases, could lead to its removal from the data lake.

Before contributing or making any changes in this repository, please review the following resources. If you are still 
unsure of the correct sensitivity, please ask in [#data-infrastructure](https://twitch.slack.com/archives/C03C62S07).
- [Data Sensitivity Overview](https://wiki.xarth.tv/display/DI/Data+Sensitivity)
- [Sensitive Field Examples](https://wiki.xarth.tv/display/DI/Sensitive+Field+Examples)
- [Field _channel_id_ example in SSR](https://git.xarth.tv/spade/events/blob/dd9b1dfe9c36feffcf33c9e73df90e6dc5ec2ad0/fields/channel_id.yaml)

## Setting up the repository
The file structure and contents of the repository serve as the primary user interface for the Spade Schema Registry.
We recommend reviewing the [wiki documentation](https://wiki.xarth.tv/display/DI/Spade+Schema+Registry) and exploring 
the repository to familiarize yourself with its structure and key components before contributing. You should also 
confirm that you meet all the [prerequisites](https://wiki.xarth.tv/display/DI/SSR+Prerequisites) before getting started.

There are presently two versions of this repository:

- [events_playground](https://git.xarth.tv/spade/events_playground) is the playground repository that allows you to
  familiarize yourself with SSR and its workflow before making changes that impact production events.
- [events](https://git.xarth.tv/spade/events) is the repository where production event definitions live. These schema
  definitions are the source of truth for what is ingested into the Spade pipeline. Once an event is defined here, it
  will become read-only in Blueprint and all future updates to it will need to be made in SSR.
  
When you’re ready to contribute to this repository (e.g. to add new events or make changes to existing ones), follow the
[SSR Setup Steps](https://wiki.xarth.tv/display/DI/SSR+Setup+Steps). Please note, you will need to replace
_events_playground_ with _events_ in these steps if your changes are intended for production event definitions.

## Making your changes
Making changes in SSR follows a standard Git development flow. Once you’ve cloned and set up the repository, you’ll open
a branch, make your changes, test them locally (with the provided scripts) and commit those changes once all tests are 
passing.

If this is your first time working with SSR, we recommend you complete the tutorials for 
[adding a new event](https://wiki.xarth.tv/display/DI/SSR+Tutorial+1%3A+Hello+World) or 
[importing an existing event](https://wiki.xarth.tv/display/DI/SSR+Tutorial+2%3A+Importing+an+Event). These tutorials 
will allow you to become familiar with the SSR workflow in the 
[playground repository](https://git.xarth.tv/spade/events_playground) mentioned above.

If you’re new to Git-based development and not sure where to start, we recommend following the published guide on 
[setting up and using Atom](https://wiki.xarth.tv/display/DI/Setting+up+and+using+Atom). This guide provides an interface
recommendation (GitHub’s Atom editor) and corresponding instructions for those who prefer to work in a GUI development 
environment over the CLI.

## Owning your changes
SSR leverages GitHub’s 
[code owners feature](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-on-github/about-code-owners)
for establishing ownership of any new event, field, and/or group added to the repository. Ownership in SSR is assigned 
to an [existing team in the Spade org](https://git.xarth.tv/orgs/spade/teams) via the
[CODEOWNERS file](https://git.xarth.tv/spade/events/blob/dd9b1dfe9c36feffcf33c9e73df90e6dc5ec2ad0/.github/CODEOWNERS).

For imported objects, ownership will be automatically assigned by the import script at the time of import.

For new objects, you will need to establish ownership prior to committing your changes. A
[script](https://git.xarth.tv/spade/events/blob/dd9b1dfe9c36feffcf33c9e73df90e6dc5ec2ad0/scripts/update_codeowners.sh)
has been provided to help with this process. When your changes are complete and the only remaining test failures are
related to test_codeowners, run the following (with the virtual environment activated):

`./scripts/update_codeowners.sh @spade/YOUR_TEAM_HERE`

`(e.g.   ./scripts/update_codeowners.sh @spade/data-platform)`

This will add the appropriate entries to the CODEOWNERS file for the fields/groups/events you are currently working on.
You can see examples of what this looks like [here](https://git.xarth.tv/spade/events/blob/mainline/.github/CODEOWNERS#L25).

## Submitting a pull request (PR)
As you commit your changes, SSR will check them for issues. Once you submit a PR, a build will start that will test your
changes for logical consistency, compliance with established conventions, and alignment with what is defined in Blueprint.
The guardrails that will result in a build failure and block the merge to the default branch are:

- Any modification to the field type (e.g., changing varchar type to float) that results in a change in the field type
  for an existing event
- Any modification to the sensitivity labeling of a field that results in a change in the field sensitivity for an 
  existing event
- Any modification to a field which results in a change in the inbound field name for an existing event
- Any modification that changes more than two event schemas which are older than 30 days (due to Tahoe Classic limitations)
- Any change made through this repository between 4 A.M - 8 A.M UTC (due to Tahoe Classic limitations)

Guardrails related to Tahoe Classic limitations will be removed once we complete our Tahoe Clean initiative and all our
data in Tahoe is GDPR compliant.

The migration of an existing event from Blueprint includes additional checks for certain functionality that will no longer
be supported in SSR. The list of deprecated items include the usage of the following field names, types and transforms:
- Any field named as asn
- Any field with type as int (use long instead)
- Any transform of f@timestamp@unix-utc or f@timestamp@unix (use timestamp instead)
- Any field with transforms of userIdWithMapping and ipAsn

If you wish to migrate an event to SSR that currently leverages any of the above, you will need to work with the Data
Platform team via a [DISD ticket](https://support.di.xarth.tv/) to resolve these issues before the event can be defined in SSR.

## Requesting a review for your PR
When your PR is ready for review, you will need to add the _Ready for Review_ label to it. This will trigger an
automatic review request that will notify members of the team via GitHub’s usual notification mechanisms, in addition to
sending a notification to the owning team’s specified Slack channel (pending its subscription to Orko notifications has
been enabled). Once the changes in your PR have been reviewed and approved by the owning team, you will be able to merge
your changes to the default branch (_mainline_).

The owning team information for each object can be found in the
[CODEOWNERS](https://git.xarth.tv/spade/events/blob/dd9b1dfe9c36feffcf33c9e73df90e6dc5ec2ad0/.github/CODEOWNERS) file.
Ownership will also be [reflected in Blueprint](https://wiki.xarth.tv/display/DI/Event+Owner%27s+Manual#Blueprint_Event_Ownership)
for events that are defined in SSR.

## Reviewing a PR
Code covered by [CODEOWNERS](https://git.xarth.tv/spade/events/blob/dd9b1dfe9c36feffcf33c9e73df90e6dc5ec2ad0/.github/CODEOWNERS)
will need a single reviewer by **each team owning a file** modified (not referenced) in the PR. Peer review is the first
line of defense against preventable data quality issues entering Twitch’s data ecosystem. Reviewers should make a best
effort to ensure the changes align with 
[best practices](https://wiki.xarth.tv/pages/viewpage.action?spaceKey=DI&title=Event+Owner%27s+Manual) and add
maintainable value to the data ecosystem.

## Releasing your changes
After a PR is merged to _mainline_, it is [validated](https://jenkins-og.xarth.tv/job/spade-schema-registry-validate/)
and [published](https://jenkins-og.xarth.tv/job/spade-schema-registry-publish/) by Jenkins. Your changes will come to
rest with the event newly-defined in SSR reflected as read-only in Blueprint. All future changes to that event will now
be required to be made via SSR.

It is possible the build process takes a little while to complete and could fail for a number of reasons. If you notice
your changes have failed, you will need to engage Data Platform to resolve the issue. To engage on-call support for the
resolution, please create a [DISD ticket](https://support.di.xarth.tv/).

## Getting help
If you find yourself stuck or in need of assistance at any point, come talk with us in
[#spade-schema-registry](https://twitch.slack.com/archives/C01GRJVPS02)!
