GitHub Integration

Prerequisite: You must have a GitHub account in order to successfully complete the following exercises.

In this section, we will see how to leverage QDS Notebooks integration with GitHub to save your Spark application and the dashboard for future references, version control and collaboration purposes. This integration between QDS and GitHub is also crucial for iterative development of complex ETL workflows and Machine Learning applications.

Note: Committing to GitHub will save your queries, code, commands and results to your repository and everything that is saved, can be restored at a later date even if the associated cluster is not running.

Create New GitHub Repository And Token

  1. Create new Public or Private repo in your account
  2. Create new personal access token by browsing to https://github.com/settings/tokens/new
    • Check ‘repo’ checkbox
    • Click Generate token
    • COPY generated token to clipboard

Configure GitHub Token in QDS

  1. Switch over to Control Panel
  2. Click on My Accounts
  3. Under GitHub Token column, click on Configure
  4. In the popup, PASTE token generated in previous exercise

Link Spark Notebook with GitHub Profile

  1. Switch over to Notebooks
  2. Under My Home → Notebooks, click on your Notebook
  3. On the top right corner, click on Versions
  4. Click on Link now
  5. Set Branch to master
  6. Set Object Path to Quick Start-data (for example)
  7. Click Save

Commit Notebook to GitHub

  1. Switch over to Notebooks
  2. Under My Home → Notebooks, click on your Notebook
  3. On the top right corner, click on Versions
  4. Click on Push
  5. Enter ‘First commit’ (it’s the “Hello World” of commit messages!) for Comment
  6. Click Save

To confirm your code was committed successfully to your GitHub account, switch over to the new repo you created in previous exercise and see if a (JSON) file named Quick Start-data exists.

To restore this version, switch over to Notebooks:

  1. Under My Home → Notebooks, click on your Notebook
  2. On the top right corner, click on Versions
  3. Click on Restore