lambdacd

0.1.0-alpha14-SNAPSHOT


a library to create a continous delivery pipeline in code

dependencies

org.clojure/clojure
1.6.0
org.clojure/data.json
0.2.5
me.raynes/conch
0.8.0
org.clojure/core.async
0.1.338.0-5c5012-alpha
compojure
1.1.8
org.clojure/tools.logging
0.3.0
org.slf4j/slf4j-api
1.7.5
ch.qos.logback/logback-core
1.0.13
ch.qos.logback/logback-classic
1.0.13
ring-server
0.3.1
ring/ring-json
0.3.1
hiccup
1.0.5
bidi
1.18.7
cljsjs/react
0.12.2-5
cljs-ajax
0.3.10
reagent
0.5.0-alpha3
reagent-utils
0.1.2
prone
0.8.0
selmer
0.8.0
environ
1.0.0
org.clojure/clojurescript
0.0-2850



(this space intentionally left almost blank)
 

The pipeline definiton.

This namespace contains the definition of the continuous delivery pipeline, i.e. the order and structure in which build steps are executed. The steps themselves are specified in their own namespace to keep those two things seperate.

Also in this namespace ring-handler and startup functions that form the entrypoints into the system.

(ns todopipeline.pipeline
  (:require [lambdacd.core :as core]
            [lambdacd.util :as utils]
            [ring.server.standalone :as ring-server]
            [clojure.tools.logging :as log])
  (:use [lambdacd.steps.control-flow]
        [todopipeline.steps]))

the definition of the pipeline as a list of steps that are executed in order.

(def pipeline-def
  `(
    ;; the first step is usually a step that waits for some event to occur, e.g.
    ;; a manual trigger or some change in the repo
    ;; the `either` control-flow element allows us to assemble a new trigger out of the two existing ones:
    ;; wait for either a change in the repository or the manual trigger.
     (either
       lambdacd.steps.manualtrigger/wait-for-manual-trigger
       wait-for-greeting
       wait-for-frontend-repo
       wait-for-backend-repo)
     ;; you could also wait for a repository to change. to try, point the step to a repo you control,
    ;; uncomment this, run and see the magic happen (the first build will immediately run since there is no known state)
    ; wait-for-frontend-repo
    ;; this step executes his child-steps (the arguments after the in-parallel) in parallel and waits
    ;; until all of them are done. if one of them fails, the whole step fails.
    (in-parallel
      ;; these child steps do some actual work with the checked out git repo
      (with-frontend-git
        client-package)
      (with-backend-git
        server-test
        server-package))
    ;; the package-scripts copy deploy-scripts and artifacts into the mockrepo directory,
    ;; execute the depoy-steps from there
    (in-cwd "/tmp/mockrepo"
      client-deploy-ci
      server-deploy-ci)
    ;; now we want the build to fail, just to show it's working.
    some-failing-step
    some-step-that-cant-be-reached))
(defn -main [& args]
  (let [home-dir (utils/create-temp-dir)
        ;; # The Configuration.
        ;; This is where you define the run-time configuration of LambdaCD. This configuration is passed on to the
        ;; individual build-steps in the `:config`-value of the context and will be used by the infrastructure and
        ;; build-steps. :home-dir is the directory where LambdaCD will store all it's internal data that should be persisted
        ;; over time, such as the last seen revisions of various git-repositories, the build history and so on.
        config { :home-dir home-dir :dont-wait-for-completion true}
        ;; wiring everything together everything that's necessary for lambdacd to run
        pipeline (core/mk-pipeline pipeline-def config)
        ;; the ring handler
        app (:ring-handler pipeline)
        ;; and the function that starts a thread that runs the actual pipeline.
        start-pipeline-thread (:init pipeline)]
    (log/info "LambdaCD Home Directory is " home-dir)
    (start-pipeline-thread)
    (ring-server/serve app {:open-browser? false
                            :port 8080})))
 

Build steps

This namespace contains the actual buildsteps that test, package and deploy your application. They are used in the pipeline-definition you saw in todopipeline.pipeline

(ns todopipeline.steps
  (:require [lambdacd.steps.shell :as shell]
            [lambdacd.internal.execution :as execution]
            [lambdacd.steps.git :as git]
            [lambdacd.steps.manualtrigger :as manualtrigger]
            [lambdacd.util :as util]))

Let's define some constants

(def backend-repo "git@github.com:flosell/todo-backend-compojure.git")
(def frontend-repo "git@github.com:flosell/todo-backend-client.git")

This step does nothing more than to delegate to a library-function. It's a function that just waits until something changes in the repo. Once done, it returns and the build can go on

(defn wait-for-frontend-repo [_ ctx]
  (let [wait-result (git/wait-for-git ctx frontend-repo "master")
        frontend-revision (:revision wait-result)]
    {:frontend-revision frontend-revision
     :backend-revision "HEAD"
     :status :success}))
(defn wait-for-backend-repo [_ ctx]
  (let [wait-result (git/wait-for-git ctx backend-repo "master")
        backend-revision (:revision wait-result)]
    {:backend-revision backend-revision
     :frontend-revision "HEAD"
     :status :success}))

Define some nice syntactic sugar that lets us run arbitrary build-steps with a repository checked out. The steps get executed with the folder where the repo is checked out as :cwd argument. The ^{:display-type :container} is a hint for the UI to display the child-steps as well.

(defn ^{:display-type :container} with-frontend-git [& steps]
  (fn [args ctx]
    (git/checkout-and-execute frontend-repo (:frontend-revision args) args ctx steps)))
(defn ^{:display-type :container} with-backend-git [& steps]
  (fn [args ctx]
    (git/checkout-and-execute backend-repo (:backend-revision args) args ctx steps)))
(defn wait-for-greeting [args ctx]
  (manualtrigger/parameterized-trigger {:greeting { :desc "some greeting"}} ctx))

The steps that do the real work testing, packaging, publishing our code. They get the :cwd argument from the with-*-git steps we defined above.

(defn client-package [{cwd :cwd greeting :greeting} ctx]
  (shell/bash ctx cwd
    (str "echo \"This is an optional greeting: " greeting "\)
    "bower install"
    "./package.sh"
    "./publish.sh"))
(defn server-test [{cwd :cwd} ctx]
  (println "server test cwd: " cwd)
  (shell/bash ctx cwd
    "lein test"))
(defn server-package [{cwd :cwd} ctx]
  (println "server package cwd: " cwd)
  (shell/bash ctx cwd
    "lein uberjar"
    "./publish.sh"))
(defn server-deploy-ci [{cwd :cwd} ctx]
  (shell/bash ctx cwd "./deploy-server.sh backend_ci /tmp/mockrepo/server-snapshot.tar.gz"))
(defn client-deploy-ci [{cwd :cwd} ctx]
  (shell/bash ctx cwd "./deploy-frontend.sh frontend_ci /tmp/mockrepo/client-snapshot.tar.gz"))

This is just a step that shows you what output steps actually have (since you have only used library functions up to here). It's just a map with some information. :status has a special meaning in the sense that it needs to be there and be :success for the step to be treated as successful and for the build to continue. all other data can be (more or less) arbitrary and will be passed on to the next build-step as input arguments. more or less means that some values have special meaning for the UI to display things. Check out the implementation of shell/bash if you want to know more.

(defn some-step-that-cant-be-reached [& _]
  { :some-info "hello world"
    :status :success})

Another step that just fails using bash. We could have made a failing step easier as well by just returning { :status :failure }

(defn some-failing-step [_ ctx]
  (shell/bash ctx "/" "echo \"i am going to fail now...\ "exit 1"))