Utah Tech Podcast

This article is going to do a walkthrough of how to use Salt to configure Hashicorp Vault. It is going to also use Consul as a backend in this example.

This article assumes that you are familiar with the following:

Salt
Vault
Consul

We are going to do the following things here:

Setup Vault (“not in a dev capacity”)
Setup Consul systemd unit
Setup Vault systemd unit
Setup basic Vault policies
Setup Salt to talk to Vault

Assumptions I am going to use in this example:

CentOS Linux release 7.4
Vault Version: 0.8.3
Consul Version: 0.9.3

Lets get started…

So first I will create a path for the files: In my case it is:

/srv/salt/release-mgmt/states/services/

1	/srv/salt/release-mgmt/states/services/

In there, I create: consul-app and vault-app folders.

/srv/salt/release-mgmt/states/services/consul-app

1	/srv/salt/release-mgmt/states/services/consul-app

/srv/salt/release-mgmt/states/services/vault-app

1	/srv/salt/release-mgmt/states/services/vault-app

Vault Setup:

From there I setup an init.sls file with the following contents:

include:
  - services.vault-app.install

1 2	include: - services.vault-app.install

This is going to just be the starting place for the state file. I will be creating a install.sls file to perform the install.

/srv/salt/release-mgmt/states/services/vault-app/install.sls

1	/srv/salt/release-mgmt/states/services/vault-app/install.sls

{% import 'services/vault-app/vault-params.jinja' as params %}

vault:
  file.managed:
    - source: https://releases.hashicorp.com/vault/{{ salt['pillar.get']('vault_version') }}/vault_{{ salt['pillar.get']('vault_version') }}_linux_amd64.zip
    - name: /tmp/vault_{{ salt['pillar.get']('vault_version') }}_linux_amd64.zip
    - source_hash: https://releases.hashicorp.com/vault/{{ salt['pillar.get']('vault_version') }}/vault_{{ salt['pillar.get']('vault_version') }}_SHA256SUMS
    - template: jinja

  archive.extracted:
    - name: /usr/bin
    - source: /tmp/vault_{{ salt['pillar.get']('vault_version') }}_linux_amd64.zip
    - user: {{ params.vault_user }}
    - group: {{ params.vault_group }}
    - enforce_toplevel: False

  group.present:
    - name: {{ params.vault_group }}
    - addusers:
      - {{ params.vault_user }}
    - require:
      - user: {{ params.vault_user }}

  user.present:
    - name: {{ params.vault_user }}
    - shell: /bin/bash


set_user_mlock:
  cmd.run:
    - name: '{{ params.vault_user_setcap }} && touch {{ params.vault_config_base_path }}/mlock.ran'
    - unless: test -f  {{ params.vault_config_base_path }}/mlock.ran

/etc/sysconfig/vault:
  file.directory:
    - user: {{ params.vault_user }}
    - group: {{ params.vault_group }}
    - dir_mode: 755
    - file_mode: 644
    - recurse:
      - user
      - group
      - mode    

vault_configs:
  file.managed:
    - name: {{ params.vault_config_dest }}
    - source: {{ params.vault_config_source }}
    - group: {{ params.vault_user }}
    - user: {{ params.vault_group }}
    - require:
      - file: {{ params.vault_config_base_path }} 

vault_systemd_unit:
  file.managed:
    - name: {{ params.vault_service_file_dest }}
    - source: {{ params.vault_service_file_source }}
    - template: jinja
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: vault_systemd_unit

vault_running:
  service.running:
    - name: vault
    - watch:
      - module: vault_systemd_unit
    - require:
      - cmd: set_user_mlock

{% import 'services/vault-app/vault-params.jinja' as params %}

vault:

file.managed:

- source: https://releases.hashicorp.com/vault/{{ salt['pillar.get']('vault_version') }}/vault_{{ salt['pillar.get']('vault_version') }}_linux_amd64.zip

- name: /tmp/vault_{{ salt['pillar.get']('vault_version') }}_linux_amd64.zip

- source_hash: https://releases.hashicorp.com/vault/{{ salt['pillar.get']('vault_version') }}/vault_{{ salt['pillar.get']('vault_version') }}_SHA256SUMS

- template: jinja

archive.extracted:

- name: /usr/bin

- source: /tmp/vault_{{ salt['pillar.get']('vault_version') }}_linux_amd64.zip

- user: {{ params.vault_user }}

- group: {{ params.vault_group }}

- enforce_toplevel: False

group.present:

- name: {{ params.vault_group }}

- addusers:

- {{ params.vault_user }}

- require:

- user: {{ params.vault_user }}

user.present:

- name: {{ params.vault_user }}

- shell: /bin/bash

set_user_mlock:

cmd.run:

- name: '{{ params.vault_user_setcap }} && touch {{ params.vault_config_base_path }}/mlock.ran'

- unless: test -f {{ params.vault_config_base_path }}/mlock.ran

/etc/sysconfig/vault:

file.directory:

- user: {{ params.vault_user }}

- group: {{ params.vault_group }}

- dir_mode: 755

- file_mode: 644

- recurse:

- user

- group

- mode

vault_configs:

file.managed:

- name: {{ params.vault_config_dest }}

- source: {{ params.vault_config_source }}

- group: {{ params.vault_user }}

- user: {{ params.vault_group }}

- require:

- file: {{ params.vault_config_base_path }}

vault_systemd_unit:

file.managed:

- name: {{ params.vault_service_file_dest }}

- source: {{ params.vault_service_file_source }}

- template: jinja

module.run:

- name: service.systemctl_reload

- onchanges:

- file: vault_systemd_unit

vault_running:

service.running:

- name: vault

- watch:

- module: vault_systemd_unit

- require:

- cmd: set_user_mlock

We will need some vault policies, I have defined a generic on here as:

/srv/salt/release-mgmt/states/services/vault-app/vault_policies_install.sls

1	/srv/salt/release-mgmt/states/services/vault-app/vault_policies_install.sls

{% import 'services/vault-app/vault-params.jinja' as params %}
{% for policy_name, policy_file in salt['pillar.get']('vault_policies').items() %} 

vault_policies:
  file.managed:
    - name: {{ params.vault_policies_dest }}/{{ policy_file }}
    - source: {{ params.vault_policies_source }}/{{ policy_file }}
    - group: {{ params.vault_user }}
    - user: {{ params.vault_group }}
    - template: jinja
    - context:
        policy_file: {{ policy_file }}

  cmd.run:
    - name: vault policy-write {{ policy_name }} {{ params.vault_policies_dest }}/{{ policy_file }} 
    - env:
        VAULT_ADDR: 'http://127.0.0.1:8200'


{% endfor %}

{% import 'services/vault-app/vault-params.jinja' as params %}

{% for policy_name, policy_file in salt['pillar.get']('vault_policies').items() %}

vault_policies:

file.managed:

- name: {{ params.vault_policies_dest }}/{{ policy_file }}

- source: {{ params.vault_policies_source }}/{{ policy_file }}

- group: {{ params.vault_user }}

- user: {{ params.vault_group }}

- template: jinja

- context:

policy_file: {{ policy_file }}

cmd.run:

- name: vault policy-write {{ policy_name }} {{ params.vault_policies_dest }}/{{ policy_file }}

- env:

VAULT_ADDR: 'http://127.0.0.1:8200'

{% endfor %}

We will need the params file for all of the parameters that have been set:

/srv/salt/release-mgmt/states/services/vault-app/vault-params.jinja

1	/srv/salt/release-mgmt/states/services/vault-app/vault-params.jinja

{# vault parmas #}
{% set vault_user = 'vault' %}
{% set vault_group = 'vault' %}
{% set files_root_path = 'salt://services/vault-app/files' %}
{% set vault_config_base_path = '/etc/sysconfig/vault' %}
{% set vault_config_source = files_root_path + '/' + 'vault.hcl' %}
{% set vault_config_dest = vault_config_base_path + '/' + 'vault.hcl' %}
{% set vault_service_file_dest = '/etc/systemd/system/vault.service' %}
{% set vault_service_file_source = files_root_path + '/' + 'vault.service'%}
{% set vault_user_setcap = 'sudo setcap cap_ipc_lock=+ep $(readlink -f $(which vault))'%}



{# vault policy params #}
{% set vault_policies_dest = vault_config_base_path %}
{% set vault_policies_source = files_root_path %}

{# vault parmas #}

{% set vault_user = 'vault' %}

{% set vault_group = 'vault' %}

{% set files_root_path = 'salt://services/vault-app/files' %}

{% set vault_config_base_path = '/etc/sysconfig/vault' %}

{% set vault_config_source = files_root_path + '/' + 'vault.hcl' %}

{% set vault_config_dest = vault_config_base_path + '/' + 'vault.hcl' %}

{% set vault_service_file_dest = '/etc/systemd/system/vault.service' %}

{% set vault_service_file_source = files_root_path + '/' + 'vault.service'%}

{% set vault_user_setcap = 'sudo setcap cap_ipc_lock=+ep $(readlink -f $(which vault))'%}

{# vault policy params #}

{% set vault_policies_dest = vault_config_base_path %}

{% set vault_policies_source = files_root_path %}

And at last the pillar data…

Install this in your pillar for the env you are going to run it against(in my case “dev”).

vault_version: 0.8.3
vault_policies:
  dev-policy: dev-policy.hcl

vault_version: 0.8.3

vault_policies:

dev-policy: dev-policy.hcl

Now the last piece is all the files we are going to manage:

I create a directory called:

/srv/salt/release-mgmt/states/services/vault-app/files/

1	/srv/salt/release-mgmt/states/services/vault-app/files/

In the files directory I create the following files:

/srv/salt/release-mgmt/states/services/vault-app/files/dev-policy.hcl

1	/srv/salt/release-mgmt/states/services/vault-app/files/dev-policy.hcl

/srv/salt/release-mgmt/states/services/vault-app/files/vault.hcl

1	/srv/salt/release-mgmt/states/services/vault-app/files/vault.hcl

/srv/salt/release-mgmt/states/services/vault-app/files/vault.service

1	/srv/salt/release-mgmt/states/services/vault-app/files/vault.service

And the contents for each will be:

/srv/salt/release-mgmt/states/services/vault-app/files/dev-policy.hcl

1	/srv/salt/release-mgmt/states/services/vault-app/files/dev-policy.hcl

path "secret/*" {
capabilities = ["read", "list"]
}

path "auth/*" {
capabilities = ["read", "list","sudo","create","update","delete"]
}

path "secret/*" {

capabilities = ["read", "list"]

}

path "auth/*" {

capabilities = ["read", "list","sudo","create","update","delete"]

}

/srv/salt/release-mgmt/states/services/vault-app/files/vault.hcl

1	/srv/salt/release-mgmt/states/services/vault-app/files/vault.hcl

storage "consul" {
address = "127.0.0.1:8500"
path = "vault"
}

listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = 1
}

storage "consul" {

address = "127.0.0.1:8500"

path = "vault"

}

listener "tcp" {

address = "0.0.0.0:8200"

tls_disable = 1

}

/srv/salt/release-mgmt/states/services/vault-app/files/vault.service

1	/srv/salt/release-mgmt/states/services/vault-app/files/vault.service

{% import 'services/vault-app/vault-params.jinja' as params %}
[Unit]
Description=vault server
Requires=network-online.target
After=network-online.target

[Service]
User=vault
EnvironmentFile=-{{ params.vault_config_base_path }}
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/usr/bin/vault server -config={{ params.vault_config_dest }} 
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT

[Install]
WantedBy=multi-user.target

{% import 'services/vault-app/vault-params.jinja' as params %}

[Unit]

Description=vault server

Requires=network-online.target

After=network-online.target

[Service]

User=vault

EnvironmentFile=-{{ params.vault_config_base_path }}

Environment=GOMAXPROCS=2

Restart=on-failure

ExecStart=/usr/bin/vault server -config={{ params.vault_config_dest }}

ExecReload=/bin/kill -HUP $MAINPID

KillSignal=SIGINT

[Install]

WantedBy=multi-user.target

Consul Setup:

This is a basic consul setup to be able to use Consul as the Vault backend.

/srv/salt/release-mgmt/states/services/consul-app/consul-params.jinja

1	/srv/salt/release-mgmt/states/services/consul-app/consul-params.jinja

{# consul parmas #}
{% set consul_user = 'consul' %}
{% set consul_group = 'consul' %}
{% set files_root_path = 'salt://services/consul-app/files' %}
{% set consul_config_base_path = '/usr/local/consul' %}
{% set consul_service_file_dest = '/etc/systemd/system/consul.service' %}
{% set consul_service_file_source = files_root_path + '/' + 'consul.service'%}

{# consul parmas #}

{% set consul_user = 'consul' %}

{% set consul_group = 'consul' %}

{% set files_root_path = 'salt://services/consul-app/files' %}

{% set consul_config_base_path = '/usr/local/consul' %}

{% set consul_service_file_dest = '/etc/systemd/system/consul.service' %}

{% set consul_service_file_source = files_root_path + '/' + 'consul.service'%}

/srv/salt/release-mgmt/states/services/consul-app/install.sls

1	/srv/salt/release-mgmt/states/services/consul-app/install.sls

{% import 'services/consul-app/consul-params.jinja' as params %}

consul:
  file.managed:
    - source: https://releases.hashicorp.com/consul/{{ salt['pillar.get']('consul_version') }}/consul_{{ salt['pillar.get']('consul_version') }}_linux_amd64.zip
    - name: /tmp/consul_{{ salt['pillar.get']('consul_version') }}_linux_amd64.zip
    - source_hash: https://releases.hashicorp.com/consul/{{ salt['pillar.get']('consul_version') }}/consul_{{ salt['pillar.get']('consul_version') }}_SHA256SUMS


  archive.extracted:
    - name: /usr/bin
    - source: /tmp/consul_{{ salt['pillar.get']('consul_version') }}_linux_amd64.zip
    - user: {{ params.consul_user }}
    - group: {{ params.consul_group }}
    - enforce_toplevel: False

  group.present:
    - name: {{ params.consul_group }}
    - addusers: 
      - {{ params.consul_user }}
    - require: 
      - user: {{ params.consul_user }}

  user.present:
    - name: {{ params.consul_user }}
    - shell: /bin/bash

consul_systemd_unit:
  file.managed:
    - name: {{ params.consul_service_file_dest }}
    - source: {{ params.consul_service_file_source }}
    - template: jinja
  module.run:
    - name: service.systemctl_reload
    - onchanges:
      - file: consul_systemd_unit

consul_running:
  service.running:
    - name: consul
    - watch:
      - module: consul_systemd_unit

{% import 'services/consul-app/consul-params.jinja' as params %}

consul:

file.managed:

- source: https://releases.hashicorp.com/consul/{{ salt['pillar.get']('consul_version') }}/consul_{{ salt['pillar.get']('consul_version') }}_linux_amd64.zip

- name: /tmp/consul_{{ salt['pillar.get']('consul_version') }}_linux_amd64.zip

- source_hash: https://releases.hashicorp.com/consul/{{ salt['pillar.get']('consul_version') }}/consul_{{ salt['pillar.get']('consul_version') }}_SHA256SUMS

archive.extracted:

- name: /usr/bin

- source: /tmp/consul_{{ salt['pillar.get']('consul_version') }}_linux_amd64.zip

- user: {{ params.consul_user }}

- group: {{ params.consul_group }}

- enforce_toplevel: False

group.present:

- name: {{ params.consul_group }}

- addusers:

- {{ params.consul_user }}

- require:

- user: {{ params.consul_user }}

user.present:

- name: {{ params.consul_user }}

- shell: /bin/bash

consul_systemd_unit:

file.managed:

- name: {{ params.consul_service_file_dest }}

- source: {{ params.consul_service_file_source }}

- template: jinja

module.run:

- name: service.systemctl_reload

- onchanges:

- file: consul_systemd_unit

consul_running:

service.running:

- name: consul

- watch:

- module: consul_systemd_unit

/srv/salt/release-mgmt/states/services/consul-app/files/consul.service

1	/srv/salt/release-mgmt/states/services/consul-app/files/consul.service

[Unit]
Description=consul agent
Requires=network-online.target
After=network-online.target

[Unit]

Description=consul agent

Requires=network-online.target

After=network-online.target

[Service]
User=consul
EnvironmentFile=-/etc/sysconfig/consul
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/usr/bin/consul agent -server -bootstrap-expect 1 -data-dir /usr/local/consul -bind 127.0.0.1
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT

[Install]
WantedBy=multi-user.target

[Service]

User=consul

EnvironmentFile=-/etc/sysconfig/consul

Environment=GOMAXPROCS=2

Restart=on-failure

ExecStart=/usr/bin/consul agent -server -bootstrap-expect 1 -data-dir /usr/local/consul -bind 127.0.0.1

ExecReload=/bin/kill -HUP $MAINPID

KillSignal=SIGINT

[Install]

WantedBy=multi-user.target

With this setup you should have a very basic Vault setup. Beyond this you will need to add some k/v pairs and test out the commands from your salt-master.

An example is:

salt '<target>' vault.read_secret "secret/somesecret"

1	salt '<target>' vault.read_secret "secret/somesecret"

And dont forget to wire up the salt master for Vault:

https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.vault.html

Hopefully I have not missed anything, if I did please leave a comment and I will update the post.

Next up will be InfuxDB, Grafana, and Telegraf…

Welcome to the Blog @ utahtechpodcast.com!!!

Hashicorp Vault & Salt Setup.

Vault Setup:

Consul Setup: