Register your services to Consul

In the previous tutorial, you deployed a Consul server with all security features enabled and explored how to use Consul as a KV store and DNS server.∂ç

Consul's value shines when you use them with Consul clients to serve as a distributed health monitoring platform for your services and as a centralized service catalog.

In this tutorial, you will deploy Consul client agents to your virtual machine (VM) workloads. Then, you will register the services to the Consul catalog and setup a distributed monitoring system using Consul health checks.

In this tutorial, you will:

• Deploy your VM environment on AWS EC2 using Terraform
• Configure Consul client agents for the different VMs
• Start Consul client instances on your workload VMs
• Configure your terminal to communicate with the Consul datacenter
• Verify Consul datacenter members
• Query Consul catalog using CLI, API and DNS interfaces
• Modify a service definition and update the service in Consul catalog

Tutorial scenario

This tutorial uses HashiCups, a demo coffee shop application made up of several microservices running on VMs.

At the beginning of the tutorial, you have an instance of HashiCups running on four VMs and one Consul server (you deployed this in the previous tutorial).

By the end of this tutorial, you will have deployed and started a Consul client agent on each VMs that hosts HashiCups. In addition, you will have registered the HashiCups services in the Consul service catalog and setup health checks for each service.

Prerequisites

$ git clone https://github.com/hashicorp-education/learn-consul-get-started-vms

$ git clone git@github.com:hashicorp-education/learn-consul-get-started-vms

$ cd learn-consul-get-started-vms/self-managed/infrastructure/aws

$ terraform init
Initializing the backend...
Initializing provider plugins...
...
Terraform has been successfully initialized!
...

$ terraform apply --auto-approve -var-file=./conf/01_consul.tfvars
## ...
Do you want to perform these actions?
 Terraform will perform the actions described above.
 Only 'yes' will be accepted to approve.
 Enter a value: yes
## ...
Apply complete! Resources: 49 added, 0 changed, 0 destroyed.

Outputs:

connection_string = "ssh -i certs/id_rsa.pem admin@`terraform output -raw ip_bastion`"
ip_bastion = "<redacted-output>"
remote_ops = "export BASTION_HOST=<redacted-output>"
retry_join = "provider=aws tag_key=ConsulJoinTag tag_value=auto-join-hcoc"
ui_consul = "https://<redacted-output>:8443"
ui_grafana = "http://<redacted-output>:3000/d/hashicups/hashicups"
ui_hashicups = "http://<redacted-output>"

Login into the bastion host VM

Terraform output provides a series of useful information, including bastion host IP address.

Login to the bastion host using ssh.

$ ssh -i certs/id_rsa.pem admin@`terraform output -raw ip_bastion`

Verify Consul binary

Verify that the VMs you want to deploy the Consul agents on have the Consul binary.

For example to check Consul installation on the Database VM.

Login to the Database VM from the bastion host.

$ ssh -i certs/id_rsa hashicups-db

Verify Consul binary is installed.

$ consul version
Consul v1.15.2
Revision 5e08e229
Build Date 2023-03-30T17:51:19Z
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

Return to the bastion host by exiting the SSH session.

$ exit

Repeat the steps for all VMs you want to add to the Consul service mesh.

Configure environment

This tutorial and interactive lab environment uses scripts in the tutorial's GitHub repository to generate the Consul configuration files for your client agents.

Login to the bastion host.

$ ssh -i certs/id_rsa.pem admin@`terraform output -raw ip_bastion`

Define scenario environment variables again.

$ export DATACENTER="dc1"; \
export DOMAIN="consul"; \
export OUTPUT_FOLDER="./assets/scenario/conf/"; \
export CONSUL_CONFIG_DIR="/etc/consul.d/"

Configure the Consul CLI to interact with the Consul server.

$ export CONSUL_HTTP_ADDR="https://consul-server-0:8443" \
export CONSUL_HTTP_SSL=true \
export CONSUL_CACERT="${OUTPUT_FOLDER}secrets/consul-agent-ca.pem" \
export CONSUL_TLS_SERVER_NAME="server.${DATACENTER}.${DOMAIN}"

To interact with Consul, you need to set CONSUL_HTTP_TOKEN to a valid Consul token. For this tutorial, you will use the token you created during the ACL bootstrap.

$ export CONSUL_HTTP_TOKEN=`cat ./acl-token-bootstrap.json | jq -r ".SecretID"`

$ export CONSUL_HTTP_TOKEN=`cat ${OUTPUT_FOLDER}secrets/acl-token-bootstrap.json | jq -r ".SecretID"`

Finally, export the token you generated for DNS, to be used as default token for clients.

$ export CONSUL_DNS_TOKEN=`cat ${OUTPUT_FOLDER}secrets/acl-token-dns.json | jq -r ".SecretID"`

Generate Consul clients configuration

To be able to connect to Consul servers you will setup the retry_join value using the Terraform output.

$ export CONSUL_RETRY_JOIN="<use value of Terraform output retry_join here>"

For Consul service definition to be created properly, define a Consul token to be included in the service definition file.

$ export CONSUL_AGENT_TOKEN="${CONSUL_HTTP_TOKEN}"

Generate configuration for Database agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-db"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - [hashicups-db]
[generate_consul_client_config.sh] - Parameter Check
[generate_consul_client_config.sh] - Cleaning Scenario before apply.
[generate_consul_client_config.sh] - [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
[generate_consul_client_config.sh] - Generate folder structure
[generate_consul_client_config.sh] - Copy available configuration
[generate_consul_client_config.sh] - Generating configuration for Consul agent hashicups-db

To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] [hashicups-db]
[generate_consul_service_config.sh] - Parameter Check
[generate_consul_service_config.sh] - Generating service definition for service discovery
[generate_consul_service_config.sh] - Generating service definition for service discovery

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Generate configuration for API agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-api"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - [hashicups-api]
[generate_consul_client_config.sh] - Parameter Check
[generate_consul_client_config.sh] - Cleaning Scenario before apply.
[generate_consul_client_config.sh] - [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
[generate_consul_client_config.sh] - Generate folder structure
[generate_consul_client_config.sh] - Copy available configuration
[generate_consul_client_config.sh] - Generating configuration for Consul agent hashicups-api

To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] [hashicups-api]
[generate_consul_service_config.sh] - Parameter Check
[generate_consul_service_config.sh] - Generating service definition for service discovery
[generate_consul_service_config.sh] - Generating service definition for service discovery

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Generate configuration for Frontend agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-frontend"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - [hashicups-frontend]
[generate_consul_client_config.sh] - Parameter Check
[generate_consul_client_config.sh] - Cleaning Scenario before apply.
[generate_consul_client_config.sh] - [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
[generate_consul_client_config.sh] - Generate folder structure
[generate_consul_client_config.sh] - Copy available configuration
[generate_consul_client_config.sh] - Generating configuration for Consul agent hashicups-frontend

To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] [hashicups-frontend]
[generate_consul_service_config.sh] - Parameter Check
[generate_consul_service_config.sh] - Generating service definition for service discovery
[generate_consul_service_config.sh] - Generating service definition for service discovery

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Generate configuration for NGINX agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-nginx"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - [hashicups-nginx]
[generate_consul_client_config.sh] - Parameter Check
[generate_consul_client_config.sh] - Cleaning Scenario before apply.
[generate_consul_client_config.sh] - [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
[generate_consul_client_config.sh] - Generate folder structure
[generate_consul_client_config.sh] - Copy available configuration
[generate_consul_client_config.sh] - Generating configuration for Consul agent hashicups-nginx

To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] [hashicups-nginx]
[generate_consul_service_config.sh] - Parameter Check
[generate_consul_service_config.sh] - Generating service definition for service discovery
[generate_consul_service_config.sh] - Generating service definition for service discovery

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Check generated files

Once you have generated the configuration files, your directory should look like the following:

$ tree ${OUTPUT_FOLDER}hashicups*
./assets/scenario/conf/hashicups-api
|-- agent-acl-tokens.hcl
|-- agent-gossip-encryption.hcl
|-- consul-agent-ca.pem
|-- consul.hcl
|-- svc
|   |-- service_discovery
|   |   `-- svc-hashicups-api.hcl
|   `-- service_mesh
|       `-- svc-hashicups-api.hcl
`-- svc-hashicups-api.hcl
./assets/scenario/conf/hashicups-db
|-- agent-acl-tokens.hcl
|-- agent-gossip-encryption.hcl
|-- consul-agent-ca.pem
|-- consul.hcl
|-- svc
|   |-- service_discovery
|   |   `-- svc-hashicups-db.hcl
|   `-- service_mesh
|       `-- svc-hashicups-db.hcl
`-- svc-hashicups-db.hcl
./assets/scenario/conf/hashicups-frontend
|-- agent-acl-tokens.hcl
|-- agent-gossip-encryption.hcl
|-- consul-agent-ca.pem
|-- consul.hcl
|-- svc
|   |-- service_discovery
|   |   `-- svc-hashicups-frontend.hcl
|   `-- service_mesh
|       `-- svc-hashicups-frontend.hcl
`-- svc-hashicups-frontend.hcl
./assets/scenario/conf/hashicups-nginx
|-- agent-acl-tokens.hcl
|-- agent-gossip-encryption.hcl
|-- consul-agent-ca.pem
|-- consul.hcl
|-- svc
|   |-- service_discovery
|   |   `-- svc-hashicups-nginx.hcl
|   `-- service_mesh
|       `-- svc-hashicups-nginx.hcl
`-- svc-hashicups-nginx.hcl

12 directories, 28 files

Each directory corresponds to a Consul client configuration for the respective node.

The scripts generated multiple configuration files to separate the configuration so it is easier to read and modify them for your environment. The following are the generated files and a description of what they do:

• The agent-acl-tokens.hcl file contains tokens for the Consul agent.
• The agent-gossip-encryption.hcl file configures gossip encryption.
• The consul-agent-ca.pem file is the public certificate for Consul CA.
• The consul.hcl file contains node specific configuration and it is needed, with this specific name, if you want to configure Consul as a systemd daemon.
• The svc-hashicups-*.hcl files contains the service definition for the services that need to be registered on each node.

Visit the agent configuration and service configuration documentation to interpret the files or to modify them when applying them to your environment.

Copy configuration on client VMs

After the script generates the client configuration, you will copy these files into the respective Consul configuration directories in each client node.

First, define the Consul configuration directory.

$ export CONSUL_REMOTE_CONFIG_DIR=/etc/consul.d/

Then, remove existing configuration from the VMs.

$ ssh -i certs/id_rsa hashicups-db "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"; \
  ssh -i certs/id_rsa hashicups-api "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"; \
  ssh -i certs/id_rsa hashicups-frontend "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"; \
  ssh -i certs/id_rsa hashicups-nginx "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*";

Finally, copy the configuration files into the different VMs.

$ scp -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-db/* hashicups-db:${CONSUL_REMOTE_CONFIG_DIR}; \
  scp -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-api/* hashicups-api:${CONSUL_REMOTE_CONFIG_DIR}; \
  scp -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-frontend/* hashicups-frontend:${CONSUL_REMOTE_CONFIG_DIR}; \
  scp -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-nginx/* hashicups-nginx:${CONSUL_REMOTE_CONFIG_DIR}

Start Consul on client VMs

Now that you have copied the configuration files to each client VMs, you will start the Consul client agent on each VM.

Setup Database Consul client agent

Login to the Database VM from the bastion host.

$ ssh -i certs/id_rsa hashicups-db

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The command starts the Consul server in the background to not lock the terminal. You can access the Consul server log through the /tmp/consul-server.log file.

Once the Consul agent is started, exit the ssh session to return to the bastion host.

$ exit

Setup API Consul client agent

Login to the API VM from the bastion host.

$ ssh -i certs/id_rsa hashicups-api

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The process is started in background to not lock the terminal. Consul server log can be accessed in the /tmp/consul-client.log file.

Once the Consul agent is started , exit the ssh session to return to the bastion host.

$ exit

Setup Frontend Consul client agent

Login to the Frontend VM from the bastion host.

$ ssh -i certs/id_rsa hashicups-frontend

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The process is started in background to not lock the terminal. Consul server log can be accessed in the /tmp/consul-client.log file.

Once the Consul agent is started , exit the ssh session to return to the bastion host.

$ exit

Setup NGINX Consul client agent

Login to the NGINX VM from the bastion host.

$ ssh -i certs/id_rsa hashicups-nginx

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The process is started in background to not lock the terminal. Consul server log can be accessed in the /tmp/consul-client.log file.

Once the Consul agent is started , exit the ssh session to return to the bastion host.

$ exit

Verify Consul datacenter members

After you started all Consul agents, verify they successfully joined the Consul datacenter. If you are using the interactive lab environment, go to the Operator tab.

$ consul members
Node                Address          Status  Type    Build   Protocol  DC   Partition  Segment
consul-server-0     10.0.4.189:8301  alive   server  1.15.2  2         dc1  default    <all>
hashicups-api       10.0.4.75:8301   alive   client  1.15.2  2         dc1  default    <default>
hashicups-db        10.0.4.217:8301  alive   client  1.15.2  2         dc1  default    <default>
hashicups-frontend  10.0.4.35:8301   alive   client  1.15.2  2         dc1  default    <default>
hashicups-nginx     10.0.4.59:8301   alive   client  1.15.2  2         dc1  default    <default>

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${DATACENTER}.${DOMAIN}:8443:consul:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${DATACENTER}.${DOMAIN}:8443/v1/agent/members | jq

[
  {
    "Name": "hashicups-frontend",
    "Addr": "10.0.4.35",
    "Port": 8301,
    "Tags": {
      "build": "1.15.2:5e08e229",
      "dc": "dc1",
      "id": "68c25a98-586e-473c-9382-b81ed1f03b05",
      "role": "node",
      "segment": "",
      "vsn": "2",
      "vsn_max": "3",
      "vsn_min": "2"
    },
    "Status": 1,
    "ProtocolMin": 1,
    "ProtocolMax": 5,
    "ProtocolCur": 2,
    "DelegateMin": 2,
    "DelegateMax": 5,
    "DelegateCur": 4
  },
  {
    "Name": "hashicups-nginx",
    "Addr": "10.0.4.59",
    "Port": 8301,
    "Tags": {
      "build": "1.15.2:5e08e229",
      "dc": "dc1",
      "id": "9443f6b4-ed13-ce13-bf46-edf39a044a9d",
      "role": "node",
      "segment": "",
      "vsn": "2",
      "vsn_max": "3",
      "vsn_min": "2"
    },
    "Status": 1,
    "ProtocolMin": 1,
    "ProtocolMax": 5,
    "ProtocolCur": 2,
    "DelegateMin": 2,
    "DelegateMax": 5,
    "DelegateCur": 4
  },
  {
    "Name": "consul-server-0",
    "Addr": "10.0.4.189",
    "Port": 8301,
    "Tags": {
      "acls": "1",
      "bootstrap": "1",
      "build": "1.15.2:5e08e229",
      "dc": "dc1",
      "ft_fs": "1",
      "ft_si": "1",
      "grpc_tls_port": "8503",
      "id": "202600ef-f9c6-72c2-ed59-9f28d0087991",
      "port": "8300",
      "raft_vsn": "3",
      "role": "consul",
      "segment": "",
      "use_tls": "1",
      "vsn": "2",
      "vsn_max": "3",
      "vsn_min": "2",
      "wan_join_port": "8302"
    },
    "Status": 1,
    "ProtocolMin": 1,
    "ProtocolMax": 5,
    "ProtocolCur": 2,
    "DelegateMin": 2,
    "DelegateMax": 5,
    "DelegateCur": 4
  },
  {
    "Name": "hashicups-db",
    "Addr": "10.0.4.217",
    "Port": 8301,
    "Tags": {
      "build": "1.15.2:5e08e229",
      "dc": "dc1",
      "id": "873442d2-f63c-8203-874d-d7f6bb90f277",
      "role": "node",
      "segment": "",
      "vsn": "2",
      "vsn_max": "3",
      "vsn_min": "2"
    },
    "Status": 1,
    "ProtocolMin": 1,
    "ProtocolMax": 5,
    "ProtocolCur": 2,
    "DelegateMin": 2,
    "DelegateMax": 5,
    "DelegateCur": 4
  },
  {
    "Name": "hashicups-api",
    "Addr": "10.0.4.75",
    "Port": 8301,
    "Tags": {
      "build": "1.15.2:5e08e229",
      "dc": "dc1",
      "id": "d3a11ea4-d8ad-f6cb-0c6d-ffded929b65f",
      "role": "node",
      "segment": "",
      "vsn": "2",
      "vsn_max": "3",
      "vsn_min": "2"
    },
    "Status": 1,
    "ProtocolMin": 1,
    "ProtocolMax": 5,
    "ProtocolCur": 2,
    "DelegateMin": 2,
    "DelegateMax": 5,
    "DelegateCur": 4
  }
]

terraform output -raw ui_consul

Query services in Consul catalog

When you started the Consul client agents, they registered the service running on their node into the Consul catalog. Each service definition also contained the service's health check. You can find the service definition files in each node's respective Consul configuration file you defined earlier.

Query the healthy services using the Consul CLI, API, or DNS.

$ consul catalog services -tags
consul
hashicups-api           v1
hashicups-db            v1
hashicups-frontend      v1
hashicups-nginx         v1

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${DATACENTER}.${DOMAIN}:8443:consul:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${DATACENTER}.${DOMAIN}:8443/v1/catalog/services | jq

{
  "consul": [],
  "hashicups-api": [
    "v1"
  ],
  "hashicups-db": [
    "v1"
  ],
  "hashicups-frontend": [
    "v1"
  ],
  "hashicups-nginx": [
    "v1"
  ]
}

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${DATACENTER}.${DOMAIN}:8443:consul:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${DATACENTER}.${DOMAIN}:8443/v1/catalog/service/db | jq

[
  {
    "ID": "873442d2-f63c-8203-874d-d7f6bb90f277",
    "Node": "hashicups-db",
    "Address": "10.0.4.217",
    "Datacenter": "dc1",
    "TaggedAddresses": {
      "lan": "10.0.4.217",
      "lan_ipv4": "10.0.4.217",
      "wan": "10.0.4.217",
      "wan_ipv4": "10.0.4.217"
    },
    "NodeMeta": {
      "consul-network-segment": ""
    },
    "ServiceKind": "",
    "ServiceID": "hashicups-db-1",
    "ServiceName": "hashicups-db",
    "ServiceTags": [
      "v1"
    ],
    "ServiceAddress": "",
    "ServiceWeights": {
      "Passing": 1,
      "Warning": 1
    },
    "ServiceMeta": {},
    "ServicePort": 5432,
    "ServiceSocketPath": "",
    "ServiceEnableTagOverride": false,
    "ServiceProxy": {
      "Mode": "",
      "MeshGateway": {},
      "Expose": {}
    },
    "ServiceConnect": {},
    "CreateIndex": 50,
    "ModifyIndex": 50
  }
]

$ dig @127.0.0.1 -p 8600 hashicups-db.service.dc1.consul ANY

; <<>> DiG 9.16.37-Debian <<>> @127.0.0.1 -p 8600 hashicups-db.service.dc1.consul ANY
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44882
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;hashicups-db.service.dc1.consul. IN    ANY

;; ANSWER SECTION:
hashicups-db.service.dc1.consul. 0 IN   A   10.0.4.217

;; Query time: 4 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Wed May 17 22:21:24 UTC 2023
;; MSG SIZE  rcvd: 76

dig @consul-server-0 -p 8600 hashicups-db.service.dc1.consul

Modify service definition tags

When using Consul CLI or the API endpoints, Consul will also show you the metadata associated with the services. In this tutorial, you registered each service with the v1 tag.

In this section, you will update the database service definition to learn how to update Consul service definitions. You must run these commands on the virtual machine that hosts the services.

First, login to the Database VM from the bastion host.

$ ssh -i certs/id_rsa hashicups-db

Next, create the new configuration file. Notice that this configuration adds a v2 tag to the database service.

## -----------------------------
## svc-hashicups-db.hcl
## -----------------------------
service {
  name = "hashicups-db"
  id = "hashicups-db-1"
  tags = ["v1", "v2"]
  port = 5432


  check
  {
    id =  "check-hashicups-db",
    name = "hashicups-db status check",
    service_id = "hashicups-db-1",
    tcp  = "localhost:5432",
    interval = "5s",
    timeout = "5s"
  }
}

{
  "service": {
    "id": "hashicups-db-1",
    "name": "hashicups-db",
    "tags": ["v1", "v2"],
    "port": 5432,
    "checks": [
      {
        "ID" : "check-hashicups-db",
        "Name" : "hashicups-db status check",
        "ServiceID" : "hashicups-db-1",
        "TCP": "localhost:5432",
        "interval": "1s",
        "Timeout": "1s"
      }
    ]
  }
}

{
  "ID": "hashicups-db-1",
  "Name": "hashicups-db",
  "Tags": ["v1", "v2"],
  "Port": 5432,
  "Checks": [
    {
      "CheckID" : "check-hashicups-db",
      "Name" : "hashicups-db status check",
      "ServiceID" : "hashicups-db-1",
      "TCP": "localhost:5432",
      "Interval": "1s",
      "Timeout": "1s"
    }
  ]
}

Once you have created the new service definition file, update the service in the Consul catalog.

$ mv svc-hashicups-db.hcl /etc/consul.d/

$ export CONSUL_CONFIG_DIR="/etc/consul.d"; \
export CONSUL_HTTP_ADDR="localhost:8500"; \
export CONSUL_HTTP_TOKEN=`cat ${CONSUL_CONFIG_DIR}/agent-acl-tokens.hcl | grep agent | awk '{print $3}'| sed 's/"//g'`

$ consul reload
Configuration reload triggered

$ export CONSUL_CONFIG_DIR="/etc/consul.d"; \
export CONSUL_HTTP_ADDR="localhost:8500"; \
export CONSUL_HTTP_TOKEN=`cat ${CONSUL_CONFIG_DIR}/agent-acl-tokens.hcl | grep agent | awk '{print $3}'| sed 's/"//g'`

$ consul services register svc-hashicups-db.hcl
Registered service: db

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --data @svc-hashicups-db-api.json \
  --request PUT \
  http://127.0.0.1:8500/v1/agent/service/register

Query services by tags

After you have updated the database service definition, query it to verify the new tag.

$ consul catalog services -tags
consul
hashicups-api           v1
hashicups-db            v1,v2
hashicups-frontend      v1
hashicups-nginx         v1

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${DATACENTER}.${DOMAIN}:8443:consul:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${DATACENTER}.${DOMAIN}:8443/v1/catalog/services | jq

{
  "consul": [],
  "hashicups-api": [
    "v1"
  ],
  "hashicups-db": [
    "v2",
    "v1"
  ],
  "hashicups-frontend": [
    "v1"
  ],
  "hashicups-nginx": [
    "v1"
  ]
}

$ dig @127.0.0.1 -p 8600 v2.hashicups-db.service.dc1.consul

; <<>> DiG 9.16.37-Debian <<>> @127.0.0.1 -p 8600 v2.hashicups-db.service.dc1.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42200
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;v2.hashicups-db.service.dc1.consul. IN A

;; ANSWER SECTION:
v2.hashicups-db.service.dc1.consul. 0 IN A  10.0.4.217

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Thu May 18 00:28:09 UTC 2023
;; MSG SIZE  rcvd: 79

$ dig @consul-server-0 -p 8600 v2.hashicups-db.service.dc1.consul

Next steps

In this tutorial, you deployed Consul clients on each HashiCups' nodes VMs. In addition, you configured Consul to perform health checks on the registered services and updated a service definition.

You now have a distributed system to monitor and resolve your services all without changing your services' configuration or implementation. At this stage, you can use Consul to automatically configure and monitor your services. However, they have the same security they had before introducing Consul.

$ terraform destroy --auto-approve

In the next tutorial, you will learn how to introduce zero trust security in your network by implementing Consul service mesh.

For more information about the topics covered in this tutorial, refer to the following resources:

• Register Services with Service Definitions
• Query services with DNS
• Health Checks