Title: Creating a Microservice — UCS Tools Documentation

URL Source: https://docs.nvidia.com/ucf/2.10.0/text/UCS_MsDevelopment.html

Published Time: Thu, 30 Oct 2025 07:23:03 GMT

Markdown Content: Creating a Microservice#

This section covers the steps required to create a UCS Microservice. UCS Microservices are built using the UCS Microservice Builder CLI (MS builder) tool. The tool takes a directory with a specific structure and files with specific names and formats as input. The tool outputs a UCS Microservice specification for the microservice. It may also output Container Images used by the microservice and a Helm Chart for the microservice. The output depends on the use case and type of input to the UCS Microservice Builder CLI tool.

For more information on what a UCS Microservice is, see UCS Microservice.

For the MS builder input directory specification, see Microservice Builder Input Specification.

Set the NGC Personal key to sync microservices from NGC:

First, make sure you have followed the prerequisites for generating your NGC Personal Key.

$ ucf_app_builder_cli registry repo set-api-key -a <API-KEY> Checking which NGC UCS teams you have access to Adding NGC Catalog team ace Repository 'catalog-ace' added Adding NGC Catalog team ucs-ms

The command prints the repositories accessible using your API key. Repository names that begin with “catalog-” exist in the NGC Unified Catalog rather than the NGC Private Registry. The NGC Unified Catalog contains artifacts that are either fully public or require product enablement (subscription) such as ace-ea or nvaie to access certain artifacts.

If you get an error when setting the key that reads something like:

2024-11-05 04:18:30,844 - Registry - ERROR - Failed to download service metadata for ucstools_ms_list 2024-11-05 04:18:30,844 - AppBuilder - ERROR - Unable to fetch catalog services list from NGC

Then ensure that you added the NGC Catalog scope when you created your NGC Personal key as documented in NGC Personal Key.

Generate Scaffolding#

The UCS Microservice Builder CLI service create command generates scaffolding for the new service:

$ ucf_ms_builder_cli service create [-h] [-d DESCRIPTION] [-o OUTPUT_DIR] -n NAME -i {container-image,container-config,helm-chart} [-c CHART]

The tool supports creating microservices for three types of inputs:

• Pre-built container images

• Container build configs

• Pre-built Helm charts

Pre-built Container Images#

Use this option when container images are already available for the microservice to be built. With this method, the tool generates:

• Helm chart for the microservice

• UCS Microservice specification for the microservice

To use this method, run:

$ ucf_ms_builder_cli service create -n myservice -d "my service description" -i container-image

This command generates a template microservice in the following output directory structure:

myservice ├── changelog.txt ├── configs ├── endpoints │  ├── http-api.yaml │  └── myservice-endpoint-name.yaml ├── LICENSE.txt ├── manifest.yaml ├── manual_compliance_test_results.yaml ├── README.md ├── scripts └── tests └── dev ├── app.yaml └── params1.yaml

Container Build Config#

Use this option when container images must be built along with the microservice. With this method, the tool generates:

• Container Images for the microservice

• Helm chart for the microservice

• UCS Microservice specification for the microservice

To use this method, run:

$ ucf_ms_builder_cli service create -n myservice -d "my service description" -i container-config

This command generates a template microservice in the following output directory structure:

myservice ├── changelog.txt ├── configs ├── containers │  └── cb_config.yaml ├── endpoints │  ├── http-api.yaml │  └── myservice-endpoint-name.yaml ├── LICENSE.txt ├── manifest.yaml ├── manual_compliance_test_results.yaml ├── README.md ├── scripts ├── target_x86_64.yaml └── tests └── dev ├── app.yaml └── params1.yaml

Pre-built Helm Chart#

Use this option when the Helm chart for the microservice is already available but the UCS Microservice specification needs to be created. With this method, the tool generates:

• UCS Microservice specification for the microservice

To use this method, run:

$ ucf_ms_builder_cli service create -n myservice -d "my service description" -i helm-chart -c <helm chart link>

This command generates a template microservice in the following output directory structure:

myservice ├── changelog.txt ├── endpoints │  ├── http-api.yaml │  └── myservice-endpoint-name.yaml ├── LICENSE.txt ├── manifest.yaml ├── manual_compliance_test_results.yaml ├── README.md ├── tests │  └── dev │  ├── app.yaml │  └── params1.yaml └── values.yaml

Updating Microservice Builder Input#

As demonstrated in the previous section, the input differs slightly based on the use case and type of input to the tool. This section describes how to update the files in the input directory structure for all use cases.

Update configs#

Add any configuration files required by the microservice to this directory. Files in this directory can contain placeholders. These placeholders get overridden by a UCS application.

The files in this directory are NOT used when using a Pre-built Helm Chart.

Example of a file app_cfg.yaml in the configs directory containing placeholders:

redis: host: $egress.redis.address port: $egress.redis.port

log_level: 5

timeout: $params.timeout

The file app_cfg.yaml in the configs directory gets mounted at /opt/configs/app_cfg.yaml in containers that are part of the microservice. The placeholders get updated with actual values.

When an application containing this microservice (with redis egress endpoint connected and timeout param set to 10) is built and deployed, a file /opt/configs/app_cfg.yaml gets mounted to containers that are part of the microservice with the following contents:

redis: host: redis-svc port: 6739

log_level: 5

timeout: 10

Update scripts#

Add any script files required by the microservice to this directory.

The files in this directory are NOT used when using a Pre-built Helm Chart.

For example, a file named run.sh in the scripts directory gets mounted at /opt/scripts/run.sh in containers that are part of the microservice with exactly the same contents.

Adding Container Build Config Files to containers#

Add and update Container Build config files to the containers directory for any containers that must be built along with the microservice.

The files in this directory are NOT used when using a Pre-built Helm Chart or when using Pre-built Container Images.

Update endpoints#

Add endpoint definition files for each ingress/egress endpoint as per the UCS MS Specification under the endpoints directory. The filename must be <endpoint-name>.<ext>. The extension and content of the file will depend on the scheme of the endpoint.

For more details on the contents, formats, and file names for the endpoint definition files, see UCS Endpoint Schemes.

Update manual_compliance_test_results.yaml#

Developers must run manual compliance checks as specified in UCS Microservice Compliance and update the results (Boolean) as true / false for individual compliance rules in manual_compliance_test_results.yaml.

Add tests#

The tests directory contains UCS Applications that are basically tests for the microservice. The test apps follow the UCS Application graph file specification - UCS Tools Application.

These tests should ideally test the microservice as a whole. They should test if the helm chart for the microservice is working correctly rather than unit tests for the applications running inside the microservice.

The tests could verify, for example, if accessing APIs through the microservice endpoints works. They could also verify if the application works correctly when containerized and deployed through Kubernetes by calling APIs with known inputs and corresponding known outputs.

If the microservice being built depends on other microservices, these other microservices may be replaced by a stub microservice or simpler functional equivalent microservice to remove dependency on other microservices when testing.

Update values.yaml#

This file is used ONLY when using a Pre-built Helm Chart.

It is in the form of a standard Helm values.yaml and contains values of the pre-built helm chart that must be overridden. The file can contain the $params, $egress and $secrets placeholders which get substituted with the actual values when the application is built and deployed.

When the scaffolding is generated with Pre-built Helm Chart type - values.yaml is pulled from the helm chart and placed here. The file can be trimmed and updated by the microservice developer to keep only those values that need to be overridden by the microservice or the application using the microservice.

For example, when a scaffolding is generated for a helm chart, the values.yaml may look like this:

applicationSpecs: myservice-deployment: apptype: stateless containers: myservice-container: env: - name: REDIS_ENDPOINT value: redis-svc:6379 command: - sleep - '1000' image: repository: ubuntu tag: latest ports: - containerPort: 1000 name: http restartPolicy: Always securityContext: runAsGroup: 1000 runAsUser: 1000 services: myservice-service: ports: - name: http-api port: 1000 image: pullPolicy: IfNotPresent imagePullSecrets: [] ingress: enabled: false

This can be updated by the microservice developer to remove values not to be overridden, and add placeholders like $params and $egress:

applicationSpecs: myservice-deployment: apptype: stateless containers: myservice-container: env: - name: REDIS_ENDPOINT value: $egress.redis.address:$egress.redis.port command: - sleep - '1000' image: repository: ubuntu tag: $params.stringToEcho

Update Documentation#

The microservice builder tool accepts four pieces of documentation:

• README.md - Detailed information about the microservice. The template generated by service create command has multiple sections like detailed description, usage information, performance KPIs, supported platforms and deployment requirements, known issues etc. Microservice developer must fill in the information in these sections as applicable.

• LICENSE.txt - Contains License text for any 3rdparty software used in the microservice. This includes the licenses for the 3rdparty software used in the applications running inside the containers as well as any 3rdparty packages added to the container file system.

• changelog.txt - A list of changes in the current version as compared to the previous version.

Updating Manifest file manifest.yaml#

The manifest.yaml contains information for describing the microservice, the helm chart implementation and container build config details.

The template manifests generated for each of the three use cases differ slightly. This section describes how to update the manifest file for the first use case i.e. Pre-built Container Images and then describes the differences for other use cases where applicable.

Following is an example of the template manifest.yaml file generated for Pre-built Container Images use case:

type: msapplication specVersion: 2.5.0 name: ucf.svc.myservice chartName: myservice description: default description version: 0.0.1 tags: [] keywords: []

publish: false

egress-endpoints:

• name: myservice-endpoint-name description: Short description of endpoint protocol: TCP # Or UDP scheme: asyncio # Or grpc / rtsp / asyncio / none mandatory: True # Or False data-flow: in-out # Or in or out

ingress-endpoints:

• name: http-api description: Short description of http-api ingress endpoint scheme: http data-flow: in-out # Or in or out

secrets:

• name: some-secret-name description: Description for the secret mandatory: True mountPath: /secrets fileName: someSecretFileName

params: stringToEcho: someString #> type: string #> enum_values: someString, someOtherString #> description: String to echo in init container timeToSleep: 1000000 #> type: integer #> maximum: 2000000 #> minimum: 1000000 #> description: String to echo in init container #> flags: mandatory

metrics:

• portName: metrics # Name of pod container port which implements prometheus metrics path: /metrics # HTTP path where metrics are available

List of metrics exported by the microservice endpoint and their details

details:

• name: latency description: Average latency of handling HTTP requests in milliseconds labels: [request-type]

recommendedMin: 5 # Optional

recommendedMax: 50 # Optional

Optional. Selector for pods that implement the metrics endpoint

podSelectorLabels:

app: myservice-myservice-deployment

Files and directories will be mounted with prefix /opt/ext-files/

externalFiles:

• name: some-config.yaml # File will be available in containers at /opt/ext-files/some-config.yaml description: Some Configuration file mandatory: True isDirectory: False

tests:

• name: dev-params1 app: tests/dev/app.yaml params: tests/dev/params1.yaml ciTrigger: false timeout: 10 duration: 10 installPreReqs: true # Whether to install foundational services namespace: default # Kubernetes namespace gpuNodeLabels: "" watchAllPods: true # OR set to false and set list of pods to watch below watchPods:
• <pod-name-regex>

testerPods: # At least one tester pod is required

• name: testpod1 # Name of the test pod startSignature: <START> # Signature to look for in the logs indicating start of tests. Regex is accepted endSignature: <END> # Signature to look for in the logs indicating end of tests. Regex is accepted errorSignatures: # Signatures that indicate test failures. Regex is accepted
• <REGEX1>
• <REGEX2>

---

spec:

• name: myservice-deployment type: ucf.k8s.app.deployment parameters: apptype: stateless

• name: myservice-init-container type: ucf.k8s.initcontainer parameters: image: ubuntu imagePullPolicy: IfNotPresent args: [echo, $params.stringToEcho]

• name: "myservice-container" type: ucf.k8s.container parameters: image: repository: ubuntu tag: "latest" command: ["sleep", "$params.timeToSleep"] ports:

• containerPort: 0 # <PORT> name: http startupProbe: httpGet: path: /healthz port: http failureThreshold: 30 periodSeconds: 10 livenessProbe: httpGet: path: /healthz port: http readinessProbe: exec: command:

• cat

• /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5

• name: myservice-service type: ucf.k8s.service parameters: annotations: test-svc-annotation: annotation-value labels: test-svc-label: label-value ports:

• port: 0 # <OUT_PORT> name: http-api #externalTrafficPolicy: # Cluster / Local. Allowed when service type is NodePort or LoadBalancer

Update the following sections based on the manifest specification:

• Basic Information

• Ingress Endpoints

• Egress Endpoints

• Parameters

• Secrets

• External Files

• Metrics

• Tests along with test applications in tests directory - Add tests

Spec#

Implementation of the microservice (i.e. its helm chart) can be specified under the spec field. As part of the UCS Microservice Builder Tool, the microservice implementation must be done using composable components.

When using Pre-built Helm Chart, this section must be omitted. MSBuilder ignores this field since it does not build the microservice helm chart.

Components#

Components define a small part of the microservice implementation such as a container, a service, a volume. The list of available components can be viewed using:

$ ucf_ms_builder_cli component list List of available components:

ucf.appspec.app.workload ucf.appspec.defaultVolumeMount ucf.k8s.app.deployment ucf.k8s.configmap ucf.k8s.container ucf.k8s.imagepullsecret ucf.k8s.initcontainer ucf.k8s.podAnnotations ucf.k8s.podLabels ucf.k8s.podSecurityContext ucf.k8s.pvc ucf.k8s.restartPolicy ucf.k8s.service ucf.k8s.volume ...

To view details about a component, run:

$ ucf_ms_builder_cli component info -c ucf.k8s.service type: ucf.k8s.service description: Service properties: pod: (string ), Service pod. [Mandatory:False] ports: (array ), An explanation about the purpose of this instance. [Mandatory:True] port: (integer), Service port. [Mandatory:True] name: (string ), Name for service port (this is used as endpoint name) [Mandatory:False] targetPort: (['string', 'integer']), Target port number or name. [Mandatory:False] nodePort: (integer), Target port number or name. [Mandatory:False] range: (integer), Range of ports. 0-<range - 1> will be added added to port/targetPort/nodePort [Mandatory:False] type: (string ), Service type [Mandatory:False, Allowed Values:{ ClusterIP, NodePort }] externalTrafficPolicy: (string ), Route external traffic to node-local or cluster-wide endpoints [Mandatory:False, Allowed Values:{ Cluster, Local }] nameOverride: (boolean), Enable to set service name as set as <appname>-<service-name> [Mandatory:False] fullNameOverride: (boolean), Enable to set service name as set as <service-name> [Mandatory:False] extraSpecs: (object ), Extra specs to set on the service [Mandatory:False] annotations: (object ), Annotations to set on the service [Mandatory:False] labels: (object ), Labels to set on the service [Mandatory:False]

Commonly used components#

The following table lists the commonly used components and their purpose:

Adding components to the spec#

The components must be added under spec as a list along with their parameters. The parameter values must follow the parameter schema for the component which can be viewed using the component info command as shown above. Placeholders $params, $egress and $secrets can be used in parameter values to refer to the parameters, egress endpoint and secret respectively. These placeholders remain in the built microservice helm chart but are substituted during application build.

An example of adding components with placeholders is:

spec:

• name: myservice-deployment type: ucf.k8s.app.deployment parameters: apptype: stateless

• name: "myservice-container" type: ucf.k8s.container parameters: image: repository: ubuntu tag: "latest" env:

• name: HTTP_API_CA_CERT_PATH value: $secrets.ca-cert.path command: ["curl", "$egress.http-api.address:$egress.http-api.port", "--connect-timeout", $params.connectTimeout, "--cacert", $(HTTP_API_CA_CERT_PATH)] startupProbe: httpGet: path: /healthz port: http failureThreshold: 30 periodSeconds: 10 livenessProbe: httpGet: path: /healthz port: http readinessProbe: exec: command:

• cat

• /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5

Note

For the components defined in spec, parameters must adhere to the component schema specific to each type as shown above.

Only one workload type (e.g stateless, stateful, job, cronjob, daemonset, static-pod) can be specified in one yamldoc. To add multiple workload types (e.g. two stateless (i.e. Deployment) ) to the microservice:

---

spec:

• name: app-deployment-1 type: ucf.k8s.app.deployment parameters: apptype: stateless ...

---

spec:

• name: app-deployment-2 type: ucf.k8s.app.deployment parameters: apptype: stateless ...

Using container builder config#

For using the Container Build option to build containers along with the microservice, modify the image parameter of the ucf.k8s.container and ucf.k8s.initcontainer components to specify the container builder config file path. The path is relative to the manifest.yaml file. For example:

spec:

• name: myservice-init-container type: ucf.k8s.initcontainer parameters: image: config_path: containers/init-container.yaml ...
• name: myservice-container type: ucf.k8s.container parameters: image: config_path: containers/app-container.yaml ...

A template container builder config file is generated at containers/cb_config.yaml. The following changes must be done at a minimum for the config to work correctly.

• Update the base_image

• Update local_copy_files and stage_copy_files or remove them completely

• Update http_archives or remove completely

• Update image_name under docker_build to where the image will be uploaded.

Building a microservice#

Once the input for the microservice builder tool is ready, build the microservice using:

$ ucf_ms_builder_cli service build -d <msbuilder-input-dir>

$ ucf_ms_builder_cli service build -d myservice/ 2022-12-05 14:54:13,927 - Registry - INFO - Building: myservice/ 2022-12-05 14:54:15,943 - MsBuilder - INFO - Helm chart generated in myservice/output/helm 2022-12-05 14:54:15,962 - MsBuilder - INFO - MS spec generated in myservice/output/msspec 2022-12-05 14:54:17,034 - MsBuilder - WARNING - Mandatory compliance checks failed. Check myservice/output/compliance_test_logs.txt for more information 2022-12-05 14:54:17,176 - MsBuilder - INFO - MS spec generated in myservice/output/msspec 2022-12-05 14:54:17,185 - MsBuilder - INFO - Building test application 'dev-params1' 2022-12-05 14:54:17,327 - MsBuilder - INFO - Syncing any missing service versions to cache... 2022-12-05 14:54:17,329 - MsBuilder - INFO - Validating application ... 2022-12-05 14:54:17,335 - MsBuilder - INFO - Building application myservice-dev-params1-0.0.1 ... 2022-12-05 14:54:17,671 - MsBuilder - INFO - Application compliance report generated at myservice/output/tests/dev-params1/compliance_report.json 2022-12-05 14:54:17,758 - MsBuilder - INFO - Application Helm Chart generated in myservice/output/tests/dev-params1 2022-12-05 14:54:17,759 - MsBuilder - INFO - Tests generated in myservice/output/tests 2022-12-05 14:54:17,902 - Registry - INFO - Generating service helm chart package... 2022-12-05 14:54:17,993 - Registry - INFO - Generating test helm chart packages... 2022-12-05 14:54:18,152 - MsBuilder - INFO - Added microservice 'ucf.svc.myservice:0.0.1' to local repository

If using the Container Build option, add an additional argument -t <target-file>:

$ ucf_ms_builder_cli service build -d myservice/ -t myservice/target_x86_64.yaml

The target file describes the configuration of the target the container is being built for. A sample target file <msbuilder-input-dir>/target_x86_64.yaml with following contents is added as part of the template microservice generation:

---

platform: arch: x86_64 os: linux distribution: ubuntu_22.04 compute: cuda: 11.8 tensorrt: 8.5.1 cudnn: 8.6.0 deepstream: 6.2 triton: null vpi: 2.1.6

Container image built by the tool can be manually uploaded if required using:

$ docker images # List all images and find the image that was built with tool $ docker push <image>

Note

Some Kubernetes configurations cannot pull images if the images are in local docker cache. Kubernetes might have its own container runtime instance running and does not use the local docker cache. In this case, push the container images to a container registry where Kubernetes can pull images from. Make corresponding changes in the manifest file if required to point to the new repository. When using Microk8s, its built-in registry may be used: https://microk8s.io/docs/registry-built-in.

Build Output#

Building the microservice generates the following output based on the use case:

• UCS Microservice Specification for the microservice

• Helm Chart - for Pre-built Container Image and Container Build use cases

• Container Images - for Container Build use case

• Test Application Helm Chart - for each of the tests mentioned in the manifest.yaml

• Compliance Report for the microservice

• Compliance Check Logs are generated at <msbuilder-input-dir>/output/compliance_test_logs.txt

Once the microservice is built successfully, it gets automatically added to the local repository and is ready to use in UCS Applications.

The information for the built microservice can be viewed using Viewing Microservice Information.

For more information on compliance refer to UCS Microservice Compliance. Microservice developers can refer to the log file at <msbuilder-input-dir>/output/compliance_test_logs.txt for the reasons for any compliance failures.

Links/Buttons:

• #
• UCS Microservice Builder CLI
• UCS Microservice
• Microservice Builder Input Specification
• NGC Personal Key
• UCS Endpoint Schemes
• UCS Microservice Compliance
• UCS Tools Application
• Basic Information
• Ingress Endpoints
• Egress Endpoints
• Parameters
• Secrets
• External Files
• Metrics
• Tests
• Add tests
• https://microk8s.io/docs/registry-built-in
• Viewing Microservice Information