Commit f02a37fa authored by Viktor Nagy (GitLab)'s avatar Viktor Nagy (GitLab) 🕊
Browse files

Initial commit

version: 2
machine: true
- checkout
- run: |
docker login -u $DOCKER_USER -p $DOCKER_PASS
docker build -t ajaytripathy/kubecost-cost-model:$TAG .
docker push ajaytripathy/kubecost-cost-model:$TAG
machine: true
- checkout
- run: |
docker login -u $DOCKER_USER -p $DOCKER_PASS
docker build -t ajaytripathy/kubecost-cost-model:latest .
docker push ajaytripathy/kubecost-cost-model:latest
version: 2
- build
- deploy:
- build
- master
# Contributing to our project #
Thanks for your help improving the project!
## Getting Help ##
If you have a question about Kubecost or have encountered problems using it,
you can start by asking a question on [Slack]( or via email at [](
## Building ##
Follow these steps to build from source and deploy:
1. `docker build --rm -f "Dockerfile" -t <repo>/kubecost-cost-model:<tag> .`
2. Edit the [pulled image]( in the deployment.yaml to <repo>/kubecost-cost-model:<tag>
3. Set [this environment variable]( to the address of your prometheus server
4. `kubectl create namespace cost-model`
5. `kubectl apply -f kubernetes/ --namespace cost-model`
6. `kubectl port-forward --namespace cost-model service/cost-model 9003`
To test, build the cost-model docker container and then push it to a Kubernetes cluster with a running Prometheus.
To confirm that the server is running, you can hit [http://localhost:9003/costDataModel?timeWindow=1d](http://localhost:9003/costDataModel?timeWindow=1d)
## Running the integration tests ##
To run these tests:
* Make sure you have a kubeconfig that can point to your cluster, and have permissions to create/modify a namespace called "test"
* Connect to your the prometheus kubecost emits to on localhost:9003:
```kubectl port-forward --namespace kubecost service/kubecost-prometheus-server 9003:80```
* Temporary workaround: Copy the default.json file in this project at cloud/default.json to /models/default.json on the machine your test is running on. TODO: fix this and inject the cloud/default.json path into provider.go.
* Navigate to cost-model/test
* Run ```go test -timeout 700s``` from the testing directory. The tests right now take about 10 minutes (600s) to run because they bring up and down pods and wait for Prometheus to scrape data about them.
## Certification of Origin ##
By contributing to this project you certify that your contribution was created in whole or in part by you and that you have the right to submit it under the open source license indicated in the project. In other words, please confirm that you, as a contributor, have the legal right to make the contribution.
## Committing ###
Please write a commit message with Fixes Issue # if there is an outstanding issue that is fixed. It’s okay to submit a PR without a corresponding issue, just please try be detailed in the description about the problem you’re addressing.
Please run go fmt on the project directory. Lint can be okay (for example, comments on exported functions are nice but not required on the server).
Please email us ( or reach out to us on [Slack]( if you need help or have any questions!
FROM golang:latest as build-env
RUN mkdir /app
COPY go.mod .
COPY go.sum .
# Get dependencies - will also be cached if we won't change mod/sum
RUN go mod download
# COPY the source code as the last step
COPY . .
# Build the binary
RUN set -e ;\
GIT_COMMIT=`git rev-parse HEAD` ;\
# for our purposes, we only care about dirty .go files ;\
if test -n "`git status --porcelain --untracked-files=no | grep '\.go'`"; then \
GIT_DIRTY='+dirty' ;\
fi ;\
cd cmd/costmodel;\
go build -a -installsuffix cgo \
-ldflags "-X main.gitCommit=${GIT_COMMIT}${GIT_DIRTY}" \
-o /go/bin/app
FROM alpine:3.10.2
RUN apk add --update --no-cache ca-certificates
COPY --from=build-env /go/bin/app /go/bin/app
ADD ./configs/default.json /models/default.json
ADD ./configs/azure.json /models/azure.json
ADD ./configs/aws.json /models/aws.json
ADD ./configs/gcp.json /models/gcp.json
USER 1001
ENTRYPOINT ["/go/bin/app"]
Apache License
Version 2.0, January 2004
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
implied, including, without limitation, any warranties or conditions
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
See the License for the specific language governing permissions and
limitations under the License.
Kubecost allows you to export pricing data to Prometheus and then write custom queries for cost insights. Below are instructions for accomplishing this and a set of example queries to get you started.
## Configuration
After deploying the Kubecost model (see [README]( for more info on installation), configure Prometheus to scrape the `/metrics` endpoint exposed by Kubecost. Below is a sample scrape config:
- job_name: kubecost
honor_labels: true
scrape_interval: 1m
scrape_timeout: 10s
metrics_path: /metrics
scheme: http
- targets:
- < address of cost-model service> # example: <service-name>.<namespace>:<port>
## Example queries
Below are a set of sample queries that can be run after Prometheus begins ingesting Kubecost data:
__Monthly cost of top 5 containers__
topk( 5,
container_memory_allocation_bytes* on(instance) group_left() node_ram_hourly_cost / 1024 / 1024 / 1024 * 730
container_cpu_allocation * on(instance) group_left() node_cpu_hourly_cost * 730
__Hourly memory cost for the *default* namespace__
avg(container_memory_allocation_bytes{namespace="default"}) by (instance) / 1024 / 1024 / 1024
on(instance) group_left() avg(node_ram_hourly_cost) by (instance)
__Monthly cost of currently provisioned nodes__
sum(node_total_hourly_cost) * 730
## Available Metrics
**Note:** metrics today have both *instance* and *node* labels. The *instance* label will be deprecated in a future version.
| Metric | Description |
| ------------ | ------------------------------------------------------------------------------------------------------ |
| node_cpu_hourly_cost | Hourly cost per vCPU on this node |
| node_gpu_hourly_cost | Hourly cost per GPU on this node |
| node_ram_hourly_cost | Hourly cost per Gb of memory on this node |
| node_total_hourly_cost | Total node cost per hour |
| container_cpu_allocation | Average number of CPUs requested/used over last 1m |
| container_gpu_allocation | Average number of GPUs requested over last 1m |
| container_memory_allocation_bytes | Average bytes of RAM requested/used over last 1m |
| pod_pvc_allocation | Bytes provisioned for a PVC attached to a pod |
| pv_hourly_cost | Hourly cost per GP on a persistent volume |
## Kubecost
Kubecost models give teams visibility into current and historical Kubernetes spend and resource allocation. These models provide cost transparency in Kubernetes environments that support multiple applications, teams, departments, etc.
To see more on the functionality of the full Kubecost product, please visit the [features page]( on our website.
Here is a summary of features enabled by this cost model:
- Real-time cost allocation by Kubernetes service, deployment, namespace, label, statefulset, daemonset, pod, and container
- Dynamic asset pricing enabled by integrations with AWS, Azure, and GCP billing APIs
- Supports on-prem k8s clusters with custom pricing sheets
- Allocation for in-cluster resources like CPU, GPU, memory, and persistent volumes.
- Allocation for AWS & GCP out-of-cluster resources like RDS instances and S3 buckets with key (optional)
- Easily export pricing data to Prometheus with /metrics endpoint ([learn more](
- Free and open source distribution (Apache2 license)
## Requirements
- Kubernetes version 1.8 or higher
- kube-state-metrics
- Node exporter
- Prometheus
## Getting Started
You can deploy Kubecost on any Kubernetes 1.8+ cluster in a matter of minutes, if not seconds.
Visit the Kubecost docs for [recommended install options]( Compared to building from source, installing from Helm is faster and includes all necessary dependencies.
## Contributing
We :heart: pull requests! See [``]( for information on buiding the project from source
and contributing changes.
## Licensing
Licensed under the Apache License, Version 2.0 (the "License")
## Software stack
Golang application.
## Frequently Asked Questions
#### How do you measure the cost of CPU/RAM/GPU/storage for a container, pod, deployment, etc.
The Kubecost model collects pricing data from major cloud providers, e.g. GCP, Azure and AWS, to provide the real-time cost of running workloads. Based on data from these APIs, each container/pod inherits a cost per CPU-hour, GPU-hour, Storage Gb-hour and cost per RAM Gb-hour based on the node where it was running or the class of storage provisioned. This means containers of the same size, as measured by the max of requests or usage, could be charged different resource rates if they are scheduled in seperate regions, on nodes with different usage types (on-demand vs preemptible), etc.
For on-prem clusters, these resource prices can be configured directly with custom pricing sheets (more below).
Measuring the CPU/RAM/GPU cost of a deployment, service, namespace, etc is the aggregation of its individual container costs.
#### How do you determine RAM/CPU costs for a node when this data isn’t provided by a cloud provider?
When explicit RAM or CPU prices are not provided by your cloud provider, the Kubecost model falls back to the ratio of base CPU and RAM price inputs supplied. The default values for these parameters are based on the marginal resource rates of the cloud provider, but they can be customized within Kubecost.
These base RAM/CPU prices are normalized to ensure the sum of each component is equal to the total price of the node provisioned, based on billing rates from your provider. When the sum of RAM/CPU costs is greater (or less) than the price of the node, then the ratio between the two input prices are held constant.
As an example, let's imagine a node with 1 CPU and 1 Gb of RAM that costs $20/mo. If your base CPU price is $30 and your RAM Gb price is $10, then these inputs will be normlized to $15 for CPU and $5 for RAM so that the sum equals the cost of the node. Note that the price of a CPU remains 3x the price of a Gb of RAM.
#### How do you allocate a specific amount of RAM/CPU to an individual pod or container?
Resources are allocated based on the time-weighted maximum of resource Requests and Usage over the measured period. For example, a pod with no usage and 1 CPU requested for 12 hours out of a 24 hour window would be allocated 12 CPU hours. For pods with BestEffort quality of service (i.e. no requests) allocation is done solely on resource usage.
#### How do I set my AWS Spot bids for accurate allocation?
Modify [spotCPU]( and [spotRAM]( in default.json to the price of your bid. Allocation will use these bid prices, but it does not take into account what you are actually charged by AWS. Alternatively, you can provide an AWS key to allow access to the Spot data feed. This will provide accurate Spot prices.
#### Do I need a GCP billing API key?
We supply a global key with a low limit for evaluation, but you will want to supply your own before moving to production.
Please reach out with any additional questions on [Slack]( or via email at [](
package main
import (
func Healthz(w http.ResponseWriter, _ *http.Request, _ httprouter.Params) {
w.Header().Set("Content-Length", "0")
w.Header().Set("Content-Type", "text/plain")
func main() {
rootMux := http.NewServeMux()
costmodel.Router.GET("/healthz", Healthz)
rootMux.Handle("/", costmodel.Router)
rootMux.Handle("/metrics", promhttp.Handler())
klog.Fatal(http.ListenAndServe(":9003", errors.PanicHandlerMiddleware(rootMux)))
"provider": "custom",
"description": "Default prices used to compute allocation between RAM and CPU. AWS pricing API data still used for total node cost.",
"CPU": "0.031611",
"spotCPU": "0.006655",
"RAM": "0.004237",
"GPU": "0.95",
"spotRAM": "0.000892",
"storage": "0.00005479452",
"zoneNetworkEgress": "0.01",
"regionNetworkEgress": "0.01",
"internetNetworkEgress": "0.143",
"spotLabel": "",
"spotLabelValue": "spotinstance-nodes",
"awsServiceKeyName": "AKIAXW6UVLRRY5RQGGUX",
"awsServiceKeySecret": "",
"awsSpotDataBucket": "kc-test-spot",
"awsSpotDataPrefix": "spotdata",
"athenaBucketName": "s3://aws-athena-query-results-530337586275-us-east-1",
"athenaRegion": "us-east-1",
"athenaDatabase": "athenacurcfn_athena_test",
"athenaTable": "athena_test",
"projectID": "530337586275"
"provider": "Azure",
"description": "Azure estimates based on April 2019 advertised prices",
"CPU": "0.03900",
"spotCPU": "0.007764",
"RAM": "0.001917",
"spotRAM": "0.000382",
"storage": "0.00005479452" ,
"zoneNetworkEgress": "0.01",
"regionNetworkEgress": "0.01",
"internetNetworkEgress": "0.0725",
"azureSubscriptionID": "",
"azureClientID": "" ,
"azureClientSecret": "" ,
"azureTenantID": ""
"provider": "custom",
"description": "Default prices based on GCP us-central1",
"CPU": "0.031611",
"spotCPU": "0.006655",
"RAM": "0.004237",
"spotRAM": "0.000892",
"GPU": "0.95",
"storage": "0.00005479452",
"zoneNetworkEgress": "0.01",
"regionNetworkEgress": "0.01",
"internetNetworkEgress": "0.12"
\ No newline at end of file
"provider": "custom",
"description": "Default prices used to compute allocation between RAM and CPU. AWS pricing API data still used for total node cost.",
"CPU": "0.031611",
"spotCPU": "0.006655",
"RAM": "0.004237",
"spotRAM": "0.000892",
"projectID": "guestbook-227502",
"storage": "0.00005479452",
"zoneNetworkEgress": "0.01",
"regionNetworkEgress": "0.01",
"internetNetworkEgress": "0.12",
"billingDataDataset": "billing_data.gcp_billing_export_v1_01AC9F_74CF1D_5565A2"
2019-04-17 23:34:22 UTC,gke-standard-cluster-1-pool-1-91dc432d-cg69,node,,,0.1337,
## Deploying as a pod
See this page for all [Kubecost install options](
If you would like to deploy the cost model (w/o dashboards) directly a pod on your cluster, complete the steps listed below.
1. Set [this environment variable]( to the address of your prometheus server
2. `kubectl create namespace cost-model`
3. `kubectl apply -f kubernetes/ --namespace cost-model`
4. `kubectl port-forward --namespace cost-model service/cost-model 9003`
To test that the server is running, you can hit [http://localhost:9003/costDataModel?timeWindow=1d](http://localhost:9003/costDataModel?timeWindow=1d)
**Note:** This approach provides less functionality compared to other install options referenced above. Also, Prometheus and kube-state-metrics are external dependencies for this installation path.
replace => v0.0.0-20180702182130-06c8688daad7
require ( v0.34.0 v0.5.0 // indirect v24.1.0+incompatible v11.3.2+incompatible v1.28.9 v1.1.0 // indirect v1.3.3 v0.6.1 v2.1.0+incompatible // indirect v1.1.1 v2.0.2+incompatible // indirect v0.2.0 // indirect v1.2.1 v1.2.0 v1.2.0 v2.1.0+incompatible v1.0.0 v0.0.0-20190129233127-fd36f4220a90 v1.2.0 // indirect v0.0.0-20180709203117-cd690d0c9e24 // indirect v1.3.3 // indirect v0.0.0-20190604053449-0f29369cfe45 v0.0.0-20190423024810-112230192c58 v0.4.0 v0.0.0-20190913080256-21721929cffa v0.0.0-20190913075812-e119e5e154b6 v0.0.0-20190620085101-78d2af792bab v0.4.0 v1.1.0
go 1.13
This diff is collapsed.
kind: ClusterRoleBinding
name: cost-model
kind: ClusterRole