BigQuery 筆記1 – 如何使用 bq 指令工具設定資料集(表)的到期時間

筆記：如何使用 Google Cloud BiqQuery 的指令用法

一、 設定資料集的效期

語法:
bq update --default_table_expiration ttl_seconds project_id:dataset

說明:

• ttl_seconds 最小可用值為 3600 秒 (即最短效期為1小時, 參閱範例 1)若要取消資料集預設效期僅需將 ttl_seconds 設定為0 即可 (範例 2)
• default table expiration 設定僅適用於新建立的資料表，已存在的資料表會保持原設定狀態。若要修改已存資料表效期，請參閱＂二、設定資料表的效期＂

範例 1: set expiration 1 hours later to myDataset

    bq update --default_table_expiration 3600 myProject:myDataset

範例 2: disable expiration to myDataset

    bq update --default_table_expiration 0 myProject:myDataset

二、設定資料表的效期

語法:
    bq update --expiration=SECONS PROJECT:DATASET.TABLE

範例 3: set expiration 2 hours later to myTable

    bq update --expiration 7200 myProject:myDataset.myTable

範例 4: disable expiration to myTable

    bq update --expiration 0 myProject:myDataset.myTable

參考資料

• bq command-line tool reference

DevOps Tip - Got error - not found aws-iam-authenticator for kubectl using?

Problem

I've got error to run kubectl command with aws-iam-authenticator not found

(venv) ➜  k8s git:(develop) ✗ kubectl get pod
Unable to connect to the server: getting credentials: exec: exec: "aws-iam-authenticator": executable file not found in $PATH

Solutions

Installation tool on OSX Mojave

brew install aws-iam-authenticator

Example of installation

 (venv) ➜  k8s git:(develop) ✗ brew install aws-iam-authenticator
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/core, homebrew/cask and caskroom/cask).
==> New Formulae
..
==> Downloading https://homebrew.bintray.com/bottles/

aws-iam-authenticator-0.4.0.mojave.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/52...
######################################################################## 100.0%
==> Pouring aws-iam-authenticator-0.4.0.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/aws-iam-authenticator/0.4.0: 5 files, 31.1MB

References

• Amazon EKS Guide - Installing aws-iam-authenticator

Extended Reading

• DevOps Tip - How to use shared terraform plugins by -plugin-dir argument?
• DevOps Tip - How to use terraform apply without prompt and input yes for CI/CD?

DevOps Tip - How to use terraform apply without prompt and input yes for CI/CD?

Problem

Took a memo about integration of CI/CD applied by terraform without prompt to input 'yes'
It is annoying to input yes every time if you already make sure already by `terraform plan`.
On the other hand, I need to integrate with different system for CI/CD. I need a way to approve it automatically.
By tests with commands official documents, I found the correct command and arguments.

Tested Solutions

    terraform apply -input=false -auto-approve

or as my test, you can shorten to the following command

    terraform apply -auto-approve

Test Environments

• Ubuntu 16.04.04 LTS
• OSX 10.14

Reference:

• https://learn.hashicorp.com/terraform/development/running-terraform-in-automation

Extended Reading

• DevOps Tip - How to use shared terraform plugins by -plugin-dir argument?

DevOps Tip - How to use shared terraform plugins by -plugin-dir argument?

Problem

If you executed `terraform init`, you may already have the .terraform in your folder.

Because you may have many terraform repo similar to another, you can use the -plugin-dir to reduce the usage of disk space.

As a shared plugins so that we don’t need many copies of plugins into different locations.

Example

• Here I used my OSX Macbook as a Demo Environment.

  ====================
  cd /build/setupdev/

    sudo mkdir -p  /opt/terraform/terraform.d

mv .terraform/plugins to /opt/terraform/terraform.d/plugins

• Please run the following command to apply it.

  ====================

terraform init -input=false \
-plugin-dir=/opt/terraform/terraform.d/plugins/darwin_amd64

Tree of .terraform

Tree of terraform plugins

Reference

• https://learn.hashicorp.com/terraform/development/running-terraform-in-automation

Extended Reading

• DevOps Tip - How to use terraform apply without prompt and input yes for CI/CD?