You’ll often see this priority order mentioned:

But what do these actually mean?
Why does Azure still support secrets if they’re considered bad?

This article explains each method clearly and practically, so anyone can understand and choose the right one.

Why Authentication Matters in Azure

Authentication answers one simple question:

“Who are you, and are you allowed to access this resource?”

In Azure, authentication is commonly used when:

• An app accesses Key Vault

• A Function App accesses Storage

• A CI/CD pipeline deploys infrastructure

• A Kubernetes workload calls an API

Azure supports multiple authentication methods because not all systems are equal — some are modern, some are legacy.

🟢 1. Managed Identity (Strongly Recommended)

What is Managed Identity?

Managed Identity is an identity automatically managed by Azure for Azure resources.

No secrets.
No certificates.
No credentials to store.

Azure handles everything.

How it works (simple)

1. You enable Managed Identity on an Azure resource (VM, Function App, App Service)

2. The resource asks Azure for a token

3. Azure verifies the resource and issues a token

4. The resource accesses another Azure service

Key benefits

• No credentials to leak

• Automatic rotation

• Zero configuration secrets

• Deep Azure integration

When to use it

✅ Azure Function → Key Vault
✅ VM → Storage Account
✅ App Service → SQL Database

Limitation

❌ Works only inside Azure

If both the caller and target are in Azure, Managed Identity should be your first choice.

🟢 2. Federated Identity (Modern & Preferred)

What is Federated Identity?

Federated Identity allows Azure to trust an external identity provider using OIDC (OpenID Connect).

Still no secrets.

How it works

1. An external platform (GitHub Actions, AKS, Azure DevOps) authenticates itself

2. It receives a short-lived OIDC token

3. Azure validates the token (issuer, subject, audience)

4. Azure issues an access token

Common use cases

• GitHub Actions deploying to Azure

• Kubernetes workloads accessing Azure services

• Cross-cloud automation

Key benefits

• Secretless authentication

• Short-lived tokens

• Ideal for CI/CD and automation

Limitation

❌ Requires an OIDC-capable platform

If your workload runs outside Azure but supports OIDC, use Federated Identity.

🟡 3. Certificate-Based Authentication (Acceptable)

What is Certificate Authentication?

Instead of a password (secret), the app authenticates using a private key and certificate.

This is still OAuth, but stronger than secrets.

How it works

1. A certificate is uploaded to an Azure App Registration

2. The app signs a request using the private key

3. Azure verifies the certificate

4. Azure issues a token

Why it still exists

• Many enterprise systems can’t use OIDC

• Certificates are more secure than secrets

• Works well for long-running services

Downsides

• Certificates expire

• Manual lifecycle management

• Private key must be protected

Certificates are a solid fallback when federation is not possible.

🔴 4. Client Secrets (Legacy / Fallback)

What is a Client Secret?

A client secret is essentially a password for an application.

How it works

The app sends:

• client_id

• client_secret

• tenant_id

Azure validates the secret and issues a token.

Why secrets are bad

• Easy to leak

• Often hard-coded

• Require manual rotation

• High breach risk

Why Azure still supports them

• Legacy systems

• Backward compatibility

• Simple demos and temporary setups

Secrets exist because the real world still has old systems — not because they’re recommended.

Security Comparison at a Glance

How to Choose the Right Method

Ask yourself two questions:

1. Is my workload running inside Azure?

• Yes → Use Managed Identity

• No → Go to question 2

2. Does it support OIDC?

• Yes → Use Federated Identity

• No → Use Certificate

• Only legacy available → Use Client Secret (temporary)

Azure’s Real Direction

Azure’s message is clear:

Secrets are supported, but discouraged

You can see this in:

• Short secret expiry times

• Security warnings

• Defender recommendations

• Strong push toward Managed & Federated Identity

Final Thoughts

If you remember just one thing, remember this:

The closer the identity is to the platform, the safer it is

• Azure-native → Managed Identity

• Modern external → Federated Identity

• Enterprise legacy → Certificates

• Last resort → Client Secrets

If you’re migrating from client secrets to federated identity or want help applying this to Terraform, GitHub Actions, AKS, or Azure Functions, feel free to reach out or comment.

Happy building 🚀

But what if your application has special placement requirements that the default scheduler cannot handle?

👉 That’s where custom schedulers come in. Kubernetes allows you to run multiple schedulers within the same cluster and choose which scheduler should manage which pods.

In this blog, we’ll cover:

• Why multiple schedulers are useful

• How schedulers are named and configured

• Deploying custom schedulers (binary, pod, and deployment methods)

• The leader election option for HA setups

• How to use your custom scheduler in pods

• How to verify scheduling decisions

🔹 Why Multiple Schedulers?

By default, every pod goes through default-scheduler. However:

• You may want an application to run only on GPU nodes with additional custom checks.

• You may implement a domain-specific algorithm for data locality or cost optimization.

• You may test experimental scheduling strategies without impacting the default scheduler.

With multiple schedulers:

• Normal workloads → use the default scheduler.

• Special workloads → use your custom scheduler.

🔹 Scheduler Names

Each scheduler must have a unique name.

• Default scheduler → default-scheduler

• Custom schedulers → you define names like my-scheduler, gpu-scheduler, etc.

This name is set in the scheduler configuration file:

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: my-scheduler
    leaderElection:
      leaderElect: false

If no name is specified, Kubernetes defaults to default-scheduler.

🔹 Methods to Deploy a Custom Scheduler

1. Running Scheduler as a Binary

You can download the kube-scheduler binary and run it manually:

kube-scheduler \
  --config=/etc/kubernetes/my-scheduler-config.yaml \
  --kubeconfig=/etc/kubernetes/scheduler.kubeconfig

⚠️ Rarely used in modern kubeadm-based clusters since schedulers usually run as pods.

2. Scheduler as a Pod

You can run the scheduler as a pod inside your cluster.

apiVersion: v1
kind: Pod
metadata:
  name: my-scheduler
  namespace: kube-system
spec:
  containers:
    - name: kube-scheduler
      image: k8s.gcr.io/kube-scheduler:v1.28.0
      command:
        - kube-scheduler
        - --config=/etc/kubernetes/my-scheduler-config.yaml
      volumeMounts:
        - name: config
          mountPath: /etc/kubernetes/
  volumes:
    - name: config
      configMap:
        name: my-scheduler-config

Here, the ConfigMap contains your custom scheduler config.

3. Scheduler as a Deployment

A more scalable and recommended approach is to run the scheduler as a deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-scheduler
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      component: my-scheduler
  template:
    metadata:
      labels:
        component: my-scheduler
    spec:
      serviceAccountName: scheduler-sa
      containers:
        - name: kube-scheduler
          image: k8s.gcr.io/kube-scheduler:v1.28.0
          command:
            - kube-scheduler
            - --config=/etc/kubernetes/my-scheduler-config.yaml
          volumeMounts:
            - name: config
              mountPath: /etc/kubernetes/
      volumes:
        - name: config
          configMap:
            name: my-scheduler-config

This way, Kubernetes ensures HA and restart capabilities.

🔹 The leaderElect Option Explained

When running schedulers in HA setups (multiple control-plane nodes):

• Multiple copies of the same scheduler might be running.

• Only one scheduler instance should be active at a time.

• The leaderElect: true setting ensures that a leader is elected among them.

Example:

leaderElection:
  leaderElect: true
  resourceName: my-scheduler-lock

Here:

• If you run 3 replicas of my-scheduler, only one will be leader.

• Others will stay passive until the leader fails.

👉 Always enable leaderElect in production HA clusters.

🔹 Using the Custom Scheduler in a Pod

Once deployed, tell Kubernetes which scheduler to use by adding schedulerName to your pod spec.

apiVersion: v1
kind: Pod
metadata:
  name: my-custom-pod
spec:
  schedulerName: my-scheduler
  containers:
    - name: busybox
      image: busybox
      command: ["sleep", "3600"]

Now, this pod will bypass the default scheduler and use my-scheduler.

🔹 Verifying Which Scheduler Picked the Pod

To confirm scheduling:

1. Check pod events:

kubectl get events --sort-by=.metadata.creationTimestamp -o wide

You’ll see events like:

Successfully assigned default/my-custom-pod to node1
Source: my-scheduler

2. Check scheduler logs:

kubectl logs -n kube-system <my-scheduler-pod-name>

If your pod stays in Pending, likely the scheduler is misconfigured.

✅ Summary

• Kubernetes supports multiple schedulers.

• Each scheduler must have a unique name.

• You can deploy schedulers as a binary, pod, or deployment.

• Use leaderElect: true in HA setups.

• Pods can be scheduled by a custom scheduler using schedulerName.

• Verify scheduling decisions with events and logs.

By using multiple schedulers, you can extend Kubernetes to meet specialized workload placement needs while keeping the default scheduler for general workloads.

What Are Static Pods?

Normally, the kubelet relies on the kube-apiserver to get instructions about which pods to run on a node. The API server receives scheduling decisions from the kube-scheduler, and stores the cluster state in ETCD.

But what happens if there’s no API server, no scheduler, no controllers, and no ETCD—in other words, no Kubernetes master at all?

Even in this scenario, the kubelet can still manage pods independently. These pods, created and managed by the kubelet without any control plane components, are called Static Pods.

How Static Pods Work

• The kubelet periodically checks a designated directory on the node for pod definition files (manifests).

• When it finds a manifest, it creates the pod on the node.

• If the pod crashes, the kubelet automatically restarts it.

• If the manifest file changes, the kubelet recreates the pod to apply updates.

• If the manifest is removed, the pod is automatically deleted.

Important: Static pods only exist at the pod level. You cannot create ReplicaSets, Deployments, or Services using static pod manifests. These objects require other control plane components.

Where Are Static Pods Stored?

The kubelet needs to know the path to the directory containing static pod manifests. There are two ways to locate this:

1. Check in the kubelet.service file

The static pod path may be specified directly as the --pod-manifest-path option. To check:

systemctl cat kubelet | grep pod-manifest-path

Example output:

--pod-manifest-path=/etc/kubernetes/manifests

This directory is where you place your pod manifest files.

2. Check in the kubelet config file (config.yaml or kubeconfig.yaml)

Sometimes the kubelet is configured to use a config file via the --config option. In this file, the path is specified as:

staticPodPath: "/etc/kubernetes/manifests"

• Either method will give you the correct directory for placing static pod manifests.

• If both exist, the kubelet config file usually takes precedence.

Viewing Static Pods

Once static pods are created, you cannot use kubectl to view them if the kube-apiserver is not running. This is because:

• kubectl communicates with the kube-apiserver.

• If the cluster has no API server (e.g., in a standalone kubelet scenario), kubectl has no source of truth.

Instead, use Docker (or the container runtime directly) to inspect the running static pods:

docker ps

If the node is part of a cluster with a running kube-apiserver, static pods are mirrored as read-only pods, and you can view them via:

kubectl get pods -n kube-system

You cannot edit or delete static pods via kubectl; changes must be made in the manifest files.

Static Pods vs DaemonSets

Both static pods and DaemonSet pods are ignored by the kube-scheduler.

Use Cases for Static Pods

Static pods are ideal for:

• Bootstrapping the Kubernetes control plane itself as pods (e.g., API server, Controller Manager, ETCD).

• Running critical node-level services that must exist independently of the control plane.

When using tools like kubeadm, static pods are used to deploy the cluster's control plane. The kubelet monitors these pods, automatically restarting them if they crash.

Summary

• Static Pods are created and managed directly by the kubelet.

• They are independent of the Kubernetes control plane.

• You can find the static pod manifest directory in either the kubelet.service file (--pod-manifest-path) or the kubelet config file (staticPodPath).

• Use Docker to inspect static pods if no API server is present.

• Use static pods to deploy the Kubernetes control plane itself or critical node-level services.

Static pods provide a simple, reliable way to run essential pods even when the full Kubernetes control plane is not available.

Let’s break it down.

What is a DaemonSet?

A DaemonSet is a Kubernetes object that ensures exactly one copy of a pod runs on every node in your cluster.

• When a new node joins the cluster, the DaemonSet automatically deploys a pod on it.

• When a node is removed, the pod running on it is also deleted.

Think of it as a “one pod per node manager.”

Key difference from ReplicaSet:

• ReplicaSets focus on running a specified number of pod replicas across the cluster.

• DaemonSets focus on ensuring every node has one copy of a pod.

Why Use a DaemonSet?

DaemonSets are perfect for workloads that need to run on every node, such as:

1. Monitoring agents

   • Example: You want to deploy a pod that collects logs or metrics from every node.

   • DaemonSet ensures each node automatically gets the monitoring pod.

2. Cluster networking components

   • Example: Solutions like Weave Net require an agent pod on every node to manage network traffic.

3. Node-level system components

   • Example: The kube-proxy component, which handles network rules, can run as a DaemonSet on all nodes.

✅ Tip: DaemonSets save you the trouble of manually adding or removing pods as nodes are added or removed.

How Does a DaemonSet Work?

Before Kubernetes v1.12

• Pods were scheduled manually on nodes by setting the nodeName property in the pod specification.

• Each pod was “pinned” to a specific node.

From Kubernetes v1.12 Onwards

• DaemonSets use the default scheduler along with node affinity rules.

• The scheduler automatically decides which pod goes to which node.

• You no longer need to manually specify nodes; Kubernetes takes care of it.

Creating a DaemonSet

Creating a DaemonSet is very similar to creating a ReplicaSet. The main difference is the kind:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: monitoring-daemon
spec:
  selector:
    matchLabels:
      app: monitoring
  template:
    metadata:
      labels:
        app: monitoring
    spec:
      containers:
      - name: monitoring-agent
        image: monitoring-agent:lates

Steps to create and manage DaemonSets:

1. Create the DaemonSet:

kubectl create -f monitoring-daemon.yaml

2. View all DaemonSets:

kubectl get daemonset

3. View detailed info about a DaemonSet:

kubectl describe daemonset monitoring-daemon

⚠️ Tip: Ensure the labels in the selector match the labels in the pod template. Otherwise, the DaemonSet won’t manage the pods properly.

Summary: Why DaemonSets Matter

• Automatic deployment: Ensures one pod per node without manual intervention.

• Perfect for node-level tasks: Monitoring, logging, networking, or system agents.

• Integrates with Kubernetes scheduler: Modern DaemonSets use affinity rules to schedule pods efficiently.

Key Takeaways for Learners

1. DaemonSets = One pod per node

2. Use cases: kube-proxy, monitoring agents, network agents

3. Modern scheduling: Uses default scheduler + node affinity

4. Management commands: kubectl create, kubectl get daemonset, kubectl describe daemonset

DaemonSets may seem like just another Kubernetes object, but they are crucial for maintaining cluster-wide consistency for essential services. Understanding how to deploy and manage them is a key step in mastering Kubernetes.

This is where requests and limits come in. Let’s break it down step by step.

🟢 What Are Requests and Limits?

Resource Requests

• The minimum amount of CPU and memory a container is guaranteed.

• The scheduler uses these values to decide on which node to place the pod.

• Example:

    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
  

  ➝ Pod gets at least 0.5 CPU and 512Mi RAM.

Resource Limits

• The maximum resources a container can consume.

• Prevents one pod from hogging all resources.

• Example:

    resources:
      limits:
        cpu: "1"
        memory: "1Gi"
  

  ➝ Pod can’t use more than 1 CPU and 1Gi RAM.

🖥️ CPU vs 💾 Memory

CPU

• 1 CPU = 1 vCPU (AWS), 1 core (Azure/GCP), or 1 hyperthread.

• Can specify fractions:

  • 100m = 0.1 CPU.

• If pod exceeds its CPU limit, it gets throttled (slowed down).

Memory

• Units: Mi (Mebibytes), Gi (Gibibytes).

• If pod exceeds its memory limit, it is killed (OOMKilled).

• Memory cannot be throttled like CPU.

⚖️ Scenarios to Know

1. No requests, no limits

   • Pod can take everything → others may starve. ❌ Not safe.

2. Limits only (no requests)

   • Kubernetes treats request = limit.

   • Pod is guaranteed exactly that much.

   • Safe, but not flexible.

3. Requests + Limits set

   • Pod always gets its request.

   • Can burst up to the limit.

   • Balanced, but unused limits may waste resources.

4. Requests only (no limits) ✅

   • Pod guaranteed its request.

   • Can use more if node has free capacity.

   • Best for most cases, but requires discipline (all pods should set requests).

⚖️ Memory Scenarios to Know

1. No requests, no limits

• Pod can take all memory on the node if available → may cause other pods or system processes to OOM. ❌ Not safe.

2. Limits only (no requests)

• Kubernetes treats request = limit.

• Pod is guaranteed exactly that much memory.

• Safe, but can waste memory if pod doesn’t need the full limit.

• If pod exceeds limit → killed (OOMKilled).

3. Requests + Limits set

• Pod guaranteed its request.

• Can use memory up to the limit.

• Balanced, but if pod uses more than limit → killed.

• Best practice for workloads with predictable memory usage.

4. Requests only (no limits) ✅

• Pod guaranteed its request.

• Can use more memory if node has free capacity, but if it grows too much → node might run out of memory, and pod or others may be OOMKilled.

• Safer than no requests, but requires careful monitoring and discipline.

✅ CPU

• Pod is guaranteed its request (minimum CPU it needs).

• Can use more CPU if node has spare capacity.

• CPU is throttled, so even if a pod uses more, it won’t crash others.

• Pros: Flexible, efficient, lets pods burst when resources are available.

• Cons: If a pod consumes too much CPU, it may slow down other pods sharing the same node.

⚠️ Memory

• Pod is guaranteed its request (minimum memory).

• Can use more memory if node has free capacity, but no limit means it can potentially use all memory on the node.

• Memory cannot be throttled, so if it grows too much → pod or other pods may be OOMKilled.

• Pros: Flexible if memory usage is predictable and you trust all pods to behave.

• Cons: Risky if some pods may have memory leaks or high spikes.

💡 Best Practice

1. Always set requests — ensures pods get guaranteed resources.

2. Set limits for memory if workload can spike — prevents a single pod from crashing the node.

3. CPU limits are optional — only needed if you want to prevent noisy neighbors or enforce strict resource isolation.

In short:

• ✅ CPU: requests only is safe and flexible. ⚠️ Set limits only when necessary → e.g., in multi-tenant clusters or public labs.

• ⚠️ Memory: requests only is flexible but can be risky → consider setting a reasonable limit.

• ✅ Use LimitRange at the namespace level to enforce defaults:

apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-mem-defaults
  namespace: dev
spec:
  limits:
  - default:
      cpu: "1"
      memory: "512Mi"
    defaultRequest:
      cpu: "0.5"
      memory: "256Mi"
    type: Container

This ensures every pod has a baseline, even if developers forget to specify resources.

📊 Visual Flow: How Scheduling Works

1. Pod created → Scheduler checks requests.

2. Scheduler finds a node with enough free resources.

3. Pod is scheduled onto that node.

4. At runtime:

   • If pod exceeds CPU limit → throttled.

   • If pod exceeds memory limit → killed.

   • If pod stays within requests → always guaranteed that much.

🚀 Conclusion

• Requests = Minimum guarantee

• Limits = Maximum cap

• Always set requests to prevent resource starvation.

• Use limits carefully, only when you need strict isolation.

With the right balance, you’ll keep your Kubernetes cluster fair, efficient, and stable.

LimitRange & ResourceQuota

LimitRange (CPU & Memory)

A LimitRange is a namespace-level policy that defines default resource requests and limits for pods and containers if they are not explicitly set. It ensures that no pod in the namespace runs without some resource constraints.

Key points:

• Applies at namespace level.

• Can define default requests and limits for CPU and memory.

• Can define minimum and maximum allowed values for requests and limits.

• Only affects new pods created after the LimitRange is applied. Existing pods are not affected.

Example:

apiVersion: v1
kind: LimitRange
metadata:
  name: example-limitrange
  namespace: my-namespace
spec:
  limits:
  - type: Container
    default:
      cpu: 500m
      memory: 512Mi
    defaultRequest:
      cpu: 250m
      memory: 256Mi
    max:
      cpu: 1
      memory: 1Gi
    min:
      cpu: 100m
      memory: 128Mi

• defaultRequest → scheduler guaranteed resources if pod doesn’t specify.

• default → runtime limit if pod doesn’t specify.

• max → maximum allowed resource (ceiling).

• min → minimum allowed resource (floor).

Explanation:

• If a pod does not specify requests/limits, it gets defaultRequest and default.

• Pods cannot request more than max or less than min.

✅ Ensures fair resource usage and prevents runaway pods.

ResourceQuota

A ResourceQuota is a namespace-level limit on the total resources that all pods/containers together can consume.

Key points:

• Limits the sum of resources used by all pods in the namespace.

• Can limit CPU, memory, number of pods, services, persistent volumes, etc.

• Prevents a single namespace from consuming all cluster resources.

Example:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: example-quota
  namespace: my-namespace
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 4Gi
    limits.cpu: "10"
    limits.memory: 10Gi
    pods: "10"

Explanation:

• The sum of CPU requests across all pods ≤ 4 CPU.

• The sum of memory requests across all pods ≤ 4 GiB.

• The sum of limits across all pods ≤ 10 CPU and 10 GiB memory.

• Max 10 pods in this namespace.

✅ Ensures overall resource governance at the namespace level.

⚖️ Summary

💡 Tip:

• Use LimitRange to ensure each pod has same defaults and prevents very small/huge pods.

• Use ResourceQuota to prevent a namespace from consuming all cluster resources.

Kubernetes provides multiple mechanisms to control Pod placement, from direct assignment to advanced affinity rules and taints. In this guide, we’ll cover everything you need to understand scheduling and how to dedicate nodes for specific Pods.

1. nodeName (Direct Scheduling)

The simplest way to assign a Pod to a node is using the nodeName field.

• If nodeName is set: Kubernetes bypasses the scheduler and assigns the Pod directly.

• If nodeName is not set: The Pod goes to the scheduler for placement.

Example:

apiVersion: v1
kind: Pod
metadata:
  name: fixed-node-pod
spec:
  nodeName: node01
  containers:
  - name: nginx
    image: nginx

⚠️ If node01 is unavailable, the Pod stays in Pending state.

Limitations of nodeName

• Hard binding: Pod is tied to a single node.

• No flexibility: If node01 is down or out of resources, Pod stays Pending.

• Not scalable: In dynamic clusters (nodes added/removed), nodeName creates operational headaches.

⚠️ Because of these drawbacks, nodeName is rarely used in production (only for testing/debugging).
Instead, we use labels + selectors for flexible scheduling.

2. Scheduler & Binding

When nodeName is not set, the Kubernetes scheduler:

1. Checks available nodes.

2. Considers resources, labels, taints, and affinity rules.

3. Creates a Binding object linking the Pod to a node.

You can inspect the binding:

kubectl get pod fixed-node-pod -o yaml

The spec.nodeName field will appear after scheduling, even if not set manually.

3. Labels & Selectors

• Labels: Key-value metadata attached to Kubernetes objects (Pods, Nodes).

• Selectors: Queries that match labels.

Used by the scheduler to decide Pod placement.

Example: Node Label

kubectl label nodes node01 disktype=ssd

Example: Pod with nodeSelector

spec:
  nodeSelector:
    disktype: ssd
  containers:
  - name: nginx
    image: nginx

Pod will schedule only on nodes labeled disktype=ssd.

4. Annotations

• Key-value metadata that do not affect scheduling.

• Useful for storing additional information like URLs, team info, or build data.

Example:

metadata:
  annotations:
    team: devops
    description: "This Pod is for testing purposes"

5. nodeSelector (Scheduling with Labels)

nodeSelector is the simplest way to schedule Pods based on node labels.

• You label nodes with key-value pairs.

• Pods with nodeSelector can run only on nodes with matching labels.

Example: Label a node

kubectl label nodes node01 disktype=ssd

Example: Pod with nodeSelector

apiVersion: v1
kind: Pod
metadata:
  name: ssd-pod
spec:
  nodeSelector:
    disktype: ssd
  containers:
  - name: nginx
    image: nginx

✅ This Pod will only run on nodes labeled disktype=ssd.

Limitations of nodeSelector

• Only supports exact matches (key=value).

• Cannot use advanced operators (In, NotIn, Exists).

• No way to express preferences (e.g., “prefer SSD nodes but allow others”).

• Too basic for complex production use cases.

👉 That’s why Kubernetes introduced Node Affinity, which builds on nodeSelector and adds more powerful scheduling rules.

6. Node Affinity & Anti-Affinity

Node Affinity is an advanced version of Node Selector. It allows:

• Logical operators (In, NotIn, Exists)

• Soft (preferred) or hard (required) scheduling rules

• Anti-affinity to avoid scheduling Pods on nodes already running certain workloads

Node Affinity Rules

Example: RequiredDuringSchedulingIgnoredDuringExecution

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: disktype
          operator: In
          values:
          - ssd

Pod must schedule on nodes with disktype=ssd.

Example: PreferredDuringSchedulingIgnoredDuringExecution

affinity:
  nodeAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 1
      preference:
        matchExpressions:
        - key: disktype
          operator: In
          values:
          - ssd

Pod prefers SSD nodes but can schedule elsewhere if no SSD node is available.

Example: RequiredDuringSchedulingRequiredDuringExecution

• Hard rule: Pod must be on a matching node.

• Evicted if node stops matching.

affinity:
  nodeAffinity:
    requiredDuringSchedulingRequiredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: dedicated
          operator: In
          values:
          - true

Example: PreferredDuringSchedulingRequiredDuringExecution

• Soft rule during scheduling, but Pod is evicted if the node stops matching later.

affinity:
  nodeAffinity:
    preferredDuringSchedulingRequiredDuringExecution:
    - weight: 1
      preference:
        matchExpressions:
        - key: dedicated
          operator: In
          values:
          - true

Pod Anti-Affinity

Avoid scheduling Pods on nodes already running certain Pods.

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: app
          operator: In
          values:
          - frontend
      topologyKey: "kubernetes.io/hostname"

Ensures Pods are spread across nodes, avoiding co-location.

7. Taints & Tolerations

• Taints on nodes restrict which Pods can run there.

• Tolerations on Pods allow them to ignore taints.

Taint Effects

Example:

kubectl taint nodes node01 dedicated=teamA:NoSchedule

Pod toleration:

tolerations:
- key: "dedicated"
  operator: "Equal"
  value: "teamA"
  effect: "NoSchedule"

Alone, this does not dedicate the node — Pod can still schedule on untainted nodes.

8. Dedicating Nodes: Combining Node Affinity + Taints

Sometimes in Kubernetes, you want to reserve certain nodes for specific workloads, for example:

• GPU nodes for ML workloads

• High-memory nodes for database Pods

• Critical services that should not compete with general workloads

Using only one mechanism (like taints or node affinity) is often not enough:

1. Taints & Tolerations alone

   • A taint marks a node so that only Pods with a matching toleration can be scheduled there.

   • Limitation: Pods with that toleration can still be scheduled on other untainted nodes.

• ✅ Protects nodes from unwanted Pods, but does not guarantee exclusivity.

2. Node Affinity alone

   • Node Affinity guides the scheduler to place Pods on nodes with matching labels.

   • Limitation: Other Pods without affinity rules can still be scheduled on those nodes.

     

   • ✅ Useful for preference or required placement, but cannot block others.

The Combined Approach (Recommended)

To truly dedicate a node exclusively:

1. Taint the node → blocks unwanted Pods.

2. Add Node Affinity to the Pod → ensures only the intended Pods are scheduled there.

This combination ensures:

• Only Pods with the correct toleration + affinity can be scheduled.

• All other Pods are prevented from using that node.

• Dedicated nodes are reserved for special workloads.

Step-by-Step Example

Step 1: Taint the Node

kubectl taint nodes node01 dedicated=true:NoSchedule

• Node node01 now rejects any Pod without the dedicated=true toleration.

Step 2: Pod YAML with Toleration + Node Affinity

apiVersion: v1
kind: Pod
metadata:
  name: dedicated-pod
spec:
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: dedicated
            operator: In
            values:
            - "true"
  containers:
  - name: nginx
    image: nginx

✅ Behavior:

• Pod can only schedule on node01.

• Other Pods cannot use node01.

• Node is effectively dedicated to this workload.

Why Use Both?

Use Cases

• GPU Nodes: Only ML workloads with GPU toleration + affinity can schedule there.

• High-Memory Nodes: Critical databases get dedicated memory nodes.

• Master/Control Nodes: Prevent regular workloads from running there while allowing special system Pods.

This approach is widely used in production clusters where resource isolation and workload predictability are important.

Summery

🧭 Kubernetes Pod Scheduling: From Basics to Dedicated Nodes

In Kubernetes, scheduling decides which node a Pod will run on. By default, the scheduler spreads Pods across the cluster. But in real-world scenarios, we often want more control — for example:

• Running database workloads only on SSD-backed nodes.

• Ensuring GPU workloads only land on GPU nodes.

• Keeping monitoring agents separate from application Pods.

Kubernetes gives us several tools to influence scheduling. Let’s walk through them step by step — starting from the simplest (nodeName) to the most powerful combination (Node Affinity + Taints).

1. 🔹 nodeName

The most direct scheduling method. You tell Kubernetes exactly which node to run on.

spec:
  nodeName: worker-1

✅ Advantages

• Simple and explicit — Pod always runs on the named node.

• Bypasses the scheduler (fast).

⚠️ Limitations

• Hard-coded — tied to a single node.

• No flexibility — if that node is down or full, Pod won’t run.

• Doesn’t scale in production.

👉 Useful only for debugging or extreme edge cases.

2. 🔹 nodeSelector

A more flexible alternative. Instead of hardcoding nodes, you use labels.

spec:
  nodeSelector:
    disktype: ssd

✅ Advantages

• Easy to target groups of nodes by label.

• Scheduler still chooses the best fit within that group.

• Simple to set up.

⚠️ Limitations

• Only supports exact match (=).

• No advanced expressions (OR, NOT, ranges).

• Still rigid for complex workloads.

👉 Better than nodeName, but limited for real-world scenarios.

3. 🔹 Node Affinity & Anti-Affinity

Node affinity is a more expressive version of nodeSelector. It allows rules with operators and conditions.

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: disktype
          operator: In
          values: ["ssd", "nvme"]

✅ Advantages

• Rich operators: In, NotIn, Exists, Gt, Lt.

• Supports hard rules (requiredDuringScheduling…) and soft preferences (preferredDuringScheduling…).

• Anti-affinity helps avoid placing Pods together (e.g., keep replicas apart).

⚠️ Limitations

• Only works at scheduling time → once scheduled, Pods don’t move if nodes change.

• Not exclusive → other Pods without affinity can still land on those nodes.

• No eviction → scheduler won’t reshuffle Pods later.

👉 Great for preferences and constraints, but not for isolation.

4. 🔹 Taints and Tolerations

While affinity attracts Pods to nodes, taints repel Pods unless they tolerate it.

kubectl taint nodes worker-2 dedicated=db:NoSchedule

Pod toleration:

tolerations:
- key: "dedicated"
  operator: "Equal"
  value: "db"
  effect: "NoSchedule"

✅ Advantages

• Protects reserved nodes from general workloads.

• Fine-grained control:

  • NoSchedule → don’t place Pods.

  • PreferNoSchedule → try to avoid.

  • NoExecute → evict existing Pods.

⚠️ Limitations

• Too permissive — any Pod with the same toleration can run there.

• No attraction mechanism — only prevents, doesn’t guide Pods.

• Risk of idle nodes if no Pods have tolerations.

• Team drift → multiple workloads might add tolerations and unintentionally overload special nodes.

👉 Great for blocking unwanted Pods, but not enough to guarantee dedicated use.

5. 🔹 Dedicating Nodes: Combining Node Affinity + Taints

The best practice for exclusive node usage is to combine both:

• Node Affinity → attracts the right Pods.

• Taints → repels everything else.

Example:

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: workload
          operator: In
          values: ["gpu"]
tolerations:
- key: "workload"
  operator: "Equal"
  value: "gpu"
  effect: "NoSchedule"

🎯 Result

• GPU workloads land only on GPU nodes.

• No other Pods can run there accidentally.

• Guarantees dedicated scheduling.

🚀 Scheduling Evolution Flow

1. nodeName → too rigid.

2. nodeSelector → basic labels, limited.

3. Node Affinity → flexible rules, but still shared.

4. Taints & Tolerations → node-level protection, but not exclusive.

5. Affinity + Taints → full control, dedicated nodes.

📝 Conclusion

Kubernetes scheduling evolved from hardcoding to flexible rules to node protection. Each method builds on the previous:

• Start with labels for organization (nodeSelector).

• Use affinity/anti-affinity for advanced rules.

• Apply taints/tolerations to protect nodes.

• Combine both for dedicated, isolated workloads.

👉 This layered approach gives you fine-grained control over Pod placement while keeping the cluster efficient and reliable.

💡 Pro tip: For production environments, always prefer Affinity + Taints when you want guaranteed workload isolation.

This guide explains what a Terraform state file is, why it matters, the challenges of local state management, how to use a remote backend like AWS S3, and how to implement state locking using DynamoDB - all tailored for beginners and students.

🌱 What is Terraform State?

Terraform keeps track of your real-world infrastructure using a file called terraform.tfstate.

Whenever you apply a Terraform configuration, Terraform:

1. Provisions the resources you defined (e.g., EC2 instances, VPCs, etc.)

2. Saves metadata about those resources in the state file.

This file is the single source of truth for Terraform. Without it, Terraform wouldn’t know what has already been created, updated, or deleted.

🧠 Why is Terraform State Important?

Imagine this: You create an EC2 instance using Terraform. Later, you want to add a tag. If Terraform doesn't know the instance already exists (because there’s no state file), it will try to create a new instance instead of updating the existing one. ❌

With the state file, Terraform can:

• Compare your current config with the real infrastructure.

• Determine what to add, change, or delete.

• Safely manage updates and ensure idempotency (the ability to run the same script multiple times without unintended changes).

⚠️ The Drawbacks of Local State Files

While the local state file works, it comes with serious issues:

1. Sensitive Data Exposure

The state file can include secrets, such as API keys, passwords, and sensitive resource details. If it's stored on a local machine or pushed to a Git repository, anyone with access can read them.

2. Team Collaboration Challenges

If multiple people are working on the same Terraform project:

• Everyone must remember to share or sync the updated state file.

• Forgetting to do so causes drift between actual infrastructure and Terraform’s understanding.

• Multiple versions of the state file can lead to resource conflicts or duplication.

☁️ Solution: Using Remote Backends

Terraform offers remote backends to solve these problems. One of the most popular backends is AWS S3.

🔧 How Does It Help?

• Your state file is stored in S3, not on your local machine.

• It’s always up-to-date since Terraform writes to S3 directly when changes are applied.

• You can restrict access to the S3 bucket using IAM policies.

✅ Setting Up S3 as a Remote Backend

1. Create an S3 bucket (manually or using Terraform):

resource "aws_s3_bucket" "tf_backend" {
  bucket = "my-unique-terraform-backend-bucket"
}

2. Configure your backend in backend.tf:

terraform {
  backend "s3" {
    bucket = "my-unique-terraform-backend-bucket"
    key    = "statefiles/terraform.tfstate"
    region = "us-east-1"
  }
}

3. Initialize the backend:

terraform init

Terraform will now store the state file in S3 instead of your local disk. 🎉

🔒 What About State Locking?

Here’s a new concern:

What if two people run terraform apply at the same time?

That could cause a race condition, where both changes conflict or overwrite each other.

🚪 Enter State Locking with DynamoDB

Terraform can lock the state file using AWS DynamoDB, ensuring that only one person can modify infrastructure at a time.

🔐 Setting Up DynamoDB for Locking

1. Create a DynamoDB Table:

resource "aws_dynamodb_table" "tf_locks" {
  name         = "terraform-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

2. Update your backend.tf to include locking configuration:

terraform {
  backend "s3" {
    bucket         = "my-unique-terraform-backend-bucket"
    key            = "statefiles/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"
  }
}

3. Re-initialize to apply backend changes:

terraform init

Now, Terraform uses the DynamoDB table to lock and unlock the state file safely. Only one terraform apply can run at a time.

🧪 Testing It Out

You can test your setup like this:

1. Write a simple EC2 instance configuration in main.tf.

2. Run:

terraform init
terraform apply

3. Observe that your state is no longer stored locally, but in S3.

4. Terraform now uses DynamoDB to lock the state during apply operations.

🚀 Summary

Here’s what we’ve covered:

• ✅ Terraform State tracks your infrastructure.

• ❌ Local state files pose risks: sensitive data exposure and sync issues.

• ☁️ Remote backends (like S3) solve these problems.

• 🔒 State locking with DynamoDB prevents conflicting changes.

This setup is production-grade and a must-know for anyone planning to work in a team or manage infrastructure securely using Terraform.

🧰 Bonus: Useful Terraform Commands

terraform init       # Initializes the backend and providers
terraform plan       # Shows what changes will be made
terraform apply      # Applies the changes
terraform show       # Displays the current state
terraform destroy    # Destroys the resources

Got questions or need help? Leave a comment or explore the official Terraform Backend Docs for more.

Happy coding! 🌍⚙️

If you’re a beginner, student, or new DevOps engineer, you’ve probably written Terraform code in one big file. But what happens when your infrastructure grows? What if your team wants to reuse the same code across environments or projects?

Welcome to the world of Terraform modules - a structured, maintainable, and reusable way to write your infrastructure as code.

Let’s explore this in-depth.

📦 What is a Terraform Module?

A module in Terraform is simply a folder that contains .tf files - your Terraform configurations - which can be reused across projects.

A module is to Terraform what a function is to programming.

Why Use Modules?

• ✅ Avoid code duplication

• ✅ Increase reusability across teams and environments

• ✅ Enable better testing and debugging

• ✅ Enforce standards across infrastructure

• ✅ Encourage team collaboration with ownership and modular thinking

🔧 Real-World Analogy

Imagine you're working at a company like amazon.com and the app is a monolithic Java codebase with 1 million+ lines of code. When there's a bug, it's hard to know who wrote what, where to fix it, and how to test it without deploying the entire app.

The fix? Microservices - smaller, decoupled, maintainable codebases.

Similarly in Terraform:

Without modules, your .tf file will grow with:

• EC2 Instances

• S3 Buckets

• VPCs

• Lambda Functions

• Load Balancers

• EKS clusters
  ...and more.

It becomes impossible to maintain or collaborate on.

That’s why we adopt a modular approach in Terraform - breaking things into small, logical, reusable units.

🛠 Creating a Basic Terraform Module (Step-by-Step)

Let’s build a reusable EC2 instance module.

📁 Folder Structure

terraform-project/
├── main.tf
├── modules/
│   └── ec2_instance/
│       ├── main.tf
│       ├── variables.tf
│       ├── outputs.tf

🔨 Step 1: Inside modules/ec2_instance/main.tf

resource "aws_instance" "example" {
  ami           = var.ami_value
  instance_type = var.instance_type_value
  subnet_id     = var.subnet_id_value
}

📥 Step 2: Define Variables in modules/ec2_instance/variables.tf

variable "ami_value" {
  description = "AMI ID for the EC2 instance"
}

variable "instance_type_value" {
  description = "EC2 instance type"
}

variable "subnet_id_value" {
  description = "Subnet ID for the instance"
}

📤 Step 3: Output Public IP in modules/ec2_instance/outputs.tf

output "public_ip" {
  value = aws_instance.example.public_ip
}

🧪 Step 4: Consume the Module in Root main.tf

provider "aws" {
  region = "us-east-1"
}

module "ec2_instance" {
  source              = "./modules/ec2_instance"
  ami_value           = "ami-0c55b159cbfafe1f0"
  instance_type_value = "t2.micro"
  subnet_id_value     = "subnet-0123456789abcdef0"
}

📌 How It Works

When you run:

terraform init
terraform apply

Terraform will:

• Load the ec2_instance module

• Inject the variables you passed

• Create the EC2 instance

• Output the public IP

🔁 Why This Is So Powerful

Imagine this scenario:

You have 3 dev teams, each needing EC2 instances with different configurations.

With modules:

• You reuse the same module

• Just pass different ami, instance_type, and subnet_id

• No duplicated code

Additional Benefits:

• 🧱 Modularity – Break infrastructure into building blocks

• 🔄 Reusability – Use the same logic across teams

• 🧪 Testability – Test modules in isolation

• 🔐 Security – Keep secrets in terraform.tfvars (not in code)

• 📚 Documentation & Ownership – Clear inputs and outputs make team collaboration easier

🔐 Optional: Use terraform.tfvars to Supply Values

Create a terraform.tfvars file in your root directory:

ami_value           = "ami-0c55b159cbfafe1f0"
instance_type_value = "t2.micro"
subnet_id_value     = "subnet-0123456789abcdef0"

Then just run:

terraform apply

Terraform will automatically pick up values from this file.

🗃️ Where Should You Store Modules?

• In the same repo (like above)

• In a separate GitHub repo (shared across projects)

• Use Terraform Registry (like DockerHub for modules)

❗ In production, avoid unknown public modules unless you trust the source.
Companies usually create private module registries in GitHub or Terraform Cloud.

🚀 Pro Tip: Modules Scale With Your Org

Let’s say your team writes modules for:

• ec2_instance

• s3_bucket

• eks_cluster

• vpc

• alb

Now, internal developers can simply consume those like building blocks - without writing full configurations.

module "s3_bucket" {
  source = "git::https://github.com/org/modules.git//s3"
  bucket_name = "my-app-logs"
}

✅ Summary

Terraform modules are the secret weapon for writing production-grade infrastructure:

🔚 Final Thoughts

If you’re serious about mastering Terraform, start thinking in modules today.

Instead of managing huge .tf files, break your infrastructure into clean, reusable, and testable units - just like developers do with microservices.

"Good DevOps is modular. Great DevOps is reusable."

This blog is especially for students, new learners, and junior DevOps engineers who want to build a solid foundation in Terraform - without getting overwhelmed.

🔌 What Is a Provider in Terraform?

A provider in Terraform is a plugin that connects Terraform to your cloud platform (like AWS, Azure, GCP, etc.).

Think of it as the bridge between your Terraform code and the cloud infrastructure you're trying to automate.

🔧 Example: AWS Provider

provider "aws" {
  region = "us-east-1"
}

This block tells Terraform:

• Use the AWS provider

• Target the us-east-1 region

Terraform will then use this configuration to authenticate and create resources in AWS.

🧠 Tip:

Without a provider block, Terraform won’t know:

• Which cloud to talk to

• How to authenticate

• Where to create infrastructure

☁️ Official, Partner & Community Providers

Terraform supports hundreds of providers, categorized as:

1. Official Providers – Maintained by HashiCorp
   e.g., AWS, Azure, GCP, Kubernetes

2. Partner Providers – Maintained by vendors (e.g., Oracle, Alibaba)

3. Community Providers – Maintained by the community (less reliable)

✅ Always check provider activity and documentation at: registry.terraform.io

🌐 Multi-Region and Multi-Cloud Configuration

🔁 Multi-Region Setup (Single Cloud, Multiple Regions)

To deploy resources in multiple AWS regions:

provider "aws" {
  alias  = "use1"
  region = "us-east-1"
}

provider "aws" {
  alias  = "usw2"
  region = "us-west-2"
}

resource "aws_instance" "east" {
  provider      = aws.use1
  ami           = "ami-east"
  instance_type = "t2.micro"
}

resource "aws_instance" "west" {
  provider      = aws.usw2
  ami           = "ami-west"
  instance_type = "t2.micro"
}

🌐 Multi-Cloud Setup (e.g., AWS + Azure)

provider "aws" {
  region = "us-east-1"
}

provider "azurerm" {
  features = {}
  subscription_id = "your-subscription-id"
  client_id       = "your-client-id"
  client_secret   = "your-secret"
  tenant_id       = "your-tenant-id"
}

⚠️ Each provider has a unique name and authentication mechanism. Use the official docs to get syntax and examples.

🧱 Terraform Resources

A resource represents a piece of infrastructure - like an EC2 instance, an S3 bucket, or a virtual machine.

🧩 Example (AWS EC2 Instance):

resource "aws_instance" "example" {
  ami           = var.ami_id
  instance_type = var.instance_type
}

Use documentation to find the correct syntax and resource names (e.g., aws_instance, azurerm_virtual_machine).

💡 Variables in Terraform

Hardcoding values (like AMI IDs, instance types) is bad practice. Use variables to make your Terraform configuration reusable.

🔽 Input Variables (Defined in variables.tf)

variable "ami_id" {
  description = "AMI for EC2"
  type        = string
  default     = "ami-0abcd1234"
}

variable "instance_type" {
  description = "Instance type"
  type        = string
  default     = "t2.micro"
}

🔼 Output Variables (Defined in outputs.tf)

output "instance_ip" {
  description = "Public IP of the instance"
  value       = aws_instance.example.public_ip
}

🧾 TFVARS: Dynamic Value Management

To dynamically pass values instead of hardcoding them in .tf files, use a terraform.tfvars file:

ami_id        = "ami-07a5b1a7"
instance_type = "t2.medium"

Terraform will automatically load this file during terraform apply. For custom files:

terraform apply -var-file="prod.tfvars"

🧠 Conditional Expressions in Terraform

Use conditionals when you want to assign values based on a variable (like environment type):

🔐 Example: Conditional CIDR Block

cidr_blocks = var.environment == "prod" ? ["10.0.1.0/24"] : ["10.0.2.0/24"]

This logic:

• Assigns one subnet range in production

• Assigns another for development

✅ Use it for:

• Public access controls

• Instance types

• Resource counts

• Tags and naming conventions

🔧 Built-in Functions in Terraform

Terraform provides useful built-in functions to manipulate data:

More at Terraform Function Docs

📁 Recommended Project Structure

To keep things clean and scalable, organize files like this:

terraform-project/
├── main.tf            # Core logic
├── variables.tf       # Input variables
├── outputs.tf         # Output variables
├── provider.tf        # Provider blocks
├── terraform.tfvars   # Variable values

This modular structure:

• Improves readability

• Makes collaboration easier

• Encourages reuse across environments🧪 Key Takeaways

✅ Providers connect Terraform to your cloud
✅ Use aliases for multi-region or multi-cloud setups
✅ Resources define what to deploy
✅ Variables and TFVARS make code reusable
✅ Conditionals and functions make configurations smarter
✅ Project structure matters for maintainability

🎯 Start writing small modules using providers and variables.
🔍 Read official provider docs.
🛠 Practice customizing configurations for different environments.

✅ Understand what Infrastructure as Code (IaC) really means
✅ Learn why Terraform is the top tool in the DevOps toolbox
✅ Install Terraform (even if you can't install software on your laptop!)
✅ Set up AWS authentication
✅ Write your first Terraform script
✅ Deploy your first EC2 instance
✅ Learn about Terraform’s lifecycle commands: init, plan, apply, destroy
✅ Understand Terraform's state file and its importance

🌐 Infrastructure as Code (IaC) – Explained Simply

Instead of manually creating cloud resources using the AWS Console, you write code to define and manage your infrastructure. This code can be versioned, reused, and shared - just like software code.

Example:

• Creating 1 S3 bucket? Easy via the AWS console.

• Creating 100 S3 buckets? IaC makes it fast, repeatable, and error-free.

Traditional Methods:

• ❌ Manual creation using the AWS Console - error-prone and repetitive

• ❌ AWS CLI or SDKs like Python + Boto3 - requires programming knowledge

• ❌ CloudFormation or ARM Templates - tied to specific cloud providers

💡 So Why Terraform?

Terraform gives us a universal language to define and manage infrastructure across multiple cloud providers - AWS, Azure, GCP, and more - using its own syntax called HCL (HashiCorp Configuration Language).

🔥 Key Benefits:

• 💥 Multi-cloud support with one tool

• 🧾 Readable and declarative syntax

• 🚀 Reusable modules and configurations

• 📦 Massive community and ecosystem

• ⚙️ No deep programming knowledge required

• 🔁 Version-controlled infrastructure (just like Git)

Instead of learning multiple tools like:

• AWS CloudFormation

• Azure ARM Templates

• OpenStack Heat Templates

...just learn Terraform, and it works for all of them.

🛠️ Installing Terraform

You can install Terraform in two ways:

🔹 Method 1: Local Installation

For Windows:

• Download the binary from terraform.io

• Add it to your system PATH

• Use Git Bash or PowerShell (not CMD)

For macOS:

brew tap hashicorp/tap
brew install hashicorp/tap/terraform

For Ubuntu/Linux:

sudo apt-get update && sudo apt-get install -y gnupg software-properties-common
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install terraform

🔹 Method 2: GitHub Codespaces (Recommended for Beginners)

Don’t have admin access or using a restricted office laptop?

Use GitHub Codespaces - your browser becomes your Dev environment.

• ✅ Free: 60 hours/month

• 🖥️ Environment: 2 CPUs, 4GB RAM

• 💻 Built-in Visual Studio Code

• 🌍 Access from any device with a browser

To set it up:

1. Fork the Repo

2. Click Code > Codespaces > Create codespace

3. Add Dev Container Configs for:

   • Terraform

   • AWS CLI

4. Rebuild the container - now you’re ready to use Terraform in-browser!

🔐 Setting Up AWS Authentication

Once the AWS CLI is installed, run:

aws configure

You’ll be prompted to enter:

• Access Key ID

• Secret Access Key

• Default region (e.g., us-east-1)

• Output format (json, table, or text)

🔒 Tip: Use IAM Users instead of root accounts for better security. You can generate access keys from the AWS IAM Console.

🔑 Important Note: This only allows your terminal (shell) to interact with AWS using the AWS CLI. It does not automatically grant access to Terraform.

The AWS CLI saves credentials in this location:

~/.aws/credentials

Giving Terraform Access to AWS

Once AWS CLI is configured, Terraform can also use those same credentials - if the provider block is correctly written in your .tf file.

Here's the minimal provider configuration:

provider "aws" {
  region = "us-east-1"
}

📝 Writing Your First Terraform Script

Let’s write a basic Terraform configuration to launch an EC2 instance.

Step 1: Create a file called main.tf

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-xxxxxxxxxxxxxxxxx" # Replace with valid AMI ID
  instance_type = "t2.micro"
  subnet_id     = "subnet-xxxxxxxxxxxxx"  # Replace with your Subnet ID
  key_name      = "your-key-name"         # Replace with your key pair name
}

💡 Where to find values:

• AMI ID: Launch an EC2 instance manually and copy the AMI ID

• Subnet ID: Use your default VPC subnet or create a new one

• Key Name: Use an existing EC2 key pair or create a new one from the AWS console

🔄 Terraform Lifecycle Commands

Follow these commands in order:

1️⃣ terraform init

Initializes the working directory and downloads provider plugins.

terraform init

2️⃣ terraform plan

Shows what changes Terraform will make. Think of this as a dry run.

terraform plan

3️⃣ terraform apply

Applies the configuration and creates real infrastructure on AWS.

terraform apply

Terraform will ask for confirmation - type yes.

4️⃣ terraform destroy

Destroys the infrastructure and avoids incurring AWS charges.

terraform destroy

📂 What is the Terraform State File?

Terraform creates a file called:

terraform.tfstate

This file:

• Records all the resources that Terraform created

• Tracks the current state of your infrastructure

• Is used internally to understand what changes need to be made

🔐 Important:
Manage this file securely - it may contain sensitive information like resource IDs and IPs.

In future guides, you’ll learn:

• How to use remote state storage (S3 + DynamoDB)

• How to secure and lock state files in team environments

• How state integrates with CI/CD pipelines

1. Scrum Framework Overview

Scrum follows an iterative approach, breaking down work into fixed time periods called Sprints. Each Sprint results in a potentially shippable product increment. The process involves specific roles, events, and artifacts.

Key Roles in Scrum:

• Product Owner: Defines product goals, prioritizes backlog items, and ensures value delivery.

• Scrum Master: Facilitates Scrum processes, removes impediments, and ensures adherence to Agile principles.

• Development Team: Cross-functional team responsible for delivering increments.

2. Scrum Process Flow

2.1 Product Backlog

The Product Backlog is a prioritized list of features, improvements, and fixes required for the product. The Product Owner maintains and refines this backlog continuously.

2.2 Backlog Refinement (Backlog Grooming)

Backlog Refinement (also known as Backlog Grooming) is an ongoing activity where the team discusses and clarifies backlog items, breaks them into smaller tasks, and estimates their effort.

2.3 Sprint Planning

Sprint Planning is the meeting where the team selects backlog items for the upcoming Sprint. The key outputs of Sprint Planning include:

• Sprint Goal: Defines the purpose of the Sprint.

• Sprint Backlog: A subset of Product Backlog items selected for the Sprint.

• Task Breakdown: Identifying the necessary tasks to complete selected items.

2.4 Sprint Execution

The Sprint is a time-boxed iteration (typically 1-4 weeks) where the team works on Sprint Backlog items. The team holds Daily Standup (Daily Scrum) Meetings to sync progress and discuss roadblocks.

2.5 Increment

By the end of the Sprint, the team delivers a Potentially Shippable Product Increment that meets the Definition of Done (DoD).

2.6 Sprint Review

The Sprint Review is held at the end of the Sprint, where the team demonstrates the completed work to stakeholders and gathers feedback.

2.7 Sprint Retrospective

The Sprint Retrospective follows the Sprint Review and is aimed at process improvement. The team discusses what went well, what didn’t, and how they can improve future Sprints.

3. Summary of Scrum Process Steps

1. Product Backlog Creation: Product Owner maintains and prioritizes backlog items.

2. Backlog Refinement: Team refines and estimates backlog items.

3. Sprint Planning: Team selects work for the Sprint and sets a goal.

4. Sprint Execution: Development occurs, guided by Daily Standups.

5. Increment Creation: A potentially shippable product increment is produced.

6. Sprint Review: Team showcases work and receives feedback.

7. Sprint Retrospective: The team reflects and improves the process.

8. Repeat: The process continues in cycles until the product is completed.

Scrum ensures adaptability, continuous improvement, and consistent product delivery. By following these steps, teams can efficiently manage their work and deliver high-value products in an Agile environment.

1️⃣ Azure Authentication Methods

Before configuring Terraform, understand the authentication options in Azure:

🔹 Managed Identity

✅ Best For:

• Azure resources (VMs, Functions, AKS, etc.) needing access to other Azure services.

• Secretless authentication within Azure.

✅ Types:

• System-assigned → Tied to a single resource.

• User-assigned → Reusable across multiple resources.

🔹 Service Principal

✅ Best For:

• Automation tools like Terraform, CI/CD (GitHub Actions, Azure DevOps).

• Programmatic access to Azure resources.

✅ Authentication Methods:

• Client Secret (Password-based) 🔑 → Easy but less secure.

• Certificate-based → More secure than secrets.

• Federated Identity (OIDC) 🎭 → No secret required (best for GitHub Actions & Kubernetes).

2️⃣ Prerequisites

Before proceeding, ensure you have:
✅ An Azure Subscription
✅ Azure CLI installed (az version to check)
✅ Terraform installed (terraform version to check`)

3️⃣ Authenticate with Azure CLI

Run the following command to log in to Azure:

az login

If you're using a cloud shell, authentication happens automatically.

4️⃣ Create a Service Principal for Terraform

To enable Terraform to authenticate securely, create a Service Principal:

az ad sp create-for-rbac --name "terraform-sp" --role="Contributor" --scopes="/subscriptions/YOUR_SUBSCRIPTION_ID"

This returns JSON with important credentials:

{
  "appId": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx",
  "displayName": "terraform-sp",
  "password": "xxxxxxxxxxxx",
  "tenant": "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
}

Save these values securely.

5️⃣ Configure Terraform Provider with Service Principal

Update your provider block in Terraform to use the service principal credentials:

provider "azurerm" {
  features {}
  subscription_id = "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
  tenant_id       = "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
  client_id       = "xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
  client_secret   = "xxxxxxxxxxxx"
}

This ensures Terraform authenticates using the service principal directly.

6️⃣ Verify Access with Terraform

To verify if the authentication is properly set up, run:

terraform plan

If authentication is configured correctly, Terraform will display the planned actions for your infrastructure. If there are issues with authentication, Terraform will return an error message.

7️⃣ Next Steps

• Use this authentication to deploy Azure resources with Terraform.

• Secure your credentials using an Azure Key Vault instead of storing them in Terraform files.

• Continue with infrastructure setup (next blog will cover creating an Azure resource).

This keeps it short, structured, and easy to follow. 🚀

🔹 What is a Service Principal?

A Service Principal is an identity created in Azure Active Directory (Azure AD) to authenticate applications or automation processes. It enables fine-grained access control by assigning specific roles and permissions to a non-human identity.

How to Create a Service Principal (CLI Method):

az ad sp create-for-rbac --name "my-app" --role "Contributor" --scopes "/subscriptions/{subscription-id}"

After running the above command, you will receive output containing essential credentials:

{
  "appId": "<client_id>",
  "password": "<client_secret>",
  "tenant": "<tenant_id>"
}

• appId → client_id

• password → client_secret

• tenant → tenant_id

How to Create a Service Principal (Portal Method):

1. Navigate to Azure Portal → Azure Active Directory.

2. Select App registrations → New registration.

3. Provide a name, select the supported account types, and register the application.

4. Go to Certificates & secrets to generate a client secret.

5. Assign necessary RBAC roles under Azure subscriptions.

Key Points:

✅ Requires manual management of secrets or certificates.
✅ Can be used for automation, scripts, or CI/CD pipelines.
✅ Supports role-based access control (RBAC).
✅ Needs explicit lifecycle management (creation, rotation, deletion).

🔹 What is a Managed Identity?

A Managed Identity is an Azure feature that eliminates the need for managing credentials. Azure automatically handles authentication when resources (like Virtual Machines, Functions, and App Services) need access to other Azure services.

Types of Managed Identities:

1. System-assigned – Tied to a single Azure resource and deleted when the resource is deleted.

2. User-assigned – Created independently and can be assigned to multiple resources.

How to Enable a System-Assigned Managed Identity (CLI Method):

az vm identity assign --resource-group myResourceGroup --name myVM

How to Enable a Managed Identity (Portal Method):

1. Navigate to Azure Portal → Your Resource (VM, App Service, etc.).

2. Go to Identity under the settings.

3. Enable System-assigned or User-assigned identity.

4. Assign necessary RBAC roles under Azure subscriptions.

Key Points:

✅ No need to manage credentials manually.
✅ Seamless integration with Azure services.
✅ Automatically rotates credentials for security.
✅ System-assigned identities are tied to a specific resource, while user-assigned identities can be shared.

🔹 When to Use Which?

🔹 Final Thoughts

Both Service Principals and Managed Identities serve specific purposes. If you need a reusable identity for automation, Service Principals are the way to go. If you want a secure and hassle-free authentication method for Azure resources, Managed Identities are the best choice.

Which one do you use the most? Let me know in the comments! 🚀