Jekyll2023-09-04T10:58:19+00:00http://www.devocative.org/feed.xmlEvocative DevelopmentLearning by Sharing @ Devocative.OrgMehdi BizhaniKubernetes Objects in a Nutshell2021-01-08T00:00:00+00:002021-01-08T00:00:00+00:00http://www.devocative.org/article/tech/k8s-objects<h2 id="basic">Basic</h2>
<p>Common root elements of all K8S config files:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span>
<span class="na">kind</span><span class="pi">:</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span>
<span class="na">namespace</span><span class="pi">:</span> <span class="c1"># OPTIONAL</span>
<span class="na">labels</span><span class="pi">:</span> <span class="c1"># OPTIONAL</span>
<span class="na">spec</span><span class="pi">:</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong><code class="language-plaintext highlighter-rouge">apiVersion</code> ~ <code class="language-plaintext highlighter-rouge">kind</code> mapping</strong></p>
<table>
<thead>
<tr>
<th>Kind</th>
<th>API Version</th>
<th>Short Name</th>
</tr>
</thead>
<tbody>
<tr>
<td>Pod</td>
<td>v1</td>
<td>po</td>
</tr>
<tr>
<td>ReplicationController</td>
<td>v1</td>
<td>rc</td>
</tr>
<tr>
<td>ReplicaSet</td>
<td>apps/v1</td>
<td>rs</td>
</tr>
<tr>
<td>Deployment</td>
<td>apps/v1</td>
<td>deploy</td>
</tr>
<tr>
<td>Service</td>
<td>v1</td>
<td>svc</td>
</tr>
<tr>
<td>PersistentVolume</td>
<td>v1</td>
<td>pv</td>
</tr>
<tr>
<td>PersistentVolumeClaim</td>
<td>v1</td>
<td>pvc</td>
</tr>
<tr>
<td>Namespace</td>
<td>v1</td>
<td>ns</td>
</tr>
<tr>
<td>ConfigMap</td>
<td>v1</td>
<td>cm</td>
</tr>
<tr>
<td>Secret</td>
<td>v1</td>
<td>-</td>
</tr>
</tbody>
</table>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl api-resources</code> - shows complete list of above table</li>
</ul>
<h2 id="pod">Pod</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pod</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-pod</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl describe pod sample-pod</code></li>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -it sample-pod -- sh</code> (busybox)</li>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -c nginx -it sample-pod -- bash</code> (nginx)</li>
<li>All containers in the same Pod have access each other on <code class="language-plaintext highlighter-rouge">localhost</code>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre>user<span class="nv">$ </span>kubectl <span class="nb">exec</span> <span class="nt">-c</span> busybox sample-pod <span class="nt">--</span> netstat <span class="nt">-lntp</span>
Active Internet connections <span class="o">(</span>only servers<span class="o">)</span>
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:<span class="k">*</span> LISTEN -
tcp 0 0 :::80 :::<span class="k">*</span> LISTEN -
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>EXPOSE as SERVICE - <code class="language-plaintext highlighter-rouge">kubectl expose pod sample-pod --name=sample-srv --port=8080 --target-port=80</code></li>
</ul>
<p><strong>Note:</strong> <code class="language-plaintext highlighter-rouge">kubectl run redis --image=redis --restart=Never --dry-run -o yamle > redis-pod.yml</code></p>
<h3 id="container-lifecycle-events">Container Lifecycle Events</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pod</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-pod</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">command</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">sh</span>
<span class="pi">-</span> <span class="s">-c</span>
<span class="pi">-</span> <span class="pi">|</span>
<span class="s">for i in $(seq 2 2 8); do</span>
<span class="s">echo "Main: $i" >> /var/main.log</span>
<span class="s">sleep 2</span>
<span class="s">done</span>
<span class="s">sh</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">lifecycle</span><span class="pi">:</span>
<span class="na">postStart</span><span class="pi">:</span>
<span class="na">exec</span><span class="pi">:</span>
<span class="na">command</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">sh</span>
<span class="pi">-</span> <span class="s">-c</span>
<span class="pi">-</span> <span class="pi">|</span>
<span class="s">for i in $(seq 1 1 8); do</span>
<span class="s">echo "PostStart: $i" >> /var/post-start.log</span>
<span class="s">sleep 2</span>
<span class="s">done</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Now watch the output:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre>watch <span class="nt">-t</span> <span class="s2">"
kubectl get po;
echo '</span><span class="se">\n</span><span class="s2">---------- main.log</span><span class="se">\n</span><span class="s2">';
kubectl exec -it sample-pod -c busybox -- cat /var/main.log
echo '</span><span class="se">\n</span><span class="s2">---------- post-start.log</span><span class="se">\n</span><span class="s2">';
kubectl exec -it sample-pod -c busybox -- cat /var/post-start.log
echo '</span><span class="se">\n</span><span class="s2">---------- Nginx</span><span class="se">\n</span><span class="s2">';
kubectl logs sample-pod nginx"</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<asciinema-player src="/assets/casts/k8s-objects-pod-lifecycle.cast" cols="120" rows="33" poster="npt:0:03"></asciinema-player>
<ul>
<li>Two Events [<a href="https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/">REF</a>]
<ul>
<li><code class="language-plaintext highlighter-rouge">postStart</code>
<ul>
<li>no guarantee that the hook will execute before the container <code class="language-plaintext highlighter-rouge">ENTRYPOINT</code></li>
<li>runs asynchronously relative to the Container’s code</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">preStop</code></li>
<li>For each, <strong>only use one</strong> of the followings
<ul>
<li><code class="language-plaintext highlighter-rouge">exec</code></li>
<li><code class="language-plaintext highlighter-rouge">httpGet</code></li>
<li><code class="language-plaintext highlighter-rouge">tcpSocket</code></li>
</ul>
</li>
</ul>
</li>
</ul>
<h2 id="replication-controller">Replication Controller</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ReplicationController</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-rc</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">2</span>
<span class="na">template</span><span class="pi">:</span> <span class="c1"># POD DEFINITION vvv</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="c1"># 'name' here is IGNORED</span>
<span class="na">labels</span><span class="pi">:</span> <span class="c1"># REQUIRED for its implicit selector</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-rc</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
<span class="na">ports</span><span class="pi">:</span> <span class="c1"># OPTIONAL (nginx is on port 80 by default)</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">80</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="replicaset">ReplicaSet</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ReplicaSet</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-rs</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">2</span>
<span class="na">selector</span><span class="pi">:</span> <span class="c1"># REQUIRED</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-rs</span>
<span class="na">template</span><span class="pi">:</span> <span class="c1"># POD DEFINITION vvv</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-rs</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">containerPort</span><span class="pi">:</span> <span class="m">80</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Change Replicas:
<ul>
<li>Update file with new <code class="language-plaintext highlighter-rouge">replicas</code> value and <code class="language-plaintext highlighter-rouge">kubectl replace -f FILE</code> (<code class="language-plaintext highlighter-rouge">apply</code> works the same)</li>
<li><code class="language-plaintext highlighter-rouge">kubectl scale --replicas=3 -f FILE</code></li>
<li><code class="language-plaintext highlighter-rouge">kubectl scale --replicas=3 replicaset sample-rs</code></li>
</ul>
</li>
<li>Non-Template Pod acquisitions [<a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#non-template-pod-acquisitions">REF</a>]
<blockquote>
<p>While you can create bare Pods with no problems, it is strongly recommended to make sure that
the bare Pods do not have labels which match the selector of one of your ReplicaSets.
The reason for this is because a ReplicaSet is not limited to owning Pods specified by its template–
it can acquire other Pods in the manner specified in the previous sections.</p>
</blockquote>
</li>
</ul>
<h2 id="deployment">Deployment</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Deployment</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-dep</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">2</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-dep</span>
<span class="na">template</span><span class="pi">:</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-dep</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Mostly similar to ReplicaSet!</li>
<li>Deployment Strategy - <strong>Recreate</strong> and <strong>Rolling Update</strong> (default)</li>
<li>Rolling
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl rollout status deployment sample-dep</code> - shows current deployment status</li>
<li><code class="language-plaintext highlighter-rouge">kubectl rollout restart deployment sample-dep</code> - restart deployment (i.e. redeploy)</li>
<li><code class="language-plaintext highlighter-rouge">kubectl rollout history deployment sample-dep</code> - shows all revisions</li>
<li><code class="language-plaintext highlighter-rouge">kubectl rollout history deployment sample-dep --revision=NUM</code> - shows specific revision detail</li>
<li><code class="language-plaintext highlighter-rouge">kubectl rollout undo deployment sample-dep</code> - rollbacks to previous deployment</li>
<li><code class="language-plaintext highlighter-rouge">kubectl set image deployment sample-dep busybox=1.32-glibc</code></li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">kubectl run nginx --image=nginx</code> - creates a deployment</li>
</ul>
<h2 id="service">Service</h2>
<blockquote>
<p>Kubernetes networking addresses four concerns [<a href="https://kubernetes.io/docs/concepts/services-networking/">REF</a>]:</p>
<ul>
<li>Containers within a <strong>Pod</strong> use networking to communicate via <strong>loopback</strong>.</li>
<li>Cluster networking provides communication between different Pods.</li>
<li>The <strong>Service</strong> resource lets you expose an application running in Pods to be reachable from <strong>outside your cluster</strong>.</li>
<li>You can also use <strong>Services</strong> to publish services only for consumption <strong>inside your cluster</strong>.</li>
</ul>
</blockquote>
<p>Four Types [<a href="https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types">REF</a>]</p>
<ul>
<li><strong>ClusterIP</strong>
<ul>
<li>Default Value</li>
<li>Makes the Service only reachable from within the cluster</li>
</ul>
</li>
<li><strong>NodePort</strong>
<ul>
<li>Exposes the Service on each Node’s IP at a static port</li>
<li>A <code class="language-plaintext highlighter-rouge">ClusterIP</code> Service, to which the <code class="language-plaintext highlighter-rouge">NodePort</code> Service routes, is automatically created</li>
</ul>
</li>
<li><strong>LoadBalancer</strong>
<ul>
<li>Exposes the Service externally using a cloud provider’s load balancer</li>
<li><code class="language-plaintext highlighter-rouge">NodePort</code> and <code class="language-plaintext highlighter-rouge">ClusterIP</code> Services, to which the external load balancer routes, are automatically created</li>
</ul>
</li>
<li><strong>ExternalName</strong>
<ul>
<li>Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a <code class="language-plaintext highlighter-rouge">CNAME</code> record with its value</li>
<li>No proxying of any kind is set up</li>
</ul>
</li>
</ul>
<h3 id="cluster-ip">Cluster IP</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-srv-cip</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="c1"># type: ClusterIP # NOT REQUIRED, AS DEFAULT</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-dep</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">protocol</span><span class="pi">:</span> <span class="s">TCP</span>
<span class="na">port</span><span class="pi">:</span> <span class="m">80</span>
<span class="na">targetPort</span><span class="pi">:</span> <span class="m">80</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -it sample-dep-XXXXXXXX -- sh</code>
<ul>
<li><code class="language-plaintext highlighter-rouge">wget -qO - http://sample-srv-cip</code></li>
<li>Search DNS
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre># nslookup sample-srv-cip
Server: 10.43.0.10
Address: 10.43.0.10:53
Name: sample-srv-cip.default.svc.cluster.local
Address: 10.43.110.139
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>So <code class="language-plaintext highlighter-rouge">sample-srv-cip</code> has all alternative names:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">sample-srv-cip.default</code></li>
<li><code class="language-plaintext highlighter-rouge">sample-srv-cip.default.svc</code></li>
<li><code class="language-plaintext highlighter-rouge">sample-srv-cip.default.svc.cluster.local</code></li>
</ul>
</li>
</ul>
</li>
<li><strong>FORMAT</strong> <code class="language-plaintext highlighter-rouge">SERVICE_NAME.NAMESPACE.svc.DOMAIN</code></li>
<li>Now copy deployment and service as <code class="language-plaintext highlighter-rouge">sample-dep2</code> and <code class="language-plaintext highlighter-rouge">sample-srv-cip2</code>, and apply them.
<ul>
<li>You can <code class="language-plaintext highlighter-rouge">ping</code> and <code class="language-plaintext highlighter-rouge">wget</code> other services from the pods.</li>
</ul>
</li>
</ul>
<h3 id="node-port">Node Port</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-srv-np</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">NodePort</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-dep</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">protocol</span><span class="pi">:</span> <span class="s">TCP</span>
<span class="na">port</span><span class="pi">:</span> <span class="m">80</span> <span class="c1"># REQUIRED</span>
<span class="na">targetPort</span><span class="pi">:</span> <span class="m">80</span> <span class="c1"># if not set, same as port</span>
<span class="na">nodePort</span><span class="pi">:</span> <span class="m">31111</span> <span class="c1"># DEFAULT RANGE [30000, 32767]. If not set, get randomly from the range </span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Access <code class="language-plaintext highlighter-rouge">http://NODE:31111</code></li>
</ul>
<h2 id="volume">Volume</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pod-vol</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-pod-vol</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">mountPath</span><span class="pi">:</span> <span class="s">/data</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">data-dir</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">mountPath</span><span class="pi">:</span> <span class="s">/usr/share/nginx/html</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">html-dir</span>
<span class="na">readOnly</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">data-dir</span>
<span class="na">hostPath</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/opt/test-pod</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">DirectoryOrCreate</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">html-dir</span>
<span class="na">hostPath</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/opt/test-pod</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">DirectoryOrCreate</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -it sample-pod-vol -- sh</code>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre># wget -qO - http://localhost
wget: server returned error: HTTP/1.1 403 Forbidden
# echo "Hello" > /data/index.html
# wget -qO - http://localhost
Hello
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<!--
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Script/upload-to-registry.sh"></script>
</div>
-->
<h2 id="storage">Storage</h2>
<h3 id="pv">PV</h3>
<p>[<a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/">REF</a>]</p>
<blockquote>
<ul>
<li>A PersistentVolume (PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes.</li>
<li>Independent lifecycle of any Pod that uses the PV</li>
<li>Two types of provisioning
<ul>
<li>Static</li>
<li>Dynamic
<ul>
<li>No match for user’s PVC request</li>
<li>Based on <code class="language-plaintext highlighter-rouge">StorageClass</code>, the PVC request must refer a storage class</li>
<li>Administrator must have created and configured that class for dynamic provisioning</li>
<li>Claims that request the class <code class="language-plaintext highlighter-rouge">""</code> effectively disable dynamic provisioning for themselves.</li>
</ul>
</li>
</ul>
</li>
<li>A PVC to PV binding is a <strong>one-to-one mapping</strong>, using a ClaimRef which is a bi-directional binding between the PV and the PVC</li>
<li>Reserve a PV
<ul>
<li>Specifying a PV inside PVC</li>
<li>Semi-Reserve: using <code class="language-plaintext highlighter-rouge">labels</code> in PV and <code class="language-plaintext highlighter-rouge">selectors</code> in PVC (not in the REF)</li>
</ul>
</li>
<li>Access modes:
<ul>
<li><code class="language-plaintext highlighter-rouge">ReadWriteOnce</code> – the volume can be mounted as read-write <strong>by a single node</strong></li>
<li><code class="language-plaintext highlighter-rouge">ReadOnlyMany</code> – the volume can be mounted read-only <strong>by many nodes</strong></li>
<li><code class="language-plaintext highlighter-rouge">ReadWriteMany</code> – the volume can be mounted as read-write <strong>by many nodes</strong></li>
</ul>
</li>
<li>PVC
<ul>
<li>Request for storage by a user and consume PV resources</li>
<li>Setting <code class="language-plaintext highlighter-rouge">label</code> on PV can be ignored for matching, however the <code class="language-plaintext highlighter-rouge">selector</code> is considered during matching</li>
</ul>
</li>
</ul>
</blockquote>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">PersistentVolume</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pv1</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">nfs</span>
<span class="na">target</span><span class="pi">:</span> <span class="s">sample1</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">accessModes</span><span class="pi">:</span> <span class="c1"># REQUIRED</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span>
<span class="na">capacity</span><span class="pi">:</span> <span class="c1"># REQUIRED</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">1Gi</span> <span class="c1"># UNITS: Ki,Mi,Gi,Ti (power-of-two units)</span>
<span class="c1">#volumeMode: Filesystem # Filesystem (default), Block</span>
<span class="c1">#persistentVolumeReclaimPolicy: Retain # Retain (default), Delete, Recycle (deprecated)</span>
<span class="na">nfs</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">/opt/NFS/TEST</span>
<span class="na">server</span><span class="pi">:</span> <span class="s">nfs.server.local</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl get pv</code>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre>NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
sample-pv1 1Gi RWO Retain Available 65s
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<h3 id="pvc">PVC</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">PersistentVolumeClaim</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pvc1</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">accessModes</span><span class="pi">:</span> <span class="c1"># REQUIRED</span>
<span class="pi">-</span> <span class="s">ReadWriteOnce</span>
<span class="na">resources</span><span class="pi">:</span> <span class="c1"># REQUIRED</span>
<span class="na">requests</span><span class="pi">:</span>
<span class="na">storage</span><span class="pi">:</span> <span class="s">1Gi</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">nfs</span>
<span class="na">target</span><span class="pi">:</span> <span class="s">sample1</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl get pvc</code>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre>NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
sample-pvc1 Bound sample-pv1 1Gi RWO 16s
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">kubectl get pv</code>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre>NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
sample-pv1 1Gi RWO Retain Bound default/sample-pvc1 39s
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<h3 id="usage">Usage</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pod-pv.pvc1</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">test-pod-pv.pvc1</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">mountPath</span><span class="pi">:</span> <span class="s">/data</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">html-dir</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">nginx</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:1.18.0</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">mountPath</span><span class="pi">:</span> <span class="s">/usr/share/nginx/html</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">html-dir</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">html-dir</span>
<span class="na">persistentVolumeClaim</span><span class="pi">:</span>
<span class="na">claimName</span><span class="pi">:</span> <span class="s">sample-pvc1</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="configuration-data">Configuration Data</h2>
<h3 id="config-map">Config Map</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ConfigMap</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-cm</span>
<span class="na">data</span><span class="pi">:</span>
<span class="na">K1</span><span class="pi">:</span> <span class="s">V1</span>
<span class="na">K2</span><span class="pi">:</span> <span class="s">V2</span>
<span class="s">other.txt</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">This is a text content, stored in file via ConfigMap volume!</span>
<span class="s">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pod-cm</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">envFrom</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">configMapRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-cm</span> <span class="c1"># INCLUDE ALL KEYS</span>
<span class="na">env</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">K11</span>
<span class="na">valueFrom</span><span class="pi">:</span>
<span class="na">configMapKeyRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-cm</span>
<span class="na">key</span><span class="pi">:</span> <span class="s">K1</span> <span class="c1"># JUST GET THE KEY</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">cmdata</span>
<span class="na">configMap</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-cm</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -it sample-pod-cm -- env</code>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="rouge-code"><pre>PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=sample-pod-cm
TERM=xterm
K1=V1
K2=V2
K11=V1
KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1
KUBERNETES_SERVICE_HOST=10.43.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT=tcp://10.43.0.1:443
KUBERNETES_PORT_443_TCP=tcp://10.43.0.1:443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_PORT=443
HOME=/root
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -it sample-pod-cm -- sh</code>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre># ls -l /opt/config-map
total 0
lrwxrwxrwx 1 root root 9 Dec 14 09:12 K1 -> ..data/K1
lrwxrwxrwx 1 root root 9 Dec 14 09:12 K2 -> ..data/K2
lrwxrwxrwx 1 root root 16 Dec 14 09:12 other.txt -> ..data/other.txt
# cat /opt/config-map/other.txt
This is a text content, stored in file via ConfigMap volume!
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<h3 id="secret">Secret</h3>
<ul>
<li>Store and manage sensitive information</li>
<li>Secret vs ConfigMap [<a href="https://medium.com/google-cloud/kubernetes-configmaps-and-secrets-68d061f7ab5b">Ref1</a>, <a href="https://stackoverflow.com/questions/36912372/kubernetes-secrets-vs-configmaps">Ref2</a>]
<blockquote>
<p>The big difference between Secrets and ConfigMaps are that Secrets are obfuscated with a Base64 encoding.
There may be more differences in the future, but it is good practice to use Secrets for confidential data
(like API keys) and ConfigMaps for non-confidential data (like port numbers).</p>
</blockquote>
</li>
<li>Secret can simply be created by CLI or file:
<ul>
<li>CLI (imperative)
<ul>
<li><code class="language-plaintext highlighter-rouge">echo 'Sample File for Secret' > myfile.txt</code></li>
<li><code class="language-plaintext highlighter-rouge">kubectl create secret generic sample-sec --from-file=message.txt=myfile.txt --from-literal=K1=V1</code></li>
</ul>
</li>
<li>File
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Secret</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-sec</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">Opaque</span> <span class="c1"># DEFAULT</span>
<span class="na">data</span><span class="pi">:</span>
<span class="na">K1</span><span class="pi">:</span> <span class="s">V1</span>
<span class="s">message.txt</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">Sample File for Secret</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
</li>
</ul>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Pod</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-pod-sec</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:1.32</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">env</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">KEY1</span>
<span class="na">valueFrom</span><span class="pi">:</span>
<span class="na">secretKeyRef</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-sec</span>
<span class="na">key</span><span class="pi">:</span> <span class="s">K1</span>
<span class="na">volumeMounts</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">secdata</span>
<span class="na">mountPath</span><span class="pi">:</span> <span class="s2">"</span><span class="s">/etc/sec-data"</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">secdata</span>
<span class="na">secret</span><span class="pi">:</span>
<span class="na">secretName</span><span class="pi">:</span> <span class="s">sample-sec</span>
<span class="na">items</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s">message.txt</span>
<span class="na">path</span><span class="pi">:</span> <span class="s">message.txt</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl exec -it sample-pod-sec -- sh</code>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre>/ # ls -l /etc/sec-data/
total 0
lrwxrwxrwx 1 root root 18 Dec 24 10:08 message.txt -> ..data/message.txt
/ # env
KEY1=V1
KUBERNETES_PORT=tcp://10.43.0.1:443
...
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<h2 id="namespace">Namespace</h2>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Namespace</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-ns</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>OR <code class="language-plaintext highlighter-rouge">kubectl create namespace smaple-ns</code></li>
<li><code class="language-plaintext highlighter-rouge">kubectl config set-context $(kubectl config current-context) --namespace=sample-ns</code> - Update default namesapce for <code class="language-plaintext highlighter-rouge">kubectl</code></li>
</ul>
<h3 id="resource-quota">Resource Quota</h3>
<ul>
<li>Provides constraints that limit <strong>aggregate</strong> resource consumption per namespace</li>
<li>Pod resource must be specified on enabled compute resources quota like <code class="language-plaintext highlighter-rouge">cpu</code> and <code class="language-plaintext highlighter-rouge">memory</code></li>
<li>[<a href="https://kubernetes.io/docs/concepts/policy/resource-quotas/">REF</a>]</li>
</ul>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ResourceQuota</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">sample-rq</span>
<span class="na">namespace</span><span class="pi">:</span> <span class="s">sample-ns</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">hard</span><span class="pi">:</span>
<span class="na">pods</span><span class="pi">:</span> <span class="s2">"</span><span class="s">10"</span>
<span class="s">requests.cpu</span><span class="pi">:</span> <span class="s2">"</span><span class="s">4"</span>
<span class="s">requests.memory</span><span class="pi">:</span> <span class="s">5Gi</span>
<span class="s">limits.cpu</span><span class="pi">:</span> <span class="s2">"</span><span class="s">4"</span>
<span class="s">limits.memory</span><span class="pi">:</span> <span class="s">5Gi</span>
</pre></td></tr></tbody></table></code></pre></div></div>Mehdi BizhaniSimple notes on common Kubernetes objectsKubernetes Recipe - RKE2021-01-07T00:00:00+00:002021-01-07T00:00:00+00:00http://www.devocative.org/article/tech/k8s-rke<h2 id="introduction">Introduction</h2>
<p>One of the challenges at the beginning of learning Kubernetes is installing it. Due to Kubernetes documentation,
there are limited ways [<a href="https://kubernetes.io/docs/setup/">REF</a>]:</p>
<ul>
<li>Learning Environment
<ul>
<li>Kind</li>
<li>Minikube</li>
</ul>
</li>
<li>Production Environment
<ul>
<li>kubeadm</li>
<li>kops</li>
<li>Kubespray</li>
</ul>
</li>
</ul>
<p>However, according to <a href="https://landscape.cncf.io/category=certified-kubernetes-distribution,certified-kubernetes-installer&format=card-mode&grouping=category&license=open-source&sort=stars">CNCF</a>,
there are many open source Kubernetes distributions and tools to set up Kubernetes. RKE is one of the installation tools
that helps you install your Kubernetes cluster with ease. You can also install a single node for your learning environment.</p>
<p><strong>Note</strong>: Lots of texts introduce Minikube for learning env. However, in my opinion, even for installation, it has many challenges.
Even for learning env, I recommend using a single node by RKE.</p>
<h2 id="installing-a-k8s-cluster">Installing a K8S Cluster</h2>
<p><strong>Note:</strong> in this text, <em>laptop</em> means your working <em>workstation</em> or <em>computer</em>, which is used to set up k8s on the node(s).</p>
<h3 id="on-laptop">On Laptop</h3>
<ul>
<li>Create SSH key
<ul>
<li><code class="language-plaintext highlighter-rouge">ssh-keygen -t rsa -b 4096 -f ~/.ssh/rke</code>
<ul>
<li><code class="language-plaintext highlighter-rouge">$HOME/.ssh/rke</code> - SSH private key, keep this secure</li>
<li><code class="language-plaintext highlighter-rouge">$HOME/.ssh/rke.pub</code> - SSH public key, copy this to nodes</li>
<li>Setting passphrase is optional.</li>
</ul>
</li>
</ul>
</li>
<li>Install <code class="language-plaintext highlighter-rouge">kubectl</code> (and add it to the <code class="language-plaintext highlighter-rouge">PATH</code>) [<a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">REF</a>]
<ul>
<li><code class="language-plaintext highlighter-rouge">apt-get update && sudo apt-get install -y apt-transport-https gnupg2 curl</code></li>
<li><code class="language-plaintext highlighter-rouge">curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -</code></li>
<li><code class="language-plaintext highlighter-rouge">echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list</code></li>
<li><code class="language-plaintext highlighter-rouge">apt-get update && apt-get install -y kubectl</code></li>
</ul>
</li>
</ul>
<h3 id="prepare-your-nodes">Prepare your Node(s)</h3>
<p>On each node</p>
<ul>
<li>Check your hostname and assert <code class="language-plaintext highlighter-rouge">/etc/hosts</code></li>
<li>Disable Swap
<ul>
<li><code class="language-plaintext highlighter-rouge">swapoff -a</code></li>
<li>Remove any <code class="language-plaintext highlighter-rouge">swap</code> entry from <code class="language-plaintext highlighter-rouge">/etc/fstab</code></li>
</ul>
</li>
<li>Install SSH server</li>
<li>Install Docker CE</li>
<li>Create user <code class="language-plaintext highlighter-rouge">rke</code>
<ul>
<li><code class="language-plaintext highlighter-rouge">adduser rke</code></li>
<li><code class="language-plaintext highlighter-rouge">usermod -aG docker rke</code></li>
</ul>
</li>
<li>Install your laptop’s public ssh key (<code class="language-plaintext highlighter-rouge">rke.pub</code>)
<ul>
<li><code class="language-plaintext highlighter-rouge">cat ~/.ssh/rke.pub | ssh rke@NODE "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys"</code></li>
</ul>
</li>
</ul>
<h3 id="run-rke">Run RKE</h3>
<p>On the laptop:</p>
<ul>
<li>[Optional] If you have set the passphrase, install <code class="language-plaintext highlighter-rouge">ssh-agent</code> and run it
<ul>
<li><code class="language-plaintext highlighter-rouge">eval $(ssh-agent)</code></li>
<li><code class="language-plaintext highlighter-rouge">ssh-add</code>, and enter your passphrase</li>
</ul>
</li>
<li>Test your SSH connection to your node(s)
<ul>
<li><code class="language-plaintext highlighter-rouge">ssh -i ~/.ssh/rke rke@NODE</code> - SSH login without asking <code class="language-plaintext highlighter-rouge">rke</code>’s password</li>
</ul>
</li>
<li>Download <a href="https://github.com/rancher/rke/releases/latest">RKE Release</a>, rename it to <code class="language-plaintext highlighter-rouge">rke</code>, and set it in <code class="language-plaintext highlighter-rouge">PATH</code></li>
<li><code class="language-plaintext highlighter-rouge">rke config</code>
<ul>
<li>Answer the questions to setup your k8s cluster</li>
<li>It creates a <code class="language-plaintext highlighter-rouge">cluster.yml</code></li>
<li>Modify <code class="language-plaintext highlighter-rouge">cluster.yml</code>
<ul>
<li>[Optional] Prepend all <code class="language-plaintext highlighter-rouge">system_images</code> with your private registry</li>
<li>[Optional] Set <code class="language-plaintext highlighter-rouge">ssh_agent_auth</code> to <code class="language-plaintext highlighter-rouge">true</code> if you set passphrase</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">rke config -s</code> - list all images for download and store in your private registry</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">rke up</code>
<ul>
<li>It’ll show <code class="language-plaintext highlighter-rouge">Finished building Kubernetes cluster successfully</code></li>
<li>A <code class="language-plaintext highlighter-rouge">kube_config_cluster.yml</code> is created which is <code class="language-plaintext highlighter-rouge">kubectl</code> config file</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">kubectl --kubeconfig=kube_config_cluster.yml cluster-info</code></li>
<li>Run <code class="language-plaintext highlighter-rouge">kubectl</code> without <code class="language-plaintext highlighter-rouge">--kubeconfig</code>
<ul>
<li><code class="language-plaintext highlighter-rouge">export KUBECONFIG=$(pwd)/kube_config_cluster.yml</code></li>
<li><code class="language-plaintext highlighter-rouge">mkdir -p ~/.kube && cp -f kube_config_cluster.yml ~/.kube/config && chmod 600 ~/.kube/config</code></li>
</ul>
</li>
</ul>
<p>Here is a simple <code class="language-plaintext highlighter-rouge">rke config</code> questionnaire:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
</pre></td><td class="rouge-code"><pre>(*) [+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: ~/.ssh/rke
[+] Number of Hosts [1]:
(*) [+] SSH Address of host (1) [none]: r1
[+] SSH Port of host (1) [22]:
[+] SSH Private Key Path of host (r1) [none]:
[-] You have entered empty SSH key path, trying fetch from SSH key parameter
[+] SSH Private Key of host (r1) [none]:
[-] You have entered empty SSH key, defaulting to cluster level SSH key: ~/.ssh/rke
(*) [+] SSH User of host (r1) [ubuntu]: rke
[+] Is host (r1) a Control Plane host (y/n)? [y]:
(*) [+] Is host (r1) a Worker host (y/n)? [n]: y
(*) [+] Is host (r1) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (r1) [none]:
[+] Internal IP of host (r1) [none]:
[+] Docker socket path on host (r1) [/var/run/docker.sock]:
[+] Network Plugin Type (flannel, calico, weave, canal) [canal]:
[+] Authentication Strategy [x509]:
[+] Authorization Mode (rbac, none) [rbac]:
[+] Kubernetes Docker image [rancher/hyperkube:v1.19.3-rancher1]:
[+] Cluster domain [cluster.local]:
[+] Service Cluster IP Range [10.43.0.0/16]:
[+] Enable PodSecurityPolicy [n]:
[+] Cluster Network CIDR [10.42.0.0/16]:
[+] Cluster DNS Service IP [10.43.0.10]:
[+] Add addon manifest URLs or YAML files [no]:
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Only questions with <code class="language-plaintext highlighter-rouge">(*)</code> mark are answered, others are passed with default.</li>
<li>Line 21, <code class="language-plaintext highlighter-rouge">Cluster IP Range [10.43.0.0/16]</code> is IP range for Services
<ul>
<li>Line 24, DNS service’s IP is in the Cluster IP range.</li>
</ul>
</li>
<li>Line 23, <code class="language-plaintext highlighter-rouge">CIDR [10.42.0.0/16]</code> is IP range for Pods</li>
</ul>
<p>Now, some parts of a single-node <code class="language-plaintext highlighter-rouge">cluster.yml</code></p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
</pre></td><td class="rouge-code"><pre><span class="na">nodes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">address</span><span class="pi">:</span> <span class="s">r1</span>
<span class="na">port</span><span class="pi">:</span> <span class="s2">"</span><span class="s">22"</span>
<span class="na">internal_address</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">role</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">controlplane</span>
<span class="pi">-</span> <span class="s">worker</span>
<span class="pi">-</span> <span class="s">etcd</span>
<span class="na">hostname_override</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">user</span><span class="pi">:</span> <span class="s">rke</span>
<span class="na">docker_socket</span><span class="pi">:</span> <span class="s">/var/run/docker.sock</span>
<span class="na">ssh_key</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">ssh_key_path</span><span class="pi">:</span> <span class="s">~/.ssh/rke</span>
<span class="na">ssh_cert</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">ssh_cert_path</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">labels</span><span class="pi">:</span> <span class="pi">{}</span>
<span class="na">taints</span><span class="pi">:</span> <span class="pi">[]</span>
<span class="na">services</span><span class="pi">:</span>
<span class="s">...</span>
<span class="na">network</span><span class="pi">:</span>
<span class="na">plugin</span><span class="pi">:</span> <span class="s">canal</span>
<span class="s">...</span>
<span class="nn">...</span>
<span class="na">ssh_key_path</span><span class="pi">:</span> <span class="s">~/.ssh/rke</span>
<span class="na">ssh_cert_path</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">ssh_agent_auth</span><span class="pi">:</span> <span class="no">false</span>
<span class="nn">...</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="test-cluster">Test Cluster</h3>
<ul>
<li>Create file <code class="language-plaintext highlighter-rouge">rancher-demo-deployment.yml</code>:</li>
</ul>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
</pre></td><td class="rouge-code"><pre><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Deployment</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">rancher-demo-dpl</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">replicas</span><span class="pi">:</span> <span class="m">1</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">matchLabels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">rancher-demo</span>
<span class="na">template</span><span class="pi">:</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">rancher-demo</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">rancher-demo</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">superseb/rancher-demo</span>
<span class="nn">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Service</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">rancher-demo-srv</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">selector</span><span class="pi">:</span>
<span class="na">app</span><span class="pi">:</span> <span class="s">rancher-demo</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">rancher-demo</span>
<span class="na">protocol</span><span class="pi">:</span> <span class="s">TCP</span>
<span class="na">port</span><span class="pi">:</span> <span class="m">8080</span>
<span class="na">targetPort</span><span class="pi">:</span> <span class="m">8080</span>
<span class="nn">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">networking.k8s.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Ingress</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">rancher-demo-ingress</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">rules</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">http</span><span class="pi">:</span>
<span class="na">paths</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">path</span><span class="pi">:</span> <span class="s">/</span>
<span class="na">pathType</span><span class="pi">:</span> <span class="s">Prefix</span>
<span class="na">backend</span><span class="pi">:</span>
<span class="na">service</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">rancher-demo-srv</span>
<span class="na">port</span><span class="pi">:</span>
<span class="na">number</span><span class="pi">:</span> <span class="m">8080</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">kubectl --kubeconfig=kube_config_cluster.yml apply -f rancher-demo-deployment.yml</code></li>
<li><code class="language-plaintext highlighter-rouge">kubectl --kubeconfig=kube_config_cluster.yml get all -o wide</code></li>
<li>Open <a href="http://r1/">http://r1/</a></li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><a href="https://itnext.io/setup-a-basic-kubernetes-cluster-with-ease-using-rke-a5f3cc44f26f">Setup a basic Kubernetes cluster with ease using RKE</a></li>
<li><a href="https://rancher.com/docs/rancher/v2.x/en/installation/resources/k8s-tutorials/ha-rke/">Setting up a High-availability RKE Kubernetes Cluster</a></li>
<li><a href="https://rancher.com/docs/rancher/v2.x/en/installation/install-rancher-on-k8s/">Install Rancher on a Kubernetes Cluster</a></li>
</ul>Mehdi BizhaniConfig a Kubernetes cluster with RKEDocker to the Point - Part 32020-02-24T00:00:00+00:002020-02-24T00:00:00+00:00http://www.devocative.org/article/tech/docker03<h2 id="introduction">Introduction</h2>
<p>In this post, a DevOps environment is created using Compose. The following picture shows the participating components in
this post:</p>
<ol>
<li>
<p><strong>Git Server</strong> - At its core, a git server is required for developers to push/merge their code. Then, the git server can
call a CI/CD server/agent to execute a <code class="language-plaintext highlighter-rouge">pipeline</code> based on the defined event/hook.
For git server, in this post <a href="https://hub.docker.com/r/gitlab/gitlab-ce"><code class="language-plaintext highlighter-rouge">GitLab CE</code></a> is used.</p>
</li>
<li>
<p><strong>CI/CD Server</strong> - Jenkins is one of the most famous CI/CD applications. However, <a href="https://hub.docker.com/r/gitlab/gitlab-runner"><code class="language-plaintext highlighter-rouge">GitLab Runner</code></a>
is more convenient as it is the integrated and default CI/CD tools for GitLab server.</p>
</li>
<li>
<p><strong>Artifact/Image Repository</strong> - After source code build, the output artifact must be shipped to the server.
Since Docker is used to run applications on servers, the final package must be deployed as Docker image.
So a registry is required in this case to mediate images between servers. <a href="https://hub.docker.com/r/sonatype/nexus3"><code class="language-plaintext highlighter-rouge">Sonatype Nexus 3</code></a> is
a professional repository management tool, and it supports various development/build tools such as maven, npm, and so on.</p>
</li>
<li>
<p><strong>Deployment Servers/Cluster</strong> - At the end of a CD pipeline, a server or cluster is required for executing the container(s).
The CD can be translated into following paradigms:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Continuous Delivery</code> - it targets a pre-production environment, called development/test, to evaluate the release by QA users or customers.</li>
<li><code class="language-plaintext highlighter-rouge">Continuous Deployment</code> - after QA/customer validation and verification cycles, in another pipeline, the final release can be deployed in production.</li>
</ul>
</li>
</ol>
<p><img alt="DevOps" src="/assets/images/docker/DevOps-blueprint.png" class="mx-auto d-block img-thumbnail" /></p>
<h2 id="installation">Installation</h2>
<p>The DevOps environment is installed via Docker Compose. So it is the prerequisite, and you can learn about it in <a href="/article/tech/docker02">Docker to the Point - Part 2</a>.
It is also highly recommended to setup all the containers on the Linux box without any graphical desktop and <code class="language-plaintext highlighter-rouge">NetworkManager</code> must be disabled.
You can use a virtual machine software.</p>
<p><strong><em>Note</em></strong></p>
<blockquote>
<p>During evaluation of this example, some unexpected network issues are encountered.
After some search, due to this [<a href="https://success.docker.com/article/should-you-use-networkmanager">Ref</a>],
Docker strongly recommends that <code class="language-plaintext highlighter-rouge">NetworkManager</code> must be disabled.</p>
</blockquote>
<p>Now, create the directory <code class="language-plaintext highlighter-rouge">DevOps</code> containing the following files:</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">.env</code> - a key-value file for defined variables in other files</li>
<li><code class="language-plaintext highlighter-rouge">init.sh</code> - an initialization script</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose.yml</code> - main Compose file</li>
<li><code class="language-plaintext highlighter-rouge">register-runner.sh</code> - a script to register <code class="language-plaintext highlighter-rouge">gitlab-runner</code> container in GitLab server</li>
</ol>
<p>Then, follow the below steps (commands should be executed in <code class="language-plaintext highlighter-rouge">DevOps</code> directory):</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">chmod +x init.sh && sudo ./init.sh</code></li>
<li><code class="language-plaintext highlighter-rouge">docker-compose up -d</code></li>
<li>Config <a href="#gitlab">GitLab Server</a></li>
<li>Register <a href="#gitlab-runner">GitLab Runner</a></li>
<li>Config <a href="#sonatype-nexus-3">Nexus</a></li>
</ol>
<h3 id="env"><code class="language-plaintext highlighter-rouge">.env</code></h3>
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Exec/DevOps/.env"></script>
</div>
<p><strong>Note:</strong> So based on the above <code class="language-plaintext highlighter-rouge">.env</code>, for the rest of this document <code class="language-plaintext highlighter-rouge">${GITLAB_DOMAIN}</code> means <code class="language-plaintext highlighter-rouge">gitlab.devops.local</code>
and <code class="language-plaintext highlighter-rouge">${NEXUS_DOMAIN}</code> means <code class="language-plaintext highlighter-rouge">nexus.devops.local</code>.</p>
<p><strong>Note</strong>: The above format (double-quoted values) are supported in newer version of <code class="language-plaintext highlighter-rouge">docker-compose</code>.
For this tutorial, version <code class="language-plaintext highlighter-rouge">1.27.4</code> is used.</p>
<!--
**Note**: the `GITLAB_DOMAIN` is assigned to GitLab's container hostname, and it is only useful for GilLab to generate the clone link
as shown in the following picture:
<img alt="git prj" src="/assets/images/docker/DevOps-gitlab-project.png" class="mx-auto d-block img-thumbnail w-75"/>
-->
<h3 id="initsh"><code class="language-plaintext highlighter-rouge">init.sh</code></h3>
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Exec/DevOps/init.sh"></script>
</div>
<h3 id="docker-composeyml"><code class="language-plaintext highlighter-rouge">docker-compose.yml</code></h3>
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Exec/DevOps/docker-compose.yml"></script>
</div>
<h3 id="register-runnersh"><code class="language-plaintext highlighter-rouge">register-runner.sh</code></h3>
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Exec/DevOps/register-runner.sh"></script>
</div>
<h2 id="applications">Applications</h2>
<p>Now execute <code class="language-plaintext highlighter-rouge">docker-compose ps</code> and you must see all containers state <code class="language-plaintext highlighter-rouge">Up</code> (GitLab may take some time to become <code class="language-plaintext highlighter-rouge">healthy</code>).
The following sections describe each application.</p>
<h3 id="traefik">Traefik</h3>
<p>The Traefik reverse proxy is setup due to previous article <a href="/article/tech/docker02/#traefik">Docker to the Point - Part 2</a>:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">http://HOST:8080</code> - Traefik dashboard with basic auth <code class="language-plaintext highlighter-rouge">admin:admin</code></li>
</ul>
<p><img alt="traefik01" src="/assets/images/docker/DevOps-traefik01.png" class="mx-auto d-block img-thumbnail w-75" /></p>
<p><img alt="traefik02" src="/assets/images/docker/DevOps-traefik02.png" class="mx-auto d-block img-thumbnail w-75" /></p>
<h3 id="gitlab">GitLab</h3>
<p>The GitLab server is accessible via <code class="language-plaintext highlighter-rouge">http://${GITLAB_DOMAIN}</code>. At first, it asks you the password for user <code class="language-plaintext highlighter-rouge">root</code>.
Then you can login via user <code class="language-plaintext highlighter-rouge">root</code> and the entered password:</p>
<ul>
<li>GitLab references for Docker (<a href="https://docs.gitlab.com/omnibus/docker/">GitLab Docker</a>)</li>
<li>In <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> lines 10-13, a default config is passed to container via <code class="language-plaintext highlighter-rouge">GITLAB_OMNIBUS_CONFIG</code>:
<ul>
<li><code class="language-plaintext highlighter-rouge">gitlab_rails['registry_enabled'] = false</code> and <code class="language-plaintext highlighter-rouge">registry['enable'] = false</code> - default GitLab registry for Docker is disabled</li>
<li><code class="language-plaintext highlighter-rouge">gitlab_rails['backup_keep_time'] = 604800</code> - for backup, 7-days retention window is set</li>
</ul>
</li>
<li>If you want to disable <strong>Auto DevOps</strong> - Goto <code class="language-plaintext highlighter-rouge">Admin Area > Settings > CI/CD</code></li>
<li>Create a group for all of your projects and set CI/CD variables in that group</li>
<li>The runner container must be added to GitLab server, and you need the token in <code class="language-plaintext highlighter-rouge">registry-runner.sh</code> script.
<ul>
<li>Its token is in <code class="language-plaintext highlighter-rouge">Admin Area > Overview > Runners</code> in the following picture:</li>
</ul>
</li>
</ul>
<p><img alt="gitlab-runner-token" src="/assets/images/docker/DevOps-gitlab-runner-token.png" class="mx-auto d-block img-thumbnail w-75" /></p>
<h3 id="gitlab-runner">GitLab Runner</h3>
<p>The GitLab Runner is the agent for CI/CD of GitLab server. You can register multiple runners for a single GitLab server,
and in this way the CI/CD processes can be spread over multiple nodes and CI/CD becomes scalable.</p>
<ul>
<li>GitLab Runner reference for registration via Docker (<a href="https://docs.gitlab.com/runner/register/index.html#docker">Registering Runners</a>)</li>
<li>In the previous image copy the token and execute <code class="language-plaintext highlighter-rouge">./register-runner.sh TOKEN</code>.
The <code class="language-plaintext highlighter-rouge">gitlab-runner</code> container is registered as a runner as depicted in the following picture:</li>
</ul>
<p><img alt="runner" src="/assets/images/docker/DevOps-gitlab-runner.png" class="mx-auto d-block img-thumbnail w-75" /></p>
<h3 id="sonatype-nexus-3">Sonatype Nexus 3</h3>
<p>Sonatype Nexus is one of the greatest open source repository management software. It supports various repository types such as
<em>Docker registry</em>, <em>Maven</em>, <em>NPM</em>, and so on. This post only presents its Docker registry feature.</p>
<p><strong>Note</strong>: In spite of Docker registry feature in GitLab, Nexus 3 is still the favorite repository management application for
supporting other types of repository such as maven, npm, and so on. If you only need a Docker registry, then GitLab may be more convenient.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">http://NEXUS_DOMAIN</code> - First page of Nexus</li>
<li>Login with user <code class="language-plaintext highlighter-rouge">admin</code> and the password is in <code class="language-plaintext highlighter-rouge">${VOL_BASE_DIR}/nexus/admin.password</code> temp file.</li>
<li>Create a blob store called <code class="language-plaintext highlighter-rouge">docker_hub</code></li>
<li>Create a <strong>proxy</strong> Docker repository for <code class="language-plaintext highlighter-rouge">hub.docker.com</code> without setting any HTTP port, and set blob <code class="language-plaintext highlighter-rouge">docker_hub</code></li>
<li>Create a blob store called <code class="language-plaintext highlighter-rouge">docker_private</code></li>
<li>Create a <strong>hosted</strong> Docker repository on HTTP port <code class="language-plaintext highlighter-rouge">8083</code>, and set blob <code class="language-plaintext highlighter-rouge">docker_private</code></li>
<li>Create a <strong>group</strong> Docker repository on HTTP port <code class="language-plaintext highlighter-rouge">8082</code> and add previous ones to this</li>
<li>Enable <code class="language-plaintext highlighter-rouge">Docker Bearer Token Realm</code></li>
<li><strong>To push an image to the hosted repo</strong>
<ul>
<li><code class="language-plaintext highlighter-rouge">docker login -u USER -p PASS ${NEXUS_DOMAIN}:8083</code></li>
<li><code class="language-plaintext highlighter-rouge">docker push ${NEXUS_DOMAIN}:8083/NAME:VER</code></li>
</ul>
</li>
<li><strong>To pull image from the group repo using proxy</strong> (original from <code class="language-plaintext highlighter-rouge">hub.docker.com</code>)
<ul>
<li><code class="language-plaintext highlighter-rouge">docker pull ${NEXUS_DOMAIN}/IMAGE_ADDRESS_ON_HUB</code>
<ul>
<li><code class="language-plaintext highlighter-rouge">docker pull nginx:1.18 -> docker pull ${NEXUS_DOMAIN}/nginx:1.18</code></li>
<li><code class="language-plaintext highlighter-rouge">docker pull confluentinc/cp-kafka:5.3.1-1 -> docker pull ${NEXUS_DOMAIN}/confluentinc/cp-kafka:5.3.1-1</code></li>
</ul>
</li>
</ul>
</li>
<li><strong>To pull from the group repo</strong>
<ul>
<li><code class="language-plaintext highlighter-rouge">docker pull ${NEXUS_DOMAIN}/NAME:VER</code></li>
<li>Default port is enabled in Traefik for Nexus with <code class="language-plaintext highlighter-rouge">/v2/</code> router (<code class="language-plaintext highlighter-rouge">docker-compose.yml</code> lines 43-51)</li>
</ul>
</li>
</ul>
<p><strong>Note</strong>: The <code class="language-plaintext highlighter-rouge">${NEXUS_DOMAIN}</code> (for pulling) and <code class="language-plaintext highlighter-rouge">${NEXUS_DOMAIN}:8083</code>(for pushing) must be added
as <code class="language-plaintext highlighter-rouge">insecure-registries</code> in <code class="language-plaintext highlighter-rouge">/etc/docker/daemon.json</code> config file.</p>
<h2 id="setup-finalization">Setup Finalization</h2>
<ul>
<li>To backup your Gitlab, create a symlink in <code class="language-plaintext highlighter-rouge">/etc/cron.daily</code> to the following script:</li>
</ul>
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Exec/DevOps/backup-gitlab.sh"></script>
</div>
<ul>
<li>Nexus uses OrientDB for its internal usage and config storage. So it is important to back up its config data.
For this matter, it has a built-in task. You must create one. However, this task just creates files without any retention.
The following script tries to keep some latest files and remove older ones.
So create a symlink in <code class="language-plaintext highlighter-rouge">/etc/cron.daily</code> to the following script:</li>
</ul>
<div>
<script src="/assets/embed.js?target=https://github.com/mbizhani/Dockage/blob/master/Exec/DevOps/trim-nexus-backup.sh"></script>
</div>
<h2 id="using-various-types-of-repos">Using various types of repos</h2>
<p>In this section, using nexus repos for some platforms and languages are described.</p>
<p><strong>Replace <code class="language-plaintext highlighter-rouge">NEXUS_DOMAIN</code> with your configuration in next sections.</strong></p>
<h3 id="maven">Maven</h3>
<ul>
<li>In Nexus, by default, a <code class="language-plaintext highlighter-rouge">maven-public</code> repo is created as a group repository for Maven.</li>
<li>Create/Edit <code class="language-plaintext highlighter-rouge">$HOME/.m2/settings.xml</code> based on following XML content:</li>
</ul>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
</pre></td><td class="rouge-code"><pre><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="nt"><settings</span> <span class="na">xmlns=</span><span class="s">"http://maven.apache.org/SETTINGS/1.1.0"</span>
<span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span>
<span class="na">xsi:schemaLocation=</span><span class="s">"http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd"</span><span class="nt">></span>
<span class="nt"><mirrors></span>
<span class="nt"><mirror></span>
<span class="nt"><id></span>central<span class="nt"></id></span>
<span class="nt"><name></span>central<span class="nt"></name></span>
<span class="nt"><url></span>http://NEXUS_DOMAIN/repository/maven-public/<span class="nt"></url></span>
<span class="nt"><mirrorOf></span>*<span class="nt"></mirrorOf></span>
<span class="nt"></mirror></span>
<span class="nt"></mirrors></span>
<span class="nt"></settings></span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="go">Go</h3>
<ul>
<li>Create <code class="language-plaintext highlighter-rouge">go-group</code> repo as a group repo for Go</li>
<li>Just define the environment variable <code class="language-plaintext highlighter-rouge">GOPROXY="http://NEXUS_DOMAIN/repository/go-group"</code></li>
</ul>
<h3 id="docker">Docker</h3>
<ul>
<li>Edit <code class="language-plaintext highlighter-rouge">/etc/docker/daemon.json</code>, and add or update <code class="language-plaintext highlighter-rouge">insecure-registries</code> like the following line
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="nl">"insecure-registries"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"NEXUS_DOMAIN"</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">sudo systemctl restart docker</code></li>
</ul>
<h3 id="npm">NPM</h3>
<ul>
<li>Create <code class="language-plaintext highlighter-rouge">npm-group</code> repo as a group repo for NPM</li>
<li>Create or update <code class="language-plaintext highlighter-rouge">.npmrc</code> file in <code class="language-plaintext highlighter-rouge">$HOME</code> or project root directory with line <code class="language-plaintext highlighter-rouge">registry=http://NEXUS_DOMAIN/repository/npm-group/</code></li>
</ul>Mehdi BizhaniCreate a DevOps Environment by Docker ComposeDocker to the Point - Part 22020-02-13T00:00:00+00:002020-02-13T00:00:00+00:00http://www.devocative.org/article/tech/docker02<h2 id="introduction">Introduction</h2>
<p><code class="language-plaintext highlighter-rouge">docker-compose</code> is a tool for defining and running multi-container Docker applications on a single node/host.
With Compose, you use a YAML file to configure your application’s services. Then, with a single command,
you create and start all the services from your configuration (<a href="https://docs.docker.com/compose">REF</a>).</p>
<p>For installation, some linux distributions such as Debian provide a proper package with bash completion.
Another solution is to download the single command-file from its GitHub using the following command (<a href="https://docs.docker.com/compose/install/">REF</a>):</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="nb">sudo </span>curl <span class="nt">-L</span> <span class="s2">"https://github.com/docker/compose/releases/download/1.27.4/docker-compose-</span><span class="si">$(</span><span class="nb">uname</span> <span class="nt">-s</span><span class="si">)</span><span class="s2">-</span><span class="si">$(</span><span class="nb">uname</span> <span class="nt">-m</span><span class="si">)</span><span class="s2">"</span> <span class="nt">-o</span> /usr/local/bin/docker-compose
<span class="nb">sudo chmod</span> +x /usr/local/bin/docker-compose
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="bash-completion">Bash Completion</h3>
<p>Make sure bash completion is installed.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre><span class="nb">sudo </span>curl <span class="nt">-L</span> https://raw.githubusercontent.com/docker/compose/1.27.4/contrib/completion/bash/docker-compose <span class="nt">-o</span> /etc/bash_completion.d/docker-compose
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="compose-features">Compose Features</h3>
<ul>
<li><strong>Multiple isolated environments on a single host</strong>
<ul>
<li>Uses a project name to isolate environments from each other</li>
<li>The default project name is the basename of the project directory</li>
</ul>
</li>
<li><strong>Preserve volume data when containers are created</strong>
<ul>
<li>When <code class="language-plaintext highlighter-rouge">docker-compose up</code> runs, if it finds any containers from previous runs, it copies the volumes from the old container to the new container</li>
</ul>
</li>
<li><strong>Only recreate containers that have changed</strong>
<ul>
<li>Compose caches the configuration used to create a container. When you restart a service that has not changed, Compose re-uses the existing containers. Re-using containers means that you can make changes to your environment very quickly.</li>
</ul>
</li>
<li><strong>Variables and moving a composition between environments</strong>
<ul>
<li>Variables can be used in the Compose file</li>
<li>Variables from the shell environment such as <code class="language-plaintext highlighter-rouge">${USER}</code> can be addressed</li>
<li>Variables also can be defined in a <code class="language-plaintext highlighter-rouge">.env</code> file beside the Compose file in <code class="language-plaintext highlighter-rouge">key=value</code> format</li>
<li>Variables can be used in the following formats
<ul>
<li><code class="language-plaintext highlighter-rouge">${VARIABLE}</code> or <code class="language-plaintext highlighter-rouge">$VARIABLE</code> - simple</li>
<li><code class="language-plaintext highlighter-rouge">${VARIABLE:-default}</code> evaluates to <code class="language-plaintext highlighter-rouge">default</code> if <code class="language-plaintext highlighter-rouge">VARIABLE</code> is <em>unset</em> or <em>empty</em> in the environment.</li>
<li><code class="language-plaintext highlighter-rouge">${VARIABLE-default}</code> evaluates to <code class="language-plaintext highlighter-rouge">default</code> only if <code class="language-plaintext highlighter-rouge">VARIABLE</code> is <em>unset</em> in the environment.</li>
<li><code class="language-plaintext highlighter-rouge">${VARIABLE:?err}</code> exits with an error message showing <code class="language-plaintext highlighter-rouge">err</code> if <code class="language-plaintext highlighter-rouge">VARIABLE</code> is <em>unset</em> or <em>empty</em> in the environment.</li>
<li><code class="language-plaintext highlighter-rouge">${VARIABLE?err}</code> exits with an error message showing <code class="language-plaintext highlighter-rouge">err</code> if <code class="language-plaintext highlighter-rouge">VARIABLE</code> is <em>unset</em> in the environment.</li>
</ul>
</li>
</ul>
</li>
</ul>
<h2 id="the-env-file">The <code class="language-plaintext highlighter-rouge">.env</code> file</h2>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
</pre></td><td class="rouge-code"><pre><span class="nv">VOL_BASE_DIR</span><span class="o">=</span>/opt/docker_vols
<span class="nv">TRAEFIK_VER</span><span class="o">=</span>2.1.3
<span class="nv">PORTAINER_VER</span><span class="o">=</span>1.23.0
<span class="nv">BUSYBOX_VER</span><span class="o">=</span>1.31.1-glibc
<span class="nv">NGINX_VER</span><span class="o">=</span>1.17.2
<span class="nv">CONFLUENT_VER</span><span class="o">=</span>5.3.1-1
<span class="nv">KAFDROP_VER</span><span class="o">=</span>3.23.0
<span class="nv">REDIS_VER</span><span class="o">=</span>5.0.7
<span class="nv">REDIS_PASSWD</span><span class="o">=</span>ReDis
<span class="nv">MYSQL_VER</span><span class="o">=</span>8.0.19
<span class="nv">MYSQL_PASSWS</span><span class="o">=</span>rOOt
<span class="nv">ADMINER_VER</span><span class="o">=</span>4.7.6-standalone
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="first-sample">First Sample</h2>
<p>In the following <code class="language-plaintext highlighter-rouge">yml</code>, some images are used to illustrate various <code class="language-plaintext highlighter-rouge">docker-compose</code> features alongside the above <code class="language-plaintext highlighter-rouge">.env</code> file.
Before execution, run <code class="language-plaintext highlighter-rouge">docker network create main_net</code>.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
</pre></td><td class="rouge-code"><pre><span class="na">version</span><span class="pi">:</span> <span class="s1">'</span><span class="s">3.6'</span>
<span class="na">services</span><span class="pi">:</span>
<span class="na">traefik</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">traefik:${TRAEFIK_VER:?ver}</span>
<span class="na">command</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--api.insecure"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--providers.docker"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--providers.docker.exposedbydefault=false"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--entrypoints.http.address=:80"</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">80:80</span>
<span class="pi">-</span> <span class="s">8080:8080</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/var/run/docker.sock:/var/run/docker.sock</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">web</span>
<span class="na">portainer</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">portainer/portainer:${PORTAINER_VER:-latest}</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">-H unix:///var/run/docker.sock</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.enable=true"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.portainer.rule=PathPrefix(`/portainer`)"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.portainer.middlewares=portainerRedir,portainerPStrip"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.portainerPStrip.stripprefix.prefixes=/portainer"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.portainerRedir.redirectregex.regex=^(.*)/portainer$$"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.portainerRedir.redirectregex.replacement=$${1}/portainer/"</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/var/run/docker.sock:/var/run/docker.sock</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/Portainer:/data</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">web</span>
<span class="na">nginx</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">nginx:${NGINX_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.enable=true"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.nginx.rule=PathPrefix(`/web`)"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.nginx.middlewares=nginxPStrip"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.nginxPStrip.stripprefix.prefixes=/web"</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">web</span>
<span class="na">busybox</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">busybox:${BUSYBOX_VER:-latest}</span>
<span class="na">hostname</span><span class="pi">:</span> <span class="s">busybox</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">tty</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">ext</span>
<span class="pi">-</span> <span class="s">web</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="na">web</span><span class="pi">:</span>
<span class="na">ext</span><span class="pi">:</span>
<span class="na">external</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">main_net</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Copy sample in <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> and save it in a directory called <code class="language-plaintext highlighter-rouge">Example</code> (just to simplify the sample), so the <code class="language-plaintext highlighter-rouge">example</code> is the project name.
Also create <code class="language-plaintext highlighter-rouge">.env</code> in this directory. Now run <code class="language-plaintext highlighter-rouge">docker-compose up -d</code>:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">example_web</code> docker network is created</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose ps</code> - four containers are created
<ul>
<li><code class="language-plaintext highlighter-rouge">example_busybox_1</code></li>
<li><code class="language-plaintext highlighter-rouge">example_nginx_1</code></li>
<li><code class="language-plaintext highlighter-rouge">example_portainer_1</code></li>
<li><code class="language-plaintext highlighter-rouge">example_traefik_1</code></li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose logs -f traefik</code> - watch the <code class="language-plaintext highlighter-rouge">traefik</code> service log</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose exec busybox sh</code> - the <code class="language-plaintext highlighter-rouge">sh</code> program is executed in the <code class="language-plaintext highlighter-rouge">busybox</code> service. Now execute following commands:
<ul>
<li><code class="language-plaintext highlighter-rouge">hostname</code> - result: <code class="language-plaintext highlighter-rouge">busybox</code> (line 51)</li>
<li><code class="language-plaintext highlighter-rouge">ip a</code> - result: 3 interfaces - <code class="language-plaintext highlighter-rouge">lo</code>, one for <code class="language-plaintext highlighter-rouge">web</code> and the other for <code class="language-plaintext highlighter-rouge">ext</code></li>
<li><code class="language-plaintext highlighter-rouge">ping traefik</code> and <code class="language-plaintext highlighter-rouge">ping example_traefik_1</code> show the same result. The <code class="language-plaintext highlighter-rouge">traefik</code> is set as alias for container <code class="language-plaintext highlighter-rouge">example_traefik_1</code>
(<code class="language-plaintext highlighter-rouge">docker container inspect -f '{{ .NetworkSettings.Networks.example_web.Aliases }}' example_traefik_1</code>)</li>
<li><code class="language-plaintext highlighter-rouge">telnet traefik 80</code>, <code class="language-plaintext highlighter-rouge">telnet nginx 80</code>, and <code class="language-plaintext highlighter-rouge">telnet portainer 9000</code> - you can telnet the services</li>
</ul>
</li>
<li>Traefik is the reverse proxy. Access the following URLs:
<ul>
<li><a href="http://localhost:8080">http://localhost:8080</a> - Traefik dashboard (enabled by <code class="language-plaintext highlighter-rouge">--api.insecure=true</code>)</li>
<li><a href="http://localhost/web">http://localhost/web</a> - Nginx default page</li>
<li><a href="http://localhost/portainer">http://localhost/portainer</a> - Portainer application</li>
</ul>
</li>
<li>For volumes, <code class="language-plaintext highlighter-rouge">./DIR</code> can be used to create a <code class="language-plaintext highlighter-rouge">DIR</code> in current directory on host (line 34 if <code class="language-plaintext highlighter-rouge">VOL_BASE_DIR</code> is undefined).
<ul>
<li><strong>Note</strong>: relative volume is only possible in <code class="language-plaintext highlighter-rouge">docker-compose</code> and later it is translated into absolute directory
(<code class="language-plaintext highlighter-rouge">docker container inspect -f '{{ .HostConfig.Binds }}' example_portainer_1</code>)</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose stop</code> - stop all running containers</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose start</code> - start all stopped containers</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose down</code> - stop and remove all containers and then remove <code class="language-plaintext highlighter-rouge">web</code> network (owned networks by the compose file)</li>
</ul>
<blockquote>
<p><strong>Note</strong></p>
<p>You can rerun <code class="language-plaintext highlighter-rouge">docker-compose up -d</code> every time you change your <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>, and it detects the modified services and only rebuild those services.</p>
</blockquote>
<h2 id="services">Services</h2>
<h3 id="traefik">Traefik</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
</pre></td><td class="rouge-code"><pre> <span class="na">traefik</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">traefik:${TRAEFIK_VER:-latest}</span>
<span class="na">command</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--providers.docker"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--providers.docker.exposedbydefault=false"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--entrypoints.http.address=:80"</span>
<span class="c1"># Dashboard</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--api"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--entrypoints.traefik.address=:8080"</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="c1"># Dashboard</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.enable=true"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.api.entrypoints=traefik"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.api.rule=PathPrefix(`/`)"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.api.service=api@internal"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.api.middlewares=dashAuth,dashRdir"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.dashAuth.basicauth.users=admin:$$apr1$$PSIlVhdx$$Np60QsO9D2zneaUjWdaqA0"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.dashRdir.redirectregex.regex=^(http://[^:/]+(:</span><span class="se">\\</span><span class="s">d+)?)(/|/dashboard)$$"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.dashRdir.redirectregex.replacement=$${1}/dashboard/"</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">80:80</span>
<span class="pi">-</span> <span class="s">8080:8080</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/var/run/docker.sock:/var/run/docker.sock</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Lines 8 enables dashboard and lines 9 create an <code class="language-plaintext highlighter-rouge">entrypoint</code> for the dashboard on port <code class="language-plaintext highlighter-rouge">8080</code></li>
<li>Lines 13-20 configure dashboard with basic-auth and redirection to <code class="language-plaintext highlighter-rouge">/dashbiard</code>
<ul>
<li>Line 18 defines basic-auth middleware with user <code class="language-plaintext highlighter-rouge">admin</code> (<a href="https://docs.traefik.io/v2.0/middlewares/basicauth/">REF</a>)
<ul>
<li>Using <code class="language-plaintext highlighter-rouge">echo $(htpasswd -nb user password) | sed -e s/\\$/\\$\\$/g</code> command to generate the expression</li>
<li><strong>Note</strong>: <code class="language-plaintext highlighter-rouge">htpasswd</code> is installed via <code class="language-plaintext highlighter-rouge">apache2-utils</code> package on Debian</li>
</ul>
</li>
<li>Lines 19 and 20 enables the redirection to <code class="language-plaintext highlighter-rouge">/dashboard</code>
<ul>
<li>In line 20, the <code class="language-plaintext highlighter-rouge">${1}</code> refers to the group defined in the regex in previous line</li>
</ul>
</li>
</ul>
</li>
</ul>
<h3 id="portainer">Portainer</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
</pre></td><td class="rouge-code"><pre> <span class="na">portainer</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">portainer/portainer:${PORTAINER_VER:-latest}</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">-H unix:///var/run/docker.sock</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.enable=true"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.portainer.rule=PathPrefix(`/portainer`)"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.routers.portainer.middlewares=portainerRedir,portainerPStrip"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.portainerPStrip.stripprefix.prefixes=/portainer"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.portainerRedir.redirectregex.regex=^(.*)/portainer$$"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">traefik.http.middlewares.portainerRedir.redirectregex.replacement=$${1}/portainer/"</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">/var/run/docker.sock:/var/run/docker.sock</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/Portainer:/data</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="kafka">Kafka</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
</pre></td><td class="rouge-code"><pre><span class="na">version</span><span class="pi">:</span> <span class="s1">'</span><span class="s">3.6'</span>
<span class="na">services</span><span class="pi">:</span>
<span class="na">zookeeper</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">confluentinc/cp-zookeeper:${CONFLUENT_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">ZOOKEEPER_CLIENT_PORT</span><span class="pi">:</span> <span class="m">2181</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/zookeeper/data:/var/lib/zookeeper/data</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/zookeeper/log:/var/lib/zookeeper/log</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
<span class="na">kafka</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">confluentinc/cp-kafka:${CONFLUENT_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">depends_on</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">zookeeper</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">KAFKA_ZOOKEEPER_CONNECT</span><span class="pi">:</span> <span class="s">zookeeper:2181</span>
<span class="na">KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR</span><span class="pi">:</span> <span class="m">1</span>
<span class="c1"># Multi-Net Config</span>
<span class="na">KAFKA_LISTENERS</span><span class="pi">:</span> <span class="s">INSIDE://:9092,OUTSIDE://:29092</span>
<span class="na">KAFKA_ADVERTISED_LISTENERS</span><span class="pi">:</span> <span class="s">INSIDE://kafka:9092,OUTSIDE://localhost:29092</span>
<span class="na">KAFKA_LISTENER_SECURITY_PROTOCOL_MAP</span><span class="pi">:</span> <span class="s">INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT</span>
<span class="na">KAFKA_INTER_BROKER_LISTENER_NAME</span><span class="pi">:</span> <span class="s">INSIDE</span>
<span class="c1"># All-Same-Net Simple Config</span>
<span class="c1"># KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/kafka/data:/var/lib/kafka/data</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">29092:29092</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
<span class="na">kafdrop</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">obsidiandynamics/kafdrop:${KAFDROP_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">depends_on</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">kafka</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">JVM_OPTS</span><span class="pi">:</span> <span class="s2">"</span><span class="s">-Xms16M</span><span class="nv"> </span><span class="s">-Xmx48M"</span>
<span class="na">KAFKA_BROKERCONNECT</span><span class="pi">:</span> <span class="s">kafka:9092</span>
<span class="na">SERVER_PORT</span><span class="pi">:</span> <span class="m">9090</span>
<span class="na">SERVER_SERVLET_CONTEXTPATH</span><span class="pi">:</span> <span class="s2">"</span><span class="s">/"</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">9090:9090</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Kafka Networking Config (<a href="https://docs.confluent.io/current/kafka/multi-node.html">REF</a>):
<ul>
<li>All containers attached to <code class="language-plaintext highlighter-rouge">net</code> network can connect to <code class="language-plaintext highlighter-rouge">kafka</code> server via port <code class="language-plaintext highlighter-rouge">9092</code> through <code class="language-plaintext highlighter-rouge">INSIDE://:9092</code> listener advertised by <code class="language-plaintext highlighter-rouge">INSIDE://kafka:9092</code>, such as <code class="language-plaintext highlighter-rouge">kafdrop</code></li>
<li>Other clients on the host (localhost) can connect to kafka server via port <code class="language-plaintext highlighter-rouge">29092</code> through <code class="language-plaintext highlighter-rouge">OUTSIDE://:29092</code> listener advertised by <code class="language-plaintext highlighter-rouge">OUTSIDE://localhost:29092</code></li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose exec kafka bash</code> - Check Kafka Server
<ul>
<li>Topic commands
<ul>
<li><code class="language-plaintext highlighter-rouge">kafka-topics --zookeeper zookeeper:2181 --list</code></li>
<li><code class="language-plaintext highlighter-rouge">kafka-topics --zookeeper zookeeper:2181 --create --partitions 3 --replication-factor 1 --if-not-exists --topic bar</code></li>
<li><code class="language-plaintext highlighter-rouge">kafka-topics --zookeeper zookeeper:2181 --describe --topic bar</code></li>
</ul>
</li>
<li>Messaging commands
<ul>
<li><code class="language-plaintext highlighter-rouge">seq -f "MSG_ID: %03g" 20 | kafka-console-producer --broker-list localhost:9092 --topic bar && echo 'Messages Are Sent'</code></li>
<li><code class="language-plaintext highlighter-rouge">kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic bar --max-messages 10</code></li>
<li>Note: the order of messages are based on partitions</li>
</ul>
</li>
</ul>
</li>
<li>For Clustered Deployment on Docker check [<a href="https://docs.confluent.io/5.0.0/installation/docker/docs/installation/clustered-deployment.html">REF</a>]</li>
</ul>
<h3 id="redis">Redis</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre> <span class="na">redis</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">redis:${REDIS_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">command</span><span class="pi">:</span> <span class="s">redis-server --requirepass ${REDIS_PASSWD:-redis} --appendonly yes</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">6379:6379</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/redis:/data</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">docker-compose exec redis redis-cli -a ReDis</code> - Interactive Redis CLI</li>
<li>Syntax: <code class="language-plaintext highlighter-rouge">redis-cli [-h HOST] [-p PORT] [-a PASSWORD]</code>
-Commands
<ul>
<li><code class="language-plaintext highlighter-rouge">ping</code></li>
<li><code class="language-plaintext highlighter-rouge">keys *</code> - return all keys</li>
<li><code class="language-plaintext highlighter-rouge">type KEY</code> - check type of value</li>
<li><code class="language-plaintext highlighter-rouge">del KEY</code></li>
<li>Getting values - Redis supports multiple types of data (<a href="https://redis.io/topics/data-types-intro">REF</a>)
<ul>
<li><code class="language-plaintext highlighter-rouge">get KEY</code> - string</li>
<li><code class="language-plaintext highlighter-rouge">smembers KEY</code> - set</li>
<li><code class="language-plaintext highlighter-rouge">lrange KEY START END</code> - list</li>
</ul>
</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose exec redis redis-cli -a ReDis --stat [-i INTERVAL]</code> - Continuous stats mode</li>
</ul>
<h3 id="mysql">MySQL</h3>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
</pre></td><td class="rouge-code"><pre> <span class="na">mysql</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">mysql:${MYSQL_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">3306:3306</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">MYSQL_ROOT_PASSWORD</span><span class="pi">:</span> <span class="s">${MYSQL_PASSWS:-root}</span>
<span class="na">volumes</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">${VOL_BASE_DIR:-.}/mysql:/var/lib/mysql</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
<span class="na">adminer</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">adminer:${ADMINER_VER:-latest}</span>
<span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">8080:8080</span>
<span class="na">environment</span><span class="pi">:</span>
<span class="na">ADMINER_DESIGN</span><span class="pi">:</span> <span class="s">ng9</span>
<span class="na">networks</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">net</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><a href="https://mysqlserverteam.com/whats-new-in-mysql-8-0-generally-available/">What’s New in MySQL 8.0?</a></li>
<li><code class="language-plaintext highlighter-rouge">docker-compose exec mysql -uroot -prOOt</code> - Run MySql CLI
<ul>
<li>Syntax: <code class="language-plaintext highlighter-rouge">mysql -uUSER -pPASSWORD</code> (no space!)</li>
<li><code class="language-plaintext highlighter-rouge">show databases;</code></li>
<li><code class="language-plaintext highlighter-rouge">create database DB;</code></li>
<li><code class="language-plaintext highlighter-rouge">use DB</code></li>
<li><code class="language-plaintext highlighter-rouge">show tables;</code></li>
<li><code class="language-plaintext highlighter-rouge">desc TABLE;</code></li>
<li><code class="language-plaintext highlighter-rouge">create user USER identified by 'PASSWORD';</code></li>
<li><code class="language-plaintext highlighter-rouge">grant all privileges on DB.* to USER;</code></li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">docker-compose exec mysql mysqldump -uUSER -pPASSWORD DB | grep -v Warning | gzip > DB_$(date +'%Y-%m-%d_%H-%M-%S').sql.gz</code></li>
<li><code class="language-plaintext highlighter-rouge">zcat DB.sql.gz | docker-compose exec -T mysql mysql -uUSER -pPASSWORD DB</code>
<ul>
<li><strong>Note</strong>: By default <code class="language-plaintext highlighter-rouge">docker-compose exec</code> allocates a tty, so <code class="language-plaintext highlighter-rouge">-T</code> disables pseudo-tty allocation.</li>
</ul>
</li>
</ul>Mehdi BizhaniUsing Docker Compose to run multiple containersDocker to the Point - Part 12019-03-29T00:00:00+00:002019-03-29T00:00:00+00:00http://www.devocative.org/article/tech/docker01<h2 id="introduction">Introduction</h2>
<p>Every process on your OS is executed from an application code and each process can interfere in other process’s resources.
The OS or the VM (if there is any) can provide some security and isolation mechanisms,
however it can be different for each OS type and impose difficult configuration.</p>
<p><em>Docker</em> provides a layered architecture for contents of the application(s), called an <em>Image</em>,
and a <em>virtual environment</em>, called a <em>Container</em>, for running process(s) of the application(s). This <em>virtual environment</em> is isolated by</p>
<ul>
<li>memory</li>
<li><strong>storage</strong> (similar to a <em>hypervisor</em>)</li>
<li><strong>network</strong> (similar to a <em>hypervisor</em>)</li>
</ul>
<p>The image has a layered architecture, each layer contains the differences between the preceding layer and the current layer, and
on top of the layers, there is a writable layer (the current one) which is called the <em>container layer</em><sup id="fnref:docker_layer" role="doc-noteref"><a href="#fn:docker_layer" class="footnote" rel="footnote">1</a></sup>.</p>
<p>As mentioned above, there are some similarities between <strong>Docker and a hypervisor</strong> in providing isolated resources.
The most important difference is their focus and application. If you want a <strong>full-fledged OS</strong>, the hypervisor
is the solution, and if you want a <strong>light-weight isolated virtual environment for your process(s)</strong>, Docker is the answer.
For example, if you want to run a <a href="#mysql">MySQL</a> server, Docker is the best choice, and if you want to execute a simulation app with
GUI, virtualization is the only choice.</p>
<p>The following class diagram illustrates the overall parts involved in Docker installed on your box.
The associations show the relations between these parts and multiplicity of each association end highlights the detail of that end in the relation.</p>
<p><img alt="DockerClassDiagram" src="/assets/images/docker/DockerCD.png" class="mx-auto d-block border border-dark" /></p>
<p>In a simple story, a container is started by the <code class="language-plaintext highlighter-rouge">docker daemon</code>, initiated by the <code class="language-plaintext highlighter-rouge">docker</code> client (command):</p>
<ul>
<li>At first, the image is downloaded from the registry if not already downloaded</li>
<li>Docker can create one or another independent container from one image</li>
<li>During container startup, Docker executes the command(s) provided in the image or passed as input parameters</li>
<li>Docker initializes the container’s network interface and attaches it to one of its virtual networks due to startup parameters (<a href="#nginx--jenkins--nexus">example</a>),
and even open a port on the host binding it to one of container’s internal process</li>
<li>Docker may mount a file/directory to its internal file/directory due to startup parameters</li>
</ul>
<p>Container(s) can be created and started via one of following ways</p>
<table>
<thead>
<tr>
<th>App</th>
<th>Host/Node</th>
<th>Container</th>
</tr>
</thead>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker</code></td>
<td>single</td>
<td>various unrelated containers</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker-compose</code></td>
<td>single</td>
<td>using single config file for multiple related containers or same-image containers for load balancing</td>
</tr>
<tr>
<td>Docker Swarm or Kubernetes</td>
<td>multiple</td>
<td>large deployment and centralized management for lots of containers</td>
</tr>
</tbody>
</table>
<p>This post only uses <code class="language-plaintext highlighter-rouge">docker</code>.</p>
<h2 id="installation">Installation</h2>
<p>Docker is presented in two editions: <strong>CE</strong> (community edition) and <strong>EE</strong> (enterprise edition). Most of the time, people use its CE and also in the case of this tutorial.</p>
<p>The installation is different due to your OS. So the best reference is the Docker site.</p>
<ul>
<li><a href="https://docs.docker.com/install/linux/docker-ce/debian/">Debian</a></li>
<li><a href="https://docs.docker.com/install/linux/docker-ce/ubuntu/">Ubuntu</a></li>
<li><a href="https://docs.docker.com/docker-for-windows/install/">Windows</a></li>
<li><strong>Note</strong>: There is a general installation script for Linux distributions - <code class="language-plaintext highlighter-rouge">sudo curl -sSL https://get.docker.com | sh</code></li>
</ul>
<p>After installation</p>
<ul>
<li>In Linux, by default only <code class="language-plaintext highlighter-rouge">root</code> user can execute <code class="language-plaintext highlighter-rouge">docker</code> command. Adding group <code class="language-plaintext highlighter-rouge">docker</code> to a user’s groups, the user has access to Docker.
So execute <code class="language-plaintext highlighter-rouge">usermod -a -G docker USER</code>.</li>
<li>Executing <code class="language-plaintext highlighter-rouge">ifconfig</code> in your Linux box, <code class="language-plaintext highlighter-rouge">docker0</code> must be in the list of network interfaces.
<img src="/assets/images/docker/DockerIfconfig.png" alt="DockerIfconfig" /></li>
</ul>
<h2 id="docker-cli">Docker CLI</h2>
<p>After installing Docker, the <code class="language-plaintext highlighter-rouge">docker</code> command must be in your path and can be executed from your OS CLI app.</p>
<p>Following table shows some common usages of <code class="language-plaintext highlighter-rouge">docker</code> command.</p>
<table>
<thead>
<tr>
<th>Command</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker version</code></td>
<td>Show version information of installed Docker client and server</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker image ls</code></td>
<td>List of image(s) on your host</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker image inspect IMAGE</code></td>
<td>Display detailed information on the image</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker image pull IMAGE</code></td>
<td>Download the image from a registry</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker image rm IMAGE</code></td>
<td>Remove the image from local</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker image save IMAGE -o FILE</code></td>
<td>Save one or more images to a tar archive</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker image load -i FILE</code></td>
<td>Load an image from a tar archive</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container ls</code></td>
<td>List running containers</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container ls -a</code></td>
<td>List all containers</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container stats</code></td>
<td>Display a live stream of container(s) resource usage statistics</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container run IMAGE</code></td>
<td>Download the image, if not already downloaded, and start a container from it</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container top CONTAINER</code></td>
<td>Display the running processes of a container</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container inspect CONTAINER</code></td>
<td>Display detailed information for a container</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container exec CONTAINER</code></td>
<td>Execute a command inside the container</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">docker container logs CONTAINER</code></td>
<td>Display logs of the container</td>
</tr>
</tbody>
</table>
<ul>
<li><code class="language-plaintext highlighter-rouge">IMAGE</code> - each image has a specific name to be addressed on a registry. On Docker Hub, the simple form of <code class="language-plaintext highlighter-rouge">IMAGE</code> is:
<blockquote>
<p><code class="language-plaintext highlighter-rouge">REPONAME[:TAG]</code></p>
</blockquote>
<p><code class="language-plaintext highlighter-rouge">TAG</code> is optional and its default value is <code class="language-plaintext highlighter-rouge">latest</code>. However, defining a specific tag is highly recommended to assure you about the version of the used image.
Some examples are <code class="language-plaintext highlighter-rouge">mysql:5.7</code>, <code class="language-plaintext highlighter-rouge">openjdk:8u191-alpine</code>, <code class="language-plaintext highlighter-rouge">redis:5.0.4-stretch</code>, and etc.</p>
<p>The general form of <code class="language-plaintext highlighter-rouge">IMAGE</code> on a registry server is<sup id="fnref:docker_fqn" role="doc-noteref"><a href="#fn:docker_fqn" class="footnote" rel="footnote">2</a></sup>:</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">[[HOSTNAME[:PORT]/]USERNAME/]REPONAME[:TAG]</code></p>
</blockquote>
</li>
<li><code class="language-plaintext highlighter-rouge">CONTAINER</code> refers to the name or id of a container</li>
</ul>
<p><strong>Note:</strong> as stated in the previous table, the following command shows three popular information of an <code class="language-plaintext highlighter-rouge">IMAGE</code>.
In <a href="#mysql">MySQL</a> section, details of the following command is described as an example.</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker image inspect -f '{{println "VOL =" .Config.Volumes}}{{println "ENV =" .Config.Env}}{{println "PORTS =" .ContainerConfig.ExposedPorts}}' IMAGE</code></p>
</blockquote>
<h2 id="configuration">Configuration</h2>
<p>You can config docker daemon via <code class="language-plaintext highlighter-rouge">/etc/docker/daemon.json</code>. A common snippet is as follows:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
</pre></td><td class="rouge-code"><pre><span class="p">{</span><span class="w">
</span><span class="nl">"insecure-registries"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"REG1[:PORT]"</span><span class="p">,</span><span class="w"> </span><span class="s2">"REG2[:PORT]"</span><span class="p">],</span><span class="w">
</span><span class="nl">"log-driver"</span><span class="p">:</span><span class="w"> </span><span class="s2">"json-file"</span><span class="p">,</span><span class="w">
</span><span class="nl">"log-opts"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"max-size"</span><span class="p">:</span><span class="w"> </span><span class="s2">"100m"</span><span class="p">,</span><span class="w">
</span><span class="nl">"max-file"</span><span class="p">:</span><span class="w"> </span><span class="s2">"3"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="real-examples">Real Examples</h2>
<p>Lets start real and practical examples.</p>
<h3 id="mysql">MySQL</h3>
<ul>
<li><strong>Reference</strong> <a href="https://hub.docker.com/_/mysql">Docker Hub</a></li>
<li><strong>Image Information</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker image inspect -f '{{println "VOL =" .Config.Volumes}}{{println "ENV =" .Config.Env}}{{println "PORTS =" .ContainerConfig.ExposedPorts}}' mysql:5.7</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">VOL = map[/var/lib/mysql:{}]</code></td>
<td>it has a volume entry on <code class="language-plaintext highlighter-rouge">/var/lib/mysql</code> inside the container</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">ENV = [...]</code></td>
<td>list of defined environment variables</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">PORTS = map[3306/tcp:{} 33060/tcp:{}]</code></td>
<td>exposed ports to be bound on the host</td>
</tr>
</tbody>
</table>
</li>
<li><strong>Starting container</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container run -d -v /opt/docker/mysql:/var/lib/mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=root --restart=always --name MySQL mysql:5.7</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">-d</code></td>
<td>the container is detached and executed in background</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">-v /opt/docker/mysql:/var/lib/mysql</code></td>
<td><em>HOST</em><code class="language-plaintext highlighter-rouge">:</code><em>CONTAINER</em> for mapping volume, so <code class="language-plaintext highlighter-rouge">/var/lib/mysql</code> directory of container is mapped on <code class="language-plaintext highlighter-rouge">/opt/docker/mysql</code> directory of host</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">-p 3306:3306</code></td>
<td><em>HOST</em><code class="language-plaintext highlighter-rouge">:</code><em>CONTAINER</em> for binding ports</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">-e MYSQL_ROOT_PASSWORD=root</code></td>
<td>assign value <code class="language-plaintext highlighter-rouge">root</code> to environment variable <code class="language-plaintext highlighter-rouge">MYSQL_ROOT_PASSWORD</code>, so the password of <em>root</em> user in mysql is set to <code class="language-plaintext highlighter-rouge">root</code></td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">--restart=always</code></td>
<td>on next host reboot or power-on, the container starts automatically (like services in Windows)</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">--name MySQL</code></td>
<td><code class="language-plaintext highlighter-rouge">MySQL</code> is assigned as the name to this container to be addressed easily (<code class="language-plaintext highlighter-rouge">CONTAINER</code> in above table, is this name).<br /><strong>note</strong>: this name also used as network name for the container</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">mysql:5.7</code></td>
<td>the <code class="language-plaintext highlighter-rouge">IMAGE</code> reference</td>
</tr>
</tbody>
</table>
</li>
<li><strong>Calling <em>mysql</em> client CLI</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container exec -it MySQL mysql -uroot -proot</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">-it</code> (i.e. <code class="language-plaintext highlighter-rouge">-i -t</code>)</td>
<td><code class="language-plaintext highlighter-rouge">-i</code>: Keep STDIN open even if not attached <code class="language-plaintext highlighter-rouge">-t</code>: Allocate a pseudo-TTY</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">mysql -uroot -proot</code></td>
<td>the command and its arguments executed inside the container</td>
</tr>
</tbody>
</table>
<!-- TODO: mysql client tutorial -->
</li>
<li><strong>Watching logs of mysql server</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container logs -f MySQL</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">-f</code></td>
<td>it works like <code class="language-plaintext highlighter-rouge">tail -f</code> in Linux</td>
</tr>
</tbody>
</table>
</li>
</ul>
<h3 id="redis">Redis</h3>
<ul>
<li><strong>Reference</strong> <a href="https://hub.docker.com/_/redis">Docker Hub</a></li>
<li><strong>Starting container</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container run -d -v /opt/docker/redis:/data -p 6379:6379 --restart=always --name Redis redis:5.0.4 redis-server --requirepass PASSWORD</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">redis-server --requirepass PASSWORD</code></td>
<td>passing custom command to be executed on container startup instead of its default one</td>
</tr>
</tbody>
</table>
</li>
<li><strong>Calling <em>redis-cli</em> client</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container exec -it Redis redis-cli -a PASSWORD</code></p>
</blockquote>
<!-- TODO: redis-cli tutorial -->
</li>
</ul>
<h3 id="oracle-database">Oracle Database</h3>
<ul>
<li><strong>Reference</strong> <a href="https://hub.docker.com/_/oracle-database-enterprise-edition">Docker Hub</a> (note: click on “Proceed to Checkout”)</li>
<li>Some official images, like Oracle DB, is available through Docker Store, and they need an account on <a href="https://hub.docker.com/">Docker Hub</a></li>
<li><strong>Starting container</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container run -d -it -p 1521:1521 -v /opt/docker/oradb:/ORCL --restart=always --name OracleDB store/oracle/database-enterprise:12.2.0.1</code></p>
</blockquote>
<ul>
<li>you need to login to pull this image, so execute <code class="language-plaintext highlighter-rouge">docker login</code> before pulling</li>
<li>the container for Oracle DB needs the <code class="language-plaintext highlighter-rouge">-it</code> switches, even if it is detached (<code class="language-plaintext highlighter-rouge">-d</code>)</li>
</ul>
</li>
<li><strong>Calling <em>sqlplus</em> client</strong>
<ul>
<li><code class="language-plaintext highlighter-rouge">sys</code>@CDB
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container exec -it OracleDB /bin/bash -c "source /home/oracle/.bashrc; sqlplus '/ as sysdba'"</code></p>
</blockquote>
</li>
<li><code class="language-plaintext highlighter-rouge">sys</code>@PDB
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container exec -it OracleDB /bin/bash -c "source /home/oracle/.bashrc; sqlplus sys/Oradoc_db1@ORCLPDB1 as sysdba"</code></p>
</blockquote>
</li>
<li>Oracle 12c introduces a new architecture called <strong>multitenant</strong>. In this new architecture, a database instance consists of two main sections
<ul>
<li>A <strong>container</strong>, called <em>CDB</em>, which is the basis environment of the instance</li>
<li>One or more <strong>pluggable DBs</strong>, called <em>PDB</em>, which is an end-user database for applications (this image has only one PDB)</li>
</ul>
</li>
<li>The PDB information of this image
<ul>
<li>Name is <code class="language-plaintext highlighter-rouge">ORCLPDB1</code> and its <code class="language-plaintext highlighter-rouge">sys</code> password is <code class="language-plaintext highlighter-rouge">Oradoc_db1</code></li>
<li><strong>JDBC URL</strong> - <code class="language-plaintext highlighter-rouge">jdbc:oracle:thin:@localhost:1521/orclpdb1.localdomain</code></li>
</ul>
</li>
</ul>
<!-- TODO: sqlplus tutorial -->
</li>
</ul>
<h3 id="sonatype-nexus">Sonatype Nexus</h3>
<ul>
<li><strong>Reference</strong> <a href="https://hub.docker.com/r/sonatype/nexus3">Docker Hub</a></li>
<li><strong>Starting container</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">mkdir -p /opt/docker/nexus/data</code> <br />
<code class="language-plaintext highlighter-rouge">chown -R 200:200 /opt/docker/nexus</code> <br />
<code class="language-plaintext highlighter-rouge">docker container run -d -v /opt/docker/nexus/data:/nexus-data -p 8081:8081 -e NEXUS_CONTEXT=nexus --restart=always --name Nexus sonatype/nexus3:3.15.2</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">-e NEXUS_CONTEXT=nexus</code></td>
<td>Start the server with <code class="language-plaintext highlighter-rouge">/nexus</code> context path (useful for Nginx forwarding)</td>
</tr>
</tbody>
</table>
<ul>
<li>Test the server - <code class="language-plaintext highlighter-rouge">curl -u admin:admin123 http://localhost:8081/nexus/service/metrics/ping</code></li>
<li>Visit <a href="http://localhost:8081/nexus">localhost:8081/nexus</a>, and login with default username <code class="language-plaintext highlighter-rouge">admin</code> and password <code class="language-plaintext highlighter-rouge">admin123</code></li>
</ul>
</li>
</ul>
<h3 id="jenkins-blue-ocean">Jenkins Blue Ocean</h3>
<ul>
<li><strong>Reference</strong> <a href="https://hub.docker.com/r/jenkinsci/blueocean">Docker Hub</a></li>
<li><strong>Starting container</strong>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">docker container run -d -u root -v /opt/docker/jenkins:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -p 8080:8080 -e JENKINS_OPTS="--prefix=/jenkins" --restart=always --name Jenkins jenkinsci/blueocean:1.14.0</code></p>
</blockquote>
<table>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">-u root</code></td>
<td>Username or UID (format: <code class="language-plaintext highlighter-rouge"><name|uid>[:<group|gid>]</code>), so user <code class="language-plaintext highlighter-rouge">root</code> is set as the owner of the process</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">-v /var/run/docker.sock:/var/run/docker.sock</code></td>
<td>Allow jenkins to create a container by calling Docker on your host (<strong>Docker-in-Docker</strong>)</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">-e JENKINS_OPTS="--prefix=/jenkins"</code></td>
<td>Start the server with <code class="language-plaintext highlighter-rouge">/jenkins</code> context path (useful for Nginx forwarding)</td>
</tr>
</tbody>
</table>
<ul>
<li>Visit <a href="http://localhost:8080/jenkins">localhost:8080/jenkins</a></li>
</ul>
</li>
</ul>
<h3 id="nginx--jenkins--nexus">Nginx + Jenkins + Nexus</h3>
<p>Lets mix up things and create a little bit advanced example to highlight Docker networking.
This practical example can be used as a basis for a CI (continuous integration) environment.
Following picture shows the final result of the example (note: the IPs can be different when you execute the example).</p>
<p><a name="nginx">
<img alt="Nginx" src="/assets/images/docker/Nginx.png" class="mx-auto d-block border border-dark" />
</a></p>
<p>The next script implements above picture:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
</pre></td><td class="rouge-code"><pre>docker network create DokNet1
<span class="nb">mkdir</span> <span class="nt">-p</span> /opt/docker/nexus/data
<span class="nb">chown</span> <span class="nt">-R</span> 200:200 /opt/docker/nexus
docker container run <span class="nt">-d</span> <span class="se">\</span>
<span class="nt">-v</span> /opt/docker/nexus/data:/nexus-data <span class="se">\</span>
<span class="nt">--network</span> DokNet1 <span class="nt">-e</span> <span class="nv">NEXUS_CONTEXT</span><span class="o">=</span>nexus <span class="se">\</span>
<span class="nt">--restart</span><span class="o">=</span>always <span class="nt">--name</span> Nexus sonatype/nexus3:3.15.2
docker container run <span class="nt">-d</span> <span class="nt">-u</span> root <span class="se">\</span>
<span class="nt">-v</span> /opt/docker/jenkins:/var/jenkins_home <span class="se">\</span>
<span class="nt">-v</span> /var/run/docker.sock:/var/run/docker.sock <span class="se">\</span>
<span class="nt">--network</span> DokNet1 <span class="nt">-e</span> <span class="nv">JENKINS_OPTS</span><span class="o">=</span><span class="s2">"--prefix=/jenkins"</span> <span class="se">\</span>
<span class="nt">--restart</span><span class="o">=</span>always <span class="nt">--name</span> Jenkins jenkinsci/blueocean:1.14.0
<span class="nb">mkdir</span> <span class="nt">-p</span> /opt/docker/nginx
<span class="nb">cat</span> <span class="o">></span> /opt/docker/nginx/nginx.conf <span class="o"><<</span> <span class="sh">"</span><span class="no">EOF</span><span class="sh">"
server_names_hash_bucket_size 64;
server {
listen 80;
server_name localhost;
client_max_body_size 80M;
proxy_http_version 1.1;
location /nexus/ {
proxy_pass http://Nexus:8081/nexus/;
proxy_set_header Host </span><span class="nv">$host</span><span class="sh">;
proxy_set_header X-Real-IP </span><span class="nv">$remote_addr</span><span class="sh">;
}
location /jenkins/ {
proxy_pass http://Jenkins:8080/jenkins/;
proxy_set_header Host </span><span class="nv">$host</span><span class="sh">;
proxy_set_header X-Real-IP </span><span class="nv">$remote_addr</span><span class="sh">;
}
}
</span><span class="no">EOF
</span>docker container run <span class="nt">-d</span> <span class="se">\</span>
<span class="nt">-v</span> /opt/docker/nginx/nginx.conf:/etc/nginx/conf.d/default.conf <span class="se">\</span>
<span class="nt">--network</span> DokNet1 <span class="nt">-p</span> 80:80 <span class="se">\</span>
<span class="nt">--restart</span><span class="o">=</span>always <span class="nt">--name</span> Nginx nginx:1.15.10
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Line 1 - Docker creates a virtual network, called <code class="language-plaintext highlighter-rouge">DokNet1</code>.
<ul>
<li>
<p><strong>Check it by executing <code class="language-plaintext highlighter-rouge">docker network inspect DokNet1</code></strong>
<img src="/assets/images/docker/DockerNetInspect1.png" alt="DockerNetInspect1" /></p>
</li>
<li>
<p>After this command execution, a long string is printed on screen which is identifier of this network. Its first 12 characters is also unique and we call it <code class="language-plaintext highlighter-rouge">NET_ID</code>.<br />
<img src="/assets/images/docker/DockerCreateNet.png" alt="DockerCreateNet" /></p>
</li>
<li>
<p>Executing <code class="language-plaintext highlighter-rouge">ifconfig</code>, it is in the list by the name of <code class="language-plaintext highlighter-rouge">br-NET_ID</code>
<img src="/assets/images/docker/DockerIfconfig2.png" alt="DockerIfconfig2" /></p>
</li>
</ul>
</li>
<li>Option <code class="language-plaintext highlighter-rouge">--network DokNet1</code> attaches the container to <code class="language-plaintext highlighter-rouge">DokNet1</code> network, so all of the above containers are in the same network (<a href="#nginx">picture</a>).</li>
<li>Lines 17-38 create a config file for Nginx
<ul>
<li>Line 27 - Nginx forwards request to Nexus by calling <code class="language-plaintext highlighter-rouge">http://Nexus:8081/nexus/</code>, which restates the fact that containers can access each other by the name in the network.</li>
<li>Line 33 - It is the same as previous line for Jenkins.</li>
</ul>
</li>
<li>Only Nginx has a binding port. It is not necessary for Jenkins and Nexus to bind ports.</li>
<li>Accessing web apps
<ul>
<li>Jenkins - <a href="http://localhost/jenkins">localhost/jenkins</a></li>
<li>Nexus - <a href="http://localhost/nexus">localhost/jenkins</a>. Executing <code class="language-plaintext highlighter-rouge">curl -u admin:admin123 http://localhost/nexus/service/metrics/ping</code> is another way.</li>
</ul>
</li>
<li>At the end, execute <code class="language-plaintext highlighter-rouge">docker network inspect DokNet1</code> again. You can see list of containers attached to this network.
<img src="/assets/images/docker/DockerNetInspect2.png" alt="DockerNetInspect2" /></li>
</ul>
<h2 id="references">References</h2>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:docker_layer" role="doc-endnote">
<p><a href="https://dzone.com/articles/docker-layers-explained">Docker Layers Explained</a> <a href="#fnref:docker_layer" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:docker_fqn" role="doc-endnote">
<p><a href="https://windsock.io/referencing-docker-images/">Referencing Docker Images</a> <a href="#fnref:docker_fqn" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Mehdi BizhaniRunning containers by DockerDeveloping by Scala - 022018-07-19T00:00:00+00:002018-07-19T00:00:00+00:00http://www.devocative.org/article/scala/scala02<h2 id="class">Class</h2>
<ul>
<li><strong>No need for public modifier</strong>, a class is not declared as <code class="language-plaintext highlighter-rouge">public</code></li>
<li><strong>Multiple classes in a source file</strong> with public visibility</li>
<li>Call a parameterless method <strong><em>with</em> or <em>without</em> parentheses</strong>
<ul>
<li>Use () for a <strong>mutator</strong> method</li>
<li>Drop the () for an <strong>accessor</strong> method. To enforce this style, declare the method without <code class="language-plaintext highlighter-rouge">()</code></li>
</ul>
</li>
<li><strong>No need to define accessors for a <code class="language-plaintext highlighter-rouge">var</code> field</strong>
<ul>
<li>For a <code class="language-plaintext highlighter-rouge">var</code> field, a private field with its public accessors are generated in bytecode</li>
<li>In the case of private field, accessors become private too.</li>
<li>Note: Simple getters and setters are preferred over public fields. They can be evolved as needed. So custom getter/setter without changing the client of a class is possible.
That is called the <strong>uniform access principle</strong></li>
<li>Defining a field as <code class="language-plaintext highlighter-rouge">val</code> results in a read-only property</li>
<li>In the case of custom accessors, there should not be identical to var name</li>
</ul>
</li>
<li>Parameters of the primary constructor turn into <code class="language-plaintext highlighter-rouge">var</code> fields, initialized by the passed arguments</li>
<li><strong>Nest anything inside anything</strong>
<ul>
<li>functions inside other functions, and classes inside other classes</li>
</ul>
</li>
</ul>
<p>Define custom getter</p>
<blockquote>
<p>def PROP: TYPE = …</p>
</blockquote>
<p>Define custom setter</p>
<blockquote>
<p>def PROP_=(VARIABLE: TYPE) …</p>
</blockquote>
<p><strong>Note</strong>: both <code class="language-plaintext highlighter-rouge">_=</code> are required without space between and before!</p>
<p>In the following class, important points of a class in Scala is illustrated.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
</pre></td><td class="rouge-code"><pre><span class="k">package</span> <span class="nn">org.devocative</span>
<span class="k">object</span> <span class="nc">Education</span> <span class="k">extends</span> <span class="nc">Enumeration</span> <span class="o">{</span>
<span class="k">type</span> <span class="kt">Education</span> <span class="o">=</span> <span class="nc">Value</span>
<span class="k">val</span> <span class="nv">Diploma</span><span class="o">,</span> <span class="nc">Bachelor</span><span class="o">,</span> <span class="nc">Master</span><span class="o">,</span> <span class="nc">PhD</span> <span class="k">=</span> <span class="nc">Value</span>
<span class="o">}</span>
<span class="k">package</span> <span class="nn">employee</span> <span class="o">{</span>
<span class="k">import</span> <span class="nn">java.util.</span><span class="o">{</span><span class="nc">Calendar</span><span class="o">,</span> <span class="nc">Date</span><span class="o">}</span>
<span class="k">import</span> <span class="nn">org.devocative.Education.Education</span> <span class="c1">//----- also import Education.Education</span>
<span class="k">import</span> <span class="nn">scala.beans.BeanProperty</span> <span class="c1">//----------------try to import fully qualified (scala can be omitted)</span>
<span class="c1">// although following class is not declared public, it is public</span>
<span class="k">class</span> <span class="nc">Person</span> <span class="o">{</span> <span class="n">outer</span> <span class="k">=></span> <span class="c1">//-----------------------alias 'outer' = 'Person.this', useful for inner class</span>
<span class="k">val</span> <span class="nv">id</span><span class="k">:</span> <span class="kt">Int</span> <span class="o">=</span> <span class="nv">Person</span><span class="o">.</span><span class="py">newUniqueNumber</span><span class="o">()</span> <span class="c1">//------need 'Person' qualifier</span>
<span class="k">var</span> <span class="n">name</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="k">_</span> <span class="c1">//------------------------field must be initialized, '_' means null</span>
<span class="k">var</span> <span class="n">education</span><span class="k">:</span> <span class="kt">Education</span> <span class="o">=</span> <span class="k">_</span> <span class="c1">//----------------Enumeration as a type need 'type' definition</span>
<span class="k">private</span><span class="o">[</span><span class="kt">this</span><span class="o">]</span> <span class="k">var</span> <span class="n">birth_date</span><span class="k">:</span> <span class="kt">Date</span> <span class="o">=</span> <span class="k">_</span> <span class="c1">//------object-private field</span>
<span class="nd">@BeanProperty</span> <span class="c1">//-------------------------------generating getX & setX for Java interoperability,</span>
<span class="k">var</span> <span class="n">weight</span><span class="k">:</span> <span class="kt">Int</span> <span class="o">=</span> <span class="mi">0</span> <span class="c1">//-------------------------so it can't be private!</span>
<span class="k">var</span> <span class="n">address</span><span class="k">:</span> <span class="kt">Person</span><span class="k">#</span><span class="kt">Address</span> <span class="o">=</span> <span class="k">_</span> <span class="c1">//-------------type projection, i.e. Address for any Person</span>
<span class="c1">// --------------- A U X C O N S T R U C T O R S</span>
<span class="k">def</span> <span class="nf">this</span><span class="o">(</span><span class="n">name</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span> <span class="n">birthDate</span><span class="k">:</span> <span class="kt">Date</span><span class="o">)</span> <span class="o">{</span> <span class="c1">//---use 'this' for constructor name</span>
<span class="nf">this</span><span class="o">()</span> <span class="c1">//------------------------------------calling default primary constructor is mandatory</span>
<span class="k">this</span><span class="o">.</span><span class="py">name</span> <span class="k">=</span> <span class="n">name</span> <span class="c1">// -------------------------also outer.name = name</span>
<span class="k">this</span><span class="o">.</span><span class="py">birthDate</span> <span class="k">=</span> <span class="n">birthDate</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">this</span><span class="o">(</span><span class="n">birthDate</span><span class="k">:</span> <span class="kt">Date</span><span class="o">)</span> <span class="o">{</span> <span class="c1">// ----------------this must be declared after the above one!</span>
<span class="nf">this</span><span class="o">(</span><span class="kc">null</span><span class="o">,</span> <span class="n">birthDate</span><span class="o">)</span> <span class="c1">// --------------------refer to the above one</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">this</span><span class="o">(</span><span class="n">name</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="o">{</span>
<span class="nf">this</span><span class="o">(</span><span class="n">name</span><span class="o">,</span> <span class="kc">null</span><span class="o">)</span>
<span class="o">}</span>
<span class="c1">// --------------- A C C E S S O R S</span>
<span class="k">def</span> <span class="nf">age</span> <span class="k">=</span> <span class="o">{</span> <span class="c1">//----------------------------age is an accessor, preferred to declare without ()</span>
<span class="k">val</span> <span class="nv">bd</span><span class="k">:</span> <span class="kt">Calendar</span> <span class="o">=</span> <span class="nv">Calendar</span><span class="o">.</span><span class="py">getInstance</span>
<span class="nv">bd</span><span class="o">.</span><span class="py">setTime</span><span class="o">(</span><span class="n">birth_date</span><span class="o">)</span>
<span class="nv">Calendar</span><span class="o">.</span><span class="py">getInstance</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="nv">Calendar</span><span class="o">.</span><span class="py">YEAR</span><span class="o">)</span> <span class="o">-</span> <span class="nv">bd</span><span class="o">.</span><span class="py">get</span><span class="o">(</span><span class="nv">Calendar</span><span class="o">.</span><span class="py">YEAR</span><span class="o">)</span>
<span class="o">}</span>
<span class="k">def</span> <span class="nf">birthDate</span><span class="k">:</span> <span class="kt">Date</span> <span class="o">=</span> <span class="n">birth_date</span>
<span class="k">def</span> <span class="nf">birthDate_=</span><span class="o">(</span><span class="n">date</span><span class="k">:</span> <span class="kt">Date</span><span class="o">)</span> <span class="o">{</span>
<span class="nf">if</span> <span class="o">(</span><span class="n">date</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="nv">date</span><span class="o">.</span><span class="py">compareTo</span><span class="o">(</span><span class="k">new</span> <span class="nc">Date</span><span class="o">)</span> <span class="o"><=</span> <span class="mi">0</span><span class="o">)</span>
<span class="n">birth_date</span> <span class="k">=</span> <span class="n">date</span>
<span class="k">else</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">IllegalAccessException</span><span class="o">(</span><span class="s">"Invalid Birth Date"</span><span class="o">)</span>
<span class="o">}</span>
<span class="c1">// --------------- F U C T I O N S & P R O C E D U R E S</span>
<span class="c1">// since 'birth_date' is an object-private field, can't access other.birth_date</span>
<span class="k">def</span> <span class="nf">></span><span class="o">(</span><span class="n">other</span><span class="k">:</span> <span class="kt">Person</span><span class="o">)</span><span class="k">:</span> <span class="kt">Boolean</span> <span class="o">=</span> <span class="n">birth_date</span> <span class="o">!=</span> <span class="kc">null</span> <span class="o">&&</span> <span class="nv">birth_date</span><span class="o">.</span><span class="py">compareTo</span><span class="o">(</span><span class="nv">other</span><span class="o">.</span><span class="py">birthDate</span><span class="o">)</span> <span class="o"><</span> <span class="mi">0</span>
<span class="k">def</span> <span class="nf">+</span><span class="o">(</span><span class="n">other</span><span class="k">:</span> <span class="kt">Person</span><span class="o">)</span><span class="k">:</span> <span class="kt">Set</span><span class="o">[</span><span class="kt">Person</span><span class="o">]</span> <span class="k">=</span> <span class="nc">Set</span><span class="o">[</span><span class="kt">Person</span><span class="o">](</span><span class="k">this</span><span class="o">,</span> <span class="n">other</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">sendMsg</span><span class="o">()</span> <span class="o">{</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"Hi $name ($id), You are $age years old, and your weight is $getWeight"</span><span class="o">)</span>
<span class="o">}</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">toString</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="n">name</span>
<span class="c1">// --------------- I N N E R C L A S S</span>
<span class="k">class</span> <span class="nc">Address</span><span class="o">(</span><span class="k">var</span> <span class="n">province</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span> <span class="k">var</span> <span class="n">city</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="o">{</span> <span class="c1">// class with primary constructor</span>
<span class="k">override</span> <span class="k">def</span> <span class="nf">toString</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="n">s</span><span class="s">"${outer.name} lives at $city, $province"</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// --------------- C O M P A N I O N O B J E C T</span>
<span class="k">object</span> <span class="nc">Person</span> <span class="o">{</span>
<span class="k">private</span> <span class="k">var</span> <span class="n">lastNumber</span> <span class="k">=</span> <span class="mi">0</span>
<span class="k">def</span> <span class="nf">newUniqueNumber</span><span class="o">()</span><span class="k">:</span> <span class="kt">Int</span> <span class="o">=</span> <span class="o">{</span><span class="n">lastNumber</span> <span class="o">+=</span> <span class="mi">1</span><span class="o">;</span> <span class="n">lastNumber</span><span class="o">}</span>
<span class="k">def</span> <span class="nf">apply</span><span class="o">(</span><span class="n">name</span><span class="k">:</span> <span class="kt">String</span><span class="o">)</span><span class="k">:</span> <span class="kt">Person</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Person</span><span class="o">(</span><span class="n">name</span><span class="o">)</span>
<span class="k">def</span> <span class="nf">apply</span><span class="o">(</span><span class="n">name</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span> <span class="n">birthDate</span><span class="k">:</span> <span class="kt">Date</span><span class="o">)</span><span class="k">:</span> <span class="kt">Person</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Person</span><span class="o">(</span><span class="n">name</span><span class="o">,</span> <span class="n">birthDate</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Both the class and its companion object
<ul>
<li>can access each other’s private features</li>
<li>must be defined in the <strong>same source file</strong></li>
</ul>
</li>
</ul>
<p>Now, following code shows a sample code to use the above class:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nn">java.util.Calendar</span>
<span class="k">import</span> <span class="nn">org.devocative.Education</span>
<span class="k">import</span> <span class="nn">org.devocative.employee.Person</span>
<span class="c1">// ...</span>
<span class="k">val</span> <span class="nv">cal</span><span class="k">:</span> <span class="kt">Calendar</span> <span class="o">=</span> <span class="nv">Calendar</span><span class="o">.</span><span class="py">getInstance</span>
<span class="nv">cal</span><span class="o">.</span><span class="py">set</span><span class="o">(</span><span class="nv">Calendar</span><span class="o">.</span><span class="py">YEAR</span><span class="o">,</span> <span class="mi">1980</span><span class="o">)</span>
<span class="nv">cal</span><span class="o">.</span><span class="py">set</span><span class="o">(</span><span class="nv">Calendar</span><span class="o">.</span><span class="py">MONTH</span><span class="o">,</span> <span class="mi">1</span><span class="o">)</span>
<span class="nv">cal</span><span class="o">.</span><span class="py">set</span><span class="o">(</span><span class="nv">Calendar</span><span class="o">.</span><span class="py">DATE</span><span class="o">,</span> <span class="mi">1</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">jane</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">Person</span>
<span class="c1">//jane.id = 1 ------------------------------ERROR: val variable</span>
<span class="nv">jane</span><span class="o">.</span><span class="py">birthDate</span> <span class="k">=</span> <span class="nv">cal</span><span class="o">.</span><span class="py">getTime</span>
<span class="nv">jane</span><span class="o">.</span><span class="py">name</span> <span class="k">=</span> <span class="s">"Jane"</span>
<span class="nv">jane</span><span class="o">.</span><span class="py">setWeight</span><span class="o">(</span><span class="mi">60</span><span class="o">)</span> <span class="c1">//-----------------------also jane.weight = 60</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">jane</span><span class="o">.</span><span class="py">age</span><span class="o">)</span> <span class="c1">//------------------------OUTPUT: 38</span>
<span class="nv">jane</span><span class="o">.</span><span class="py">sendMsg</span><span class="o">()</span> <span class="c1">//---------------------------OUTPUT: Hi Jane, You are 38 years old, and your weight is 60</span>
<span class="k">val</span> <span class="nv">joe</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">Person</span><span class="o">(</span><span class="s">"Joe"</span><span class="o">,</span> <span class="o">{</span><span class="nv">cal</span><span class="o">.</span><span class="py">add</span><span class="o">(</span><span class="nv">Calendar</span><span class="o">.</span><span class="py">YEAR</span><span class="o">,</span> <span class="o">-</span><span class="mi">1</span><span class="o">);</span> <span class="nv">cal</span><span class="o">.</span><span class="py">getTime</span><span class="o">})</span> <span class="c1">// calling aux constructor</span>
<span class="nv">joe</span><span class="o">.</span><span class="py">address</span> <span class="k">=</span> <span class="k">new</span> <span class="nv">joe</span><span class="o">.</span><span class="py">Address</span><span class="o">(</span><span class="s">"P"</span><span class="o">,</span> <span class="s">"C"</span><span class="o">)</span> <span class="c1">//--instantiate inner class by object reference</span>
<span class="nf">println</span><span class="o">(</span><span class="n">joe</span> <span class="o">></span> <span class="n">jane</span><span class="o">)</span> <span class="c1">//----------------------also joe.>(jane), output: true</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">joe</span><span class="o">.</span><span class="py">address</span><span class="o">)</span> <span class="c1">//---------------------OUTPUT: Joe lives at P, C</span>
<span class="nv">jane</span><span class="o">.</span><span class="py">address</span> <span class="k">=</span> <span class="nv">joe</span><span class="o">.</span><span class="py">address</span> <span class="c1">//---------------it is ok because of type projection</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">jane</span><span class="o">.</span><span class="py">address</span><span class="o">)</span> <span class="c1">//--------------------OUTPUT: Joe lives at P, C</span>
<span class="nv">jane</span><span class="o">.</span><span class="py">education</span> <span class="k">=</span> <span class="nv">Education</span><span class="o">.</span><span class="py">Bachelor</span>
<span class="k">var</span> <span class="n">children</span> <span class="k">=</span> <span class="nc">Person</span><span class="o">(</span><span class="s">"Joe Junior"</span><span class="o">)</span> <span class="o">+</span> <span class="nc">Person</span><span class="o">(</span><span class="s">"Sara"</span><span class="o">)</span> <span class="o">+</span> <span class="nc">Person</span><span class="o">(</span><span class="s">"Janet"</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Line 25 shows the same result as line 28! Although <code class="language-plaintext highlighter-rouge">Address</code> is type projected, the instance is created by <code class="language-plaintext highlighter-rouge">joe</code>!</li>
<li>Line 31, first <code class="language-plaintext highlighter-rouge">+</code> is the function defined in <code class="language-plaintext highlighter-rouge">Person</code>, while the second one is defined in <code class="language-plaintext highlighter-rouge">scala.collection.Set</code></li>
</ul>
<h2 id="object">Object</h2>
<ul>
<li>Java’s static fields and methods replacement in Scala
<ul>
<li>A class can have a companion object with the same name in the same file</li>
<li>Define an <code class="language-plaintext highlighter-rouge">apply</code> method in the companion object for constructing new instances of the related class</li>
</ul>
</li>
<li>It can extend other classes or traits</li>
<li>No constructor parameters</li>
<li>Suitable for
<ul>
<li>Utility functions or constants</li>
<li>Sharing a single immutable instance efficiently</li>
<li>Coordinate some service using single instance (the singleton design pattern)</li>
</ul>
</li>
</ul>Mehdi BizhaniA Summary for Scala Language, Part 02Developing by Scala - 012018-07-04T00:00:00+00:002018-07-04T00:00:00+00:00http://www.devocative.org/article/scala/scala01<h2 id="installation">Installation</h2>
<ol>
<li>Section <code class="language-plaintext highlighter-rouge">Other ways to install Scala</code>, download for Linux (<a href="https://www.scala-lang.org/download/">link</a>)</li>
<li>Define <code class="language-plaintext highlighter-rouge">SCALA_HOME</code> based on <a href="https://www.scala-lang.org/download/install.html">link</a></li>
<li>Add <code class="language-plaintext highlighter-rouge">$SCALA_HOME/bin</code> to <code class="language-plaintext highlighter-rouge">$PATH</code></li>
</ol>
<p>Now, create file <code class="language-plaintext highlighter-rouge">HelloWorld.scala</code>:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="k">object</span> <span class="nc">HelloWorld</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">main</span><span class="o">(</span><span class="n">args</span><span class="k">:</span> <span class="kt">Array</span><span class="o">[</span><span class="kt">String</span><span class="o">])</span><span class="k">:</span> <span class="kt">Unit</span> <span class="o">=</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">language</span> <span class="k">=</span> <span class="s">"Scala"</span>
<span class="k">val</span> <span class="nv">msg</span> <span class="k">=</span> <span class="n">s</span><span class="s">"Hello World, My Lang is ${language}"</span>
<span class="nf">println</span><span class="o">(</span><span class="n">msg</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>To Compile & Run:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
</pre></td><td class="rouge-code"><pre>scalac HelloWorld.scala
scala <span class="nt">-cp</span> <span class="nb">.</span> HelloWorld
java <span class="nt">-cp</span> .:<span class="nv">$SCALA_HOME</span>/lib/<span class="k">*</span> HelloWorld
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Line 2, use <code class="language-plaintext highlighter-rouge">scala</code> command to execute the class</li>
<li>Since <code class="language-plaintext highlighter-rouge">scalac</code> generates bytecode, line 3 executes class by <code class="language-plaintext highlighter-rouge">java</code> command with extra classpath</li>
</ul>
<p>The <code class="language-plaintext highlighter-rouge">HelloWorld</code> class can be rewritten as</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="k">object</span> <span class="nc">HelloWorld</span> <span class="k">extends</span> <span class="nc">App</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">language</span> <span class="k">=</span> <span class="s">"Scala"</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"Hello World, My Lang is $language"</span><span class="o">)</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="basics">Basics</h2>
<ul>
<li><strong>No semicolon</strong> to terminate lines, unless multiple expressions on the same line</li>
<li><strong>No need for identical name of the class & its file</strong>
<ul>
<li>in spite of Java, Scala file names do not need to be the same as class names</li>
</ul>
</li>
<li><strong>No need of directory structure corresponding to package hierarchy</strong>
<ul>
<li>however, the correspondence is recommended</li>
</ul>
</li>
<li><strong>No need to <code class="language-plaintext highlighter-rouge">public</code> modifier</strong>
<ul>
<li>by default everything is public</li>
</ul>
</li>
<li><strong>Same operator precedence</strong> as in Java
<ul>
<li>In fact all operators are the object methods using as the <em>infix operator</em> syntax</li>
</ul>
</li>
<li><strong><code class="language-plaintext highlighter-rouge">Unit</code> of Scala = <code class="language-plaintext highlighter-rouge">void</code> of Java</strong>
<ul>
<li>however <code class="language-plaintext highlighter-rouge">Unit</code> has a single value which is <code class="language-plaintext highlighter-rouge">()</code></li>
</ul>
</li>
<li><strong><code class="language-plaintext highlighter-rouge">Any</code> of Scala = <code class="language-plaintext highlighter-rouge">Object</code> of Java</strong></li>
<li><strong>No need using <code class="language-plaintext highlighter-rouge">return</code></strong>
<ul>
<li>automatically the value of last expression of every block is returned</li>
</ul>
</li>
<li><strong>No checked exception</strong></li>
<li><strong>No try-with-resource</strong> as in Java</li>
<li><strong>No<code class="language-plaintext highlighter-rouge">?:</code> operator</strong> in Scala, use <code class="language-plaintext highlighter-rouge">if-else</code> instead (<a href="#control-structures">section</a>)</li>
<li><strong>No primitive type</strong> in Scala, everything is object (the reason Scala is pure OO while Java is not)</li>
<li><strong>Relaxed method name, such as <code class="language-plaintext highlighter-rouge">+</code>, <code class="language-plaintext highlighter-rouge">*</code>, …</strong>, similar to operator overloading
<ul>
<li>In fact <code class="language-plaintext highlighter-rouge">2 + 3</code> is <code class="language-plaintext highlighter-rouge">2.+(3)</code></li>
</ul>
</li>
<li><strong>No enum keyword and enumeration like Java</strong>
<ul>
<li>use <code class="language-plaintext highlighter-rouge">object</code> extends <code class="language-plaintext highlighter-rouge">Enumeration</code></li>
</ul>
</li>
<li>Using <code class="language-plaintext highlighter-rouge">object</code> keyword to create singleton objects in Scala
<ul>
<li>Adding methods to these objects is like static methods in Java, and they are accessible by object name</li>
<li><strong>So no <code class="language-plaintext highlighter-rouge">static</code> in Scala</strong></li>
</ul>
</li>
</ul>
<p>Define a constant:</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">val</code> ID[:TYPE] = VALUE</p>
</blockquote>
<p>Define a variable:</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">var</code> ID[:TYPE] = VALUE</p>
</blockquote>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">xmax</span><span class="o">,</span> <span class="n">ymax</span> <span class="k">=</span> <span class="mi">100</span>
<span class="k">var</span> <span class="n">greeting</span><span class="o">,</span> <span class="n">message</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="kc">null</span>
<span class="c1">// assign two different constants using a tuple in a single line</span>
<span class="nf">val</span> <span class="o">(</span><span class="n">sum</span><span class="o">,</span> <span class="n">count</span><span class="o">)</span> <span class="k">=</span> <span class="o">(</span><span class="mf">1.0</span><span class="o">,</span> <span class="mi">1</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong>Note</strong>: In Scala, use <code class="language-plaintext highlighter-rouge">val</code> unless you really need to change the contents! (Scala loves immutability)</p>
<p>Support Java classes:</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nn">java.util.</span><span class="o">{</span><span class="nc">Date</span><span class="o">,</span> <span class="nc">Locale</span><span class="o">}</span>
<span class="k">import</span> <span class="nn">java.text.DateFormat._</span>
<span class="k">object</span> <span class="nc">FrenchDate</span> <span class="o">{</span>
<span class="k">def</span> <span class="nf">main</span><span class="o">(</span><span class="n">args</span><span class="k">:</span> <span class="kt">Array</span><span class="o">[</span><span class="kt">String</span><span class="o">])</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">now</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">Date</span>
<span class="k">val</span> <span class="nv">df</span> <span class="k">=</span> <span class="nf">getDateInstance</span><span class="o">(</span><span class="nc">LONG</span><span class="o">,</span> <span class="nv">Locale</span><span class="o">.</span><span class="py">FRANCE</span><span class="o">)</span>
<span class="nf">println</span><span class="o">(</span><span class="n">df</span> <span class="n">format</span> <span class="n">now</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong>TODO: look <code class="language-plaintext highlighter-rouge">lazy</code> val</strong></p>
<h2 id="control-structures">Control Structures</h2>
<ul>
<li>In Java these two have differences:
<ul>
<li><em>expression</em>, such as <code class="language-plaintext highlighter-rouge">3 + 4</code>, has a value</li>
<li><em>statement</em>, such as <code class="language-plaintext highlighter-rouge">if(...) ...</code>, carries out an action</li>
</ul>
</li>
<li>In Scala, almost all constructs have values
<ul>
<li>An <code class="language-plaintext highlighter-rouge">if</code> expression has a value</li>
<li>A block has a value - the value of its last expression</li>
</ul>
</li>
<li>Scala does not have <code class="language-plaintext highlighter-rouge">?:</code> operator! Use <code class="language-plaintext highlighter-rouge">if else</code> instead</li>
</ul>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">x</span><span class="k">:</span> <span class="kt">Double</span> <span class="o">=</span> <span class="nv">Math</span><span class="o">.</span><span class="py">random</span><span class="o">()</span>
<span class="k">val</span> <span class="nv">b</span><span class="k">:</span> <span class="kt">Boolean</span> <span class="o">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">x</span> <span class="o">></span> <span class="o">.</span><span class="mi">5</span><span class="o">)</span> <span class="kc">true</span> <span class="k">else</span> <span class="kc">false</span>
<span class="k">val</span> <span class="nv">a</span><span class="k">:</span> <span class="kt">Any</span> <span class="o">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">x</span> <span class="o">></span> <span class="o">.</span><span class="mi">5</span><span class="o">)</span> <span class="kc">true</span> <span class="k">else</span> <span class="s">"Oops!"</span>
<span class="k">val</span> <span class="nv">u</span><span class="k">:</span> <span class="kt">Any</span> <span class="o">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">x</span> <span class="o">></span> <span class="o">.</span><span class="mi">5</span><span class="o">)</span> <span class="kc">true</span> <span class="c1">// or if (x > .5) true else ()</span>
<span class="k">val</span> <span class="nv">distance</span> <span class="k">=</span> <span class="o">{</span><span class="k">val</span> <span class="nv">dx</span> <span class="k">=</span> <span class="n">x</span><span class="o">-</span><span class="n">x0</span><span class="o">;</span> <span class="k">val</span> <span class="nv">dy</span> <span class="k">=</span> <span class="n">y</span><span class="o">-</span><span class="n">y0</span><span class="o">;</span> <span class="nf">sqrt</span><span class="o">(</span><span class="n">dx</span> <span class="o">*</span> <span class="n">dx</span> <span class="o">+</span> <span class="n">dy</span> <span class="o">*</span> <span class="n">dy</span><span class="o">)}</span>
<span class="k">val</span> <span class="nv">q</span><span class="k">:</span> <span class="kt">Double</span> <span class="o">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">x</span> <span class="o">>=</span> <span class="mi">0</span><span class="o">)</span> <span class="nv">Math</span><span class="o">.</span><span class="py">sqrt</span><span class="o">(</span><span class="n">x</span><span class="o">)</span> <span class="k">else</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">IllegalArgumentException</span><span class="o">(</span><span class="s">"No Negative"</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Line 3, the if-else has a definite <code class="language-plaintext highlighter-rouge">Boolean</code> return type</li>
<li>Line 4, each branch has a different type, so the variable is of type <code class="language-plaintext highlighter-rouge">Any</code> (it is the common supertype), however the returning object is <code class="language-plaintext highlighter-rouge">Boolean</code> or <code class="language-plaintext highlighter-rouge">String</code></li>
<li>Line 5, like line 4, but the returning object is <code class="language-plaintext highlighter-rouge">Boolean</code> or <code class="language-plaintext highlighter-rouge">Unit</code>. <code class="language-plaintext highlighter-rouge">Unit</code> type has one value which is <code class="language-plaintext highlighter-rouge">()</code></li>
<li>Line 6, the value of <code class="language-plaintext highlighter-rouge">sqrt()</code> is assigned to <code class="language-plaintext highlighter-rouge">distance</code> (a good practice to initialize a <code class="language-plaintext highlighter-rouge">val</code>)</li>
<li>Line 8, the type of <code class="language-plaintext highlighter-rouge">if</code> is <code class="language-plaintext highlighter-rouge">Double</code> and the type of <code class="language-plaintext highlighter-rouge">else</code> is <strong><code class="language-plaintext highlighter-rouge">Nothing</code></strong> which is ignored!</li>
</ul>
<h3 id="loops">Loops</h3>
<p><code class="language-plaintext highlighter-rouge">while</code> is the same as Java</p>
<blockquote>
<p>while (BOOLEAN_EXPRESSION) …</p>
</blockquote>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="nf">var</span> <span class="o">(</span><span class="n">r</span><span class="o">,</span> <span class="n">n</span><span class="o">)</span> <span class="k">=</span> <span class="o">(</span><span class="mf">1.0</span><span class="o">,</span> <span class="mi">10</span><span class="o">)</span> <span class="c1">// multiple variables by tuple, r:Double, n:Int</span>
<span class="nf">while</span> <span class="o">(</span><span class="n">n</span> <span class="o">></span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="n">r</span> <span class="k">=</span> <span class="n">r</span> <span class="o">*</span> <span class="n">n</span>
<span class="n">n</span> <span class="o">-=</span> <span class="mi">1</span>
<span class="o">}</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"r = $r"</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Scala has no loop of <code class="language-plaintext highlighter-rouge">for (initialize; test; update)</code> as Java. Instead, the syntax of for is</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">10</span><span class="o">)</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">x</span><span class="k">:</span> <span class="kt">Double</span> <span class="o">=</span> <span class="nv">Math</span><span class="o">.</span><span class="py">random</span><span class="o">()</span>
<span class="k">val</span> <span class="nv">u</span><span class="k">:</span> <span class="kt">Any</span> <span class="o">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">x</span> <span class="o">></span> <span class="o">.</span><span class="mi">5</span><span class="o">)</span> <span class="kc">true</span>
<span class="nf">println</span><span class="o">(</span><span class="n">f</span><span class="s">"i = $i%02d, u = $u"</span><span class="o">)</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">s</span> <span class="k">=</span> <span class="s">"Hello"</span>
<span class="k">var</span> <span class="n">sum</span> <span class="k">=</span> <span class="mi">0</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">0</span> <span class="n">until</span> <span class="nv">s</span><span class="o">.</span><span class="py">length</span><span class="o">)</span> <span class="c1">// or for (i <- 0 to s.length - 1) </span>
<span class="n">sum</span> <span class="o">+=</span> <span class="nf">s</span><span class="o">(</span><span class="n">i</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>or</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="k">var</span> <span class="n">sum</span> <span class="k">=</span> <span class="mi">0</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">ch</span> <span class="k"><-</span> <span class="s">"Hello"</span><span class="o">)</span> <span class="n">sum</span> <span class="o">+=</span> <span class="n">ch</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Note: Scala’s <code class="language-plaintext highlighter-rouge">Int</code> has a special API for range</p>
<ul>
<li>[a, b]: <code class="language-plaintext highlighter-rouge">1.to(5)</code> or <code class="language-plaintext highlighter-rouge">1 to 5</code></li>
<li>[a, b): <code class="language-plaintext highlighter-rouge">1.until(5)</code> or <code class="language-plaintext highlighter-rouge">1 until 5</code></li>
</ul>
<h3 id="advanced-for">Advanced <code class="language-plaintext highlighter-rouge">for</code></h3>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
</pre></td><td class="rouge-code"><pre><span class="c1">// multiple generators: variable <- expression</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">3</span><span class="o">;</span> <span class="n">j</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">3</span><span class="o">)</span> <span class="nf">print</span><span class="o">(</span><span class="n">f</span><span class="s">"${10 * i + j}%3d"</span><span class="o">)</span> <span class="c1">// 11 12 13 21 22 23 31 32 33</span>
<span class="c1">// second generator with a 'guard' starting with an 'if'</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">3</span><span class="o">;</span> <span class="n">j</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">3</span> <span class="k">if</span> <span class="n">i</span> <span class="o">!=</span> <span class="n">j</span><span class="o">)</span> <span class="nf">print</span><span class="o">(</span><span class="n">f</span><span class="s">"${10 * i + j}%3d"</span><span class="o">)</span> <span class="c1">// 12 13 21 23 31 32</span>
<span class="c1">// using one or more definitions, e.g. the 'from' var</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">3</span><span class="o">;</span> <span class="n">from</span> <span class="k">=</span> <span class="mi">4</span> <span class="o">-</span> <span class="n">i</span><span class="o">;</span> <span class="n">j</span> <span class="k"><-</span> <span class="n">from</span> <span class="n">to</span> <span class="mi">3</span><span class="o">)</span> <span class="nf">print</span><span class="o">(</span><span class="n">f</span><span class="s">"${10 * i + j}%3d"</span><span class="o">)</span> <span class="c1">// 13 22 23 31 32 33</span>
<span class="c1">//or</span>
<span class="k">for</span> <span class="o">{</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">3</span>
<span class="n">from</span> <span class="k">=</span> <span class="mi">4</span> <span class="o">-</span> <span class="n">i</span>
<span class="n">j</span> <span class="k"><-</span> <span class="n">from</span> <span class="n">to</span> <span class="mi">3</span><span class="o">}</span>
<span class="nf">print</span><span class="o">(</span><span class="n">f</span><span class="s">"${10 * i + j}%3d"</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="for-comprehension">for Comprehension</h3>
<blockquote>
<p>A <code class="language-plaintext highlighter-rouge">for</code> loop with <code class="language-plaintext highlighter-rouge">yield</code> returns a sequence</p>
</blockquote>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">v</span> <span class="k">=</span> <span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="mi">10</span><span class="o">)</span> <span class="k">yield</span> <span class="n">i</span> <span class="o">%</span> <span class="mi">3</span> <span class="c1">// v is Vector(1, 2, 0, 1, 2, 0, 1, 2, 0, 1)</span>
<span class="nv">v</span><span class="o">.</span><span class="py">foreach</span><span class="o">(</span><span class="n">println</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">str</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="nf">for</span> <span class="o">(</span><span class="n">c</span> <span class="k"><-</span> <span class="s">"Hello"</span><span class="o">;</span> <span class="n">i</span> <span class="k"><-</span> <span class="mi">0</span> <span class="n">to</span> <span class="mi">1</span><span class="o">)</span> <span class="nf">yield</span> <span class="o">(</span><span class="n">c</span> <span class="o">+</span> <span class="n">i</span><span class="o">).</span><span class="py">toChar</span> <span class="c1">// HIeflmlmop</span>
<span class="k">val</span> <span class="nv">vc</span> <span class="k">=</span> <span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">0</span> <span class="n">to</span> <span class="mi">1</span><span class="o">;</span> <span class="n">c</span> <span class="k"><-</span> <span class="s">"Hello"</span><span class="o">)</span> <span class="nf">yield</span> <span class="o">(</span><span class="n">c</span> <span class="o">+</span> <span class="n">i</span><span class="o">).</span><span class="py">toChar</span> <span class="c1">// Vector(H, e, l, l, o, I, f, m, m, p)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong>Note</strong>: Lines 4 & 5 look similar, but the results are different. So the generated collection is compatible with the first generator.</p>
<h2 id="function--procedure">Function & Procedure</h2>
<ul>
<li>Scala functions are like static methods in Java (C++ also has functions)</li>
<li>No need to specify the return type, unless the function is recursive</li>
</ul>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="k">def</span> <span class="nf">abs</span><span class="o">(</span><span class="n">x</span><span class="k">:</span> <span class="kt">Double</span><span class="o">)</span> <span class="k">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">x</span> <span class="o">>=</span> <span class="mi">0</span><span class="o">)</span> <span class="n">x</span> <span class="k">else</span> <span class="o">-</span><span class="n">x</span>
<span class="k">def</span> <span class="nf">fac</span><span class="o">(</span><span class="n">n</span> <span class="k">:</span> <span class="kt">Int</span><span class="o">)</span> <span class="k">=</span> <span class="o">{</span>
<span class="k">var</span> <span class="n">r</span> <span class="k">=</span> <span class="mi">1</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">1</span> <span class="n">to</span> <span class="n">n</span><span class="o">)</span> <span class="n">r</span> <span class="k">=</span> <span class="n">r</span> <span class="o">*</span> <span class="n">i</span>
<span class="n">r</span> <span class="c1">// no need to use return keyword</span>
<span class="o">}</span>
<span class="c1">// recursive factorial must declare Int as return</span>
<span class="k">def</span> <span class="nf">fac</span><span class="o">(</span><span class="n">n</span><span class="k">:</span> <span class="kt">Int</span><span class="o">)</span><span class="k">:</span> <span class="kt">Int</span> <span class="o">=</span> <span class="nf">if</span> <span class="o">(</span><span class="n">n</span> <span class="o"><=</span> <span class="mi">0</span><span class="o">)</span> <span class="mi">1</span> <span class="k">else</span> <span class="n">n</span> <span class="o">*</span> <span class="nf">fac</span><span class="o">(</span><span class="n">n</span> <span class="o">-</span> <span class="mi">1</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Possible to define default value for parameters! To set a specific parameter, use named arguments!</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="k">def</span> <span class="nf">decorate</span><span class="o">(</span><span class="n">str</span><span class="k">:</span> <span class="kt">String</span><span class="o">,</span> <span class="n">left</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="s">"["</span><span class="o">,</span> <span class="n">right</span><span class="k">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="s">"]"</span><span class="o">)</span> <span class="k">=</span> <span class="n">left</span> <span class="o">+</span> <span class="n">str</span> <span class="o">+</span> <span class="n">right</span>
<span class="nf">println</span><span class="o">(</span><span class="nf">decorate</span><span class="o">(</span><span class="s">"Hello"</span><span class="o">))</span> <span class="c1">// [Hello]</span>
<span class="nf">println</span><span class="o">(</span><span class="nf">decorate</span><span class="o">(</span><span class="s">"Hello"</span><span class="o">,</span> <span class="s">"("</span><span class="o">))</span> <span class="c1">// (Hello]</span>
<span class="nf">println</span><span class="o">(</span><span class="nf">decorate</span><span class="o">(</span><span class="s">"Hello"</span><span class="o">,</span> <span class="n">right</span> <span class="k">=</span> <span class="s">")"</span><span class="o">))</span> <span class="c1">// [Hello)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>A procedure is just a named block with 0-to-many parameter(s), but without any return value. So in its syntax no <code class="language-plaintext highlighter-rouge">=</code>.</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
</pre></td><td class="rouge-code"><pre><span class="k">def</span> <span class="nf">box</span><span class="o">(</span><span class="n">s</span> <span class="k">:</span> <span class="kt">String</span><span class="o">)</span> <span class="o">{</span>
<span class="k">val</span> <span class="nv">border</span> <span class="k">=</span> <span class="s">"-"</span> <span class="o">*</span> <span class="o">(</span><span class="nv">s</span><span class="o">.</span><span class="py">length</span> <span class="o">+</span> <span class="mi">2</span><span class="o">)</span>
<span class="nf">print</span><span class="o">(</span><span class="n">f</span><span class="s">"$border%n|$s|%n$border%n"</span><span class="o">)</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Scala has another way for varargs in Java</p>
<ul>
<li>The type of <code class="language-plaintext highlighter-rouge">args</code> is <code class="language-plaintext highlighter-rouge">Seq</code></li>
<li>Like Java, these type of parameters must be defined as last ones</li>
<li><strong>Note</strong>: Not possible to set default value for other parameters when such parameter is declared</li>
</ul>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="k">def</span> <span class="nf">sum</span><span class="o">(</span><span class="n">args</span><span class="k">:</span> <span class="kt">Int*</span><span class="o">)</span> <span class="k">=</span> <span class="o">{</span>
<span class="k">var</span> <span class="n">result</span> <span class="k">=</span> <span class="mi">0</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">arg</span> <span class="k"><-</span> <span class="n">args</span><span class="o">)</span> <span class="n">result</span> <span class="o">+=</span> <span class="n">arg</span>
<span class="n">result</span>
<span class="o">}</span>
<span class="c1">// to call:</span>
<span class="nf">println</span><span class="o">(</span><span class="nf">sum</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="mi">66</span><span class="o">))</span> <span class="c1">// ok</span>
<span class="nf">println</span><span class="o">(</span><span class="nf">sum</span><span class="o">(</span><span class="mi">1</span> <span class="n">to</span> <span class="mi">5</span><span class="o">))</span> <span class="c1">// error</span>
<span class="nf">println</span><span class="o">(</span><span class="nf">sum</span><span class="o">(</span><span class="mi">1</span> <span class="n">to</span> <span class="mi">5</span><span class="k">:</span> <span class="k">_</span><span class="kt">*</span><span class="o">))</span> <span class="c1">// consider 1 to 5 as an argument sequence</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="arrays">Arrays</h2>
<p>Fixed-size Arrays</p>
<ul>
<li>Implemented as a Java array</li>
</ul>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">nums</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">Array</span><span class="o">[</span><span class="kt">Int</span><span class="o">](</span><span class="mi">10</span><span class="o">)</span> <span class="c1">// All initialized with zero</span>
<span class="k">val</span> <span class="nv">a</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">Array</span><span class="o">[</span><span class="kt">String</span><span class="o">](</span><span class="mi">10</span><span class="o">)</span> <span class="c1">// All initialized with null</span>
<span class="k">val</span> <span class="nv">s</span> <span class="k">=</span> <span class="nc">Array</span><span class="o">(</span><span class="s">"Hello"</span><span class="o">,</span> <span class="s">"World"</span><span class="o">)</span> <span class="c1">// No new operator!</span>
<span class="nf">s</span><span class="o">(</span><span class="mi">0</span><span class="o">)</span> <span class="k">=</span> <span class="s">"Goodbye"</span> <span class="c1">// Access element by (INDEX) not [INDEX]</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Variable-Length Arrays: <code class="language-plaintext highlighter-rouge">ArrayBuffer</code></p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nn">scala.collection.mutable.ArrayBuffer</span>
<span class="k">val</span> <span class="nv">b</span> <span class="k">=</span> <span class="nc">ArrayBuffer</span><span class="o">[</span><span class="kt">Int</span><span class="o">]()</span> <span class="c1">// also new ArrayBuffer[Int]</span>
<span class="n">b</span> <span class="o">+=</span> <span class="mi">1</span> <span class="c1">// append an element with +=</span>
<span class="n">b</span> <span class="o">+=</span> <span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">2</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="mi">5</span><span class="o">)</span> <span class="c1">// append multiple elements</span>
<span class="n">b</span> <span class="o">++=</span> <span class="nc">Array</span><span class="o">(</span><span class="mi">8</span><span class="o">,</span> <span class="mi">13</span><span class="o">,</span> <span class="mi">21</span><span class="o">)</span> <span class="c1">// append any collection with ++=</span>
<span class="nv">b</span><span class="o">.</span><span class="py">trimEnd</span><span class="o">(</span><span class="mi">5</span><span class="o">)</span> <span class="c1">// remove the last five elements</span>
<span class="nv">b</span><span class="o">.</span><span class="py">trimStart</span><span class="o">(</span><span class="mi">1</span><span class="o">)</span> <span class="c1">// remove the first element</span>
<span class="nv">b</span><span class="o">.</span><span class="py">insert</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="mi">6</span><span class="o">,</span> <span class="mi">7</span><span class="o">)</span> <span class="c1">// insert one or more elements from index 2 </span>
<span class="nv">b</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="mi">2</span><span class="o">)</span> <span class="c1">// remove element at index</span>
<span class="nv">b</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="mi">3</span><span class="o">)</span> <span class="c1">// from index 2, remove 3 elements</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>Traversing</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">b</span> <span class="k">=</span> <span class="nc">ArrayBuffer</span><span class="o">[</span><span class="kt">Int</span><span class="o">](</span><span class="mi">1</span><span class="o">,</span> <span class="mi">6</span><span class="o">,</span> <span class="mi">10</span><span class="o">)</span>
<span class="nf">for</span><span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="mi">0</span> <span class="n">until</span> <span class="nv">b</span><span class="o">.</span><span class="py">length</span><span class="o">)</span> <span class="c1">// all collections</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"$i = ${b(i)}"</span><span class="o">)</span>
<span class="nf">for</span><span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="nv">b</span><span class="o">.</span><span class="py">indices</span><span class="o">)</span> <span class="c1">// all indexed-collections (array, array buffer, list, ...)</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"$i = ${b(i)}"</span><span class="o">)</span>
<span class="nf">for</span><span class="o">(</span><span class="n">e</span> <span class="k"><-</span> <span class="n">b</span><span class="o">)</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"element = $e"</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>It is possible to use <a href="#for-comprehension">for comprehension</a> for array & array buffer</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">b</span> <span class="k">=</span> <span class="nc">ArrayBuffer</span><span class="o">[</span><span class="kt">Int</span><span class="o">](</span><span class="mi">1</span><span class="o">,</span> <span class="mi">6</span><span class="o">,</span> <span class="mi">10</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">p1</span><span class="k">:</span> <span class="kt">ArrayBuffer</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nf">for</span><span class="o">(</span><span class="n">e</span> <span class="k"><-</span> <span class="n">b</span><span class="o">)</span> <span class="k">yield</span> <span class="n">e</span> <span class="o">*</span> <span class="n">e</span>
<span class="k">val</span> <span class="nv">p2</span><span class="k">:</span> <span class="kt">Array</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nf">for</span><span class="o">(</span><span class="n">e</span> <span class="k"><-</span> <span class="nv">b</span><span class="o">.</span><span class="py">toArray</span><span class="o">)</span> <span class="k">yield</span> <span class="n">e</span> <span class="o">*</span> <span class="n">e</span>
<span class="k">val</span> <span class="nv">p3</span><span class="k">:</span> <span class="kt">ArrayBuffer</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nv">b</span><span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="nv">Math</span><span class="o">.</span><span class="py">pow</span><span class="o">(</span><span class="k">_</span><span class="o">,</span> <span class="mi">2</span><span class="o">).</span><span class="py">toInt</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">result</span> <span class="k">=</span> <span class="nf">for</span> <span class="o">(</span><span class="n">e</span> <span class="k"><-</span> <span class="n">b</span> <span class="k">if</span> <span class="n">e</span> <span class="o">>=</span> <span class="mi">0</span><span class="o">)</span> <span class="k">yield</span> <span class="n">e</span>
<span class="k">val</span> <span class="nv">positionsToRemove</span> <span class="k">=</span> <span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="nv">b</span><span class="o">.</span><span class="py">indices</span> <span class="k">if</span> <span class="nf">b</span><span class="o">(</span><span class="n">i</span><span class="o">)</span> <span class="o"><</span> <span class="mi">0</span><span class="o">)</span> <span class="k">yield</span> <span class="n">i</span>
<span class="nf">for</span><span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="nv">positionsToRemove</span><span class="o">.</span><span class="py">reverse</span><span class="o">)</span> <span class="nv">b</span><span class="o">.</span><span class="py">remove</span><span class="o">(</span><span class="n">i</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">positionsToKeep</span> <span class="k">=</span> <span class="nf">for</span> <span class="o">(</span><span class="n">i</span> <span class="k"><-</span> <span class="nv">b</span><span class="o">.</span><span class="py">indices</span> <span class="k">if</span> <span class="nf">b</span><span class="o">(</span><span class="n">i</span><span class="o">)</span> <span class="o">>=</span> <span class="mi">0</span><span class="o">)</span> <span class="k">yield</span> <span class="n">i</span>
<span class="nf">for</span> <span class="o">(</span><span class="n">j</span> <span class="k"><-</span> <span class="nv">positionsToKeep</span><span class="o">.</span><span class="py">indices</span><span class="o">)</span> <span class="nf">b</span><span class="o">(</span><span class="n">j</span><span class="o">)</span> <span class="k">=</span> <span class="nf">b</span><span class="o">(</span><span class="nf">positionsToKeep</span><span class="o">(</span><span class="n">j</span><span class="o">))</span>
<span class="nv">b</span><span class="o">.</span><span class="py">trimEnd</span><span class="o">(</span><span class="nv">b</span><span class="o">.</span><span class="py">length</span> <span class="o">-</span> <span class="nv">positionsToKeep</span><span class="o">.</span><span class="py">length</span><span class="o">)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li><code class="language-plaintext highlighter-rouge">p3</code> has the same result as <code class="language-plaintext highlighter-rouge">p2</code></li>
<li>Line 7, returns only positive elements but in a new ArrayBuffer</li>
<li>Lines 9-10 try to remove negative numbers from the original array buffer</li>
<li>Lines 12-13 try to remove negative numbers from the original array buffer, another way</li>
</ul>
<h2 id="collections">Collections</h2>
<p>Use <code class="language-plaintext highlighter-rouge">List()</code>, <code class="language-plaintext highlighter-rouge">Set()</code>, and <code class="language-plaintext highlighter-rouge">Map()</code> to create an <strong>immutable</strong> instance</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
</pre></td><td class="rouge-code"><pre><span class="k">val</span> <span class="nv">list</span> <span class="k">=</span> <span class="nc">List</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">2</span><span class="o">,</span> <span class="mi">3</span><span class="o">)</span>
<span class="nv">list</span><span class="o">.</span><span class="py">foreach</span><span class="o">(</span><span class="n">value</span> <span class="k">=></span> <span class="nf">println</span><span class="o">(</span><span class="n">value</span><span class="o">))</span>
<span class="nv">list</span><span class="o">.</span><span class="py">foreach</span><span class="o">(</span><span class="n">println</span><span class="o">)</span>
<span class="c1">// defining type of 'scores' is redundant!</span>
<span class="k">val</span> <span class="nv">scores</span><span class="k">:</span> <span class="kt">Map</span><span class="o">[</span><span class="kt">String</span>, <span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nc">Map</span><span class="o">((</span><span class="s">"A"</span><span class="o">,</span> <span class="mi">1</span><span class="o">),</span> <span class="o">(</span><span class="s">"b"</span><span class="o">,</span> <span class="mi">20</span><span class="o">),</span> <span class="s">"C"</span> <span class="o">-></span> <span class="mi">300</span><span class="o">)</span>
<span class="nv">scores</span><span class="o">.</span><span class="py">foreach</span><span class="o">((</span><span class="n">entry</span><span class="k">:</span> <span class="o">(</span><span class="kt">String</span><span class="o">,</span> <span class="kt">Int</span><span class="o">))</span> <span class="k">=></span>
<span class="nf">println</span><span class="o">(</span><span class="n">f</span><span class="s">"K=${entry._1}, V=${entry._2}%03d"</span><span class="o">))</span> <span class="c1">// formatted string</span>
<span class="c1">//or</span>
<span class="nf">for</span> <span class="o">((</span><span class="n">k</span><span class="o">,</span> <span class="n">v</span><span class="o">)</span> <span class="k"><-</span> <span class="n">scores</span><span class="o">)</span>
<span class="nf">println</span><span class="o">(</span><span class="n">f</span><span class="s">"K=$k, V=$v%03d"</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">r</span> <span class="k">=</span> <span class="nf">if</span><span class="o">(</span><span class="nv">scores</span><span class="o">.</span><span class="py">contains</span><span class="o">(</span><span class="s">"A"</span><span class="o">))</span> <span class="nf">scores</span><span class="o">(</span><span class="s">"A"</span><span class="o">)</span> <span class="k">else</span> <span class="mi">0</span>
<span class="c1">// or</span>
<span class="k">val</span> <span class="nv">r</span> <span class="k">=</span> <span class="nv">scores</span><span class="o">.</span><span class="py">getOrElse</span><span class="o">(</span><span class="s">"A"</span><span class="o">,</span> <span class="mi">0</span><span class="o">)</span>
<span class="k">val</span> <span class="nv">swap</span><span class="k">:</span> <span class="kt">Map</span><span class="o">[</span><span class="kt">Int</span>, <span class="kt">String</span><span class="o">]</span> <span class="k">=</span> <span class="nf">for</span> <span class="o">((</span><span class="n">k</span><span class="o">,</span> <span class="n">v</span><span class="o">)</span> <span class="k"><-</span> <span class="n">scores</span><span class="o">)</span> <span class="nf">yield</span> <span class="o">(</span><span class="n">v</span><span class="o">,</span> <span class="n">k</span><span class="o">)</span> <span class="c1">// Switch key & value</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Line 7,
<ul>
<li>The <code class="language-plaintext highlighter-rouge">entry</code> is a <em>tuple</em> of <code class="language-plaintext highlighter-rouge">(String, Int)</code></li>
<li><strong>Note</strong>: Unlike array or string positions, the component positions of a tuple start with 1, not 0.</li>
<li>The <code class="language-plaintext highlighter-rouge">f</code> makes the string formatted and acts like <code class="language-plaintext highlighter-rouge">printf</code> in Java. The <code class="language-plaintext highlighter-rouge">%03d</code> pads the <code class="language-plaintext highlighter-rouge">${entry._2}</code> with max two leading zeros</li>
</ul>
</li>
</ul>
<p>For <strong>mutable</strong> ones</p>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="c1">// M A P</span>
<span class="k">val</span> <span class="nv">updatingScores</span> <span class="k">=</span> <span class="nv">scala</span><span class="o">.</span><span class="py">collection</span><span class="o">.</span><span class="py">mutable</span><span class="o">.</span><span class="py">SortedMap</span><span class="o">[</span><span class="kt">String</span>, <span class="kt">Int</span><span class="o">]()</span>
<span class="nf">updatingScores</span><span class="o">(</span><span class="s">"A"</span><span class="o">)</span> <span class="k">=</span> <span class="mi">8</span>
<span class="n">updatingScores</span> <span class="o">+=</span> <span class="o">((</span><span class="s">"D"</span><span class="o">,</span> <span class="mi">10</span><span class="o">),</span> <span class="s">"B"</span> <span class="o">-></span> <span class="mi">10</span><span class="o">,</span> <span class="s">"C"</span> <span class="o">-></span> <span class="mi">7</span><span class="o">)</span>
<span class="n">updatingScores</span> <span class="o">-=</span> <span class="s">"C"</span> <span class="c1">// Remove by key</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="java-and-scala-interoperability">Java and Scala Interoperability</h2>
<ul>
<li>Scala arrays are implemented as Java arrays, so you can pass them back and forth between Java and Scala</li>
<li>Java’s collection needs conversion if Scala’s type are required!
<ul>
<li><code class="language-plaintext highlighter-rouge">import scala.collection.JavaConverters._</code> for <code class="language-plaintext highlighter-rouge">asScala</code></li>
<li><code class="language-plaintext highlighter-rouge">import scala.collection.mutable</code> the returning object is <em>mutable</em></li>
</ul>
</li>
</ul>
<div class="language-scala highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
</pre></td><td class="rouge-code"><pre><span class="k">import</span> <span class="nn">scala.collection.JavaConverters._</span>
<span class="k">import</span> <span class="nn">scala.collection.Searching.search</span>
<span class="k">import</span> <span class="nn">scala.collection.mutable</span>
<span class="k">import</span> <span class="nn">java.util.Arrays.binarySearch</span>
<span class="c1">//...</span>
<span class="k">val</span> <span class="nv">myJava</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">MyJava</span>
<span class="k">val</span> <span class="nv">fromJavaArr</span> <span class="k">=</span> <span class="nv">myJava</span><span class="o">.</span><span class="py">getArrayOfString</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">fromJavaArr</span><span class="o">.</span><span class="py">mkString</span><span class="o">(</span><span class="s">"{"</span><span class="o">,</span> <span class="s">", "</span><span class="o">,</span> <span class="s">")"</span><span class="o">))</span> <span class="c1">// output: {Hello, World)</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">myJava</span><span class="o">.</span><span class="py">sumArrayOfInteger</span><span class="o">(</span><span class="nc">Array</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">2</span><span class="o">,</span> <span class="mi">4</span><span class="o">)))</span> <span class="c1">// output: 7</span>
<span class="c1">// Error, no specific method for String type, although there is one for Object!</span>
<span class="c1">// java.util.Arrays.binarySearch(Array("b", "a").sorted, "a")</span>
<span class="k">val</span> <span class="nv">arr</span> <span class="k">=</span> <span class="nc">Array</span><span class="o">(</span><span class="s">"b"</span><span class="o">,</span> <span class="s">"a"</span><span class="o">,</span> <span class="s">"d"</span><span class="o">).</span><span class="py">sorted</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"idx=${binarySearch(arr.asInstanceOf[Array[Object]], "</span><span class="n">b</span><span class="s">")}"</span><span class="o">)</span> <span class="c1">// output: idx=1</span>
<span class="nf">println</span><span class="o">(</span><span class="n">s</span><span class="s">"idx=${arr.search("</span><span class="n">c</span><span class="s">")}"</span><span class="o">)</span> <span class="c1">// output: idx=InsertionPoint(2)</span>
<span class="k">val</span> <span class="nv">fromJavaList</span> <span class="k">=</span> <span class="nv">myJava</span><span class="o">.</span><span class="py">getListOfInteger</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">fromJavaList</span><span class="o">.</span><span class="py">stream</span><span class="o">().</span><span class="py">mapToInt</span><span class="o">(</span><span class="n">i</span> <span class="k">=></span> <span class="n">i</span><span class="o">).</span><span class="py">sum</span><span class="o">())</span> <span class="c1">// like Java i -> i</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">fromJavaList</span><span class="o">.</span><span class="py">stream</span><span class="o">().</span><span class="py">mapToInt</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">intValue</span><span class="o">).</span><span class="py">sum</span><span class="o">)</span> <span class="c1">// like Java i -> i.intValue</span>
<span class="k">val</span> <span class="nv">toScalaList</span><span class="k">:</span> <span class="kt">mutable.Seq</span><span class="o">[</span><span class="kt">Int</span><span class="o">]</span> <span class="k">=</span> <span class="nv">myJava</span><span class="o">.</span><span class="py">getListOfInteger</span>
<span class="o">.</span><span class="py">asScala</span> <span class="c1">// 1. convert list of Java to Scala</span>
<span class="o">.</span><span class="py">map</span><span class="o">(</span><span class="nv">_</span><span class="o">.</span><span class="py">toInt</span><span class="o">)</span> <span class="c1">// 2. convert Java's Integer to Scala's Int</span>
<span class="nf">println</span><span class="o">(</span><span class="nv">toScalaList</span><span class="o">.</span><span class="py">sum</span><span class="o">)</span> <span class="c1">// the sum needs 'List' & 'Int' of Scala</span>
<span class="k">val</span> <span class="nv">toScalaMap</span><span class="k">:</span> <span class="kt">mutable.Map</span><span class="o">[</span><span class="kt">String</span>, <span class="kt">Long</span><span class="o">]</span> <span class="k">=</span> <span class="nv">myJava</span><span class="o">.</span><span class="py">getAMap</span>
<span class="o">.</span><span class="py">asScala</span>
<span class="o">.</span><span class="py">map</span><span class="o">((</span><span class="n">t</span><span class="k">:</span> <span class="o">(</span><span class="kt">String</span><span class="o">,</span> <span class="kt">lang.Long</span><span class="o">))</span> <span class="k">=></span> <span class="o">(</span><span class="nv">t</span><span class="o">.</span><span class="py">_1</span><span class="o">,</span> <span class="nv">t</span><span class="o">.</span><span class="py">_2</span><span class="o">.</span><span class="py">toLong</span><span class="o">))</span>
<span class="nf">println</span><span class="o">(</span><span class="n">toScalaMap</span><span class="o">)</span> <span class="c1">// Map(A -> 0, B -> 5, C -> 10)</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Line 18, call Java’s <code class="language-plaintext highlighter-rouge">binarySearch</code> and passing an array of String, however a type conversion is necessary</li>
<li>Line 19
<ul>
<li><code class="language-plaintext highlighter-rouge">search</code> method requires <code class="language-plaintext highlighter-rouge">import scala.collection.Searching.search</code> for binary search</li>
<li>two types of result:
<ul>
<li><code class="language-plaintext highlighter-rouge">Found(n)</code>: the element is found at index <em>n</em></li>
<li><code class="language-plaintext highlighter-rouge">InsertionPoint(n)</code>: not found, however proposed insertion at index <em>n</em> (above, insert ‘c’ at index 2)</li>
</ul>
</li>
</ul>
</li>
<li>Line 27-30: two steps of conversion from Java to Scala
<ul>
<li>Line 28: call <code class="language-plaintext highlighter-rouge">asScala</code> (applicable on all Java collections) to convert to Scala equivalent</li>
<li>Line 29: call <code class="language-plaintext highlighter-rouge">map</code> to convert Java’s <code class="language-plaintext highlighter-rouge">Integer</code> to Scala’s <code class="language-plaintext highlighter-rouge">Int</code></li>
</ul>
</li>
</ul>Mehdi BizhaniA Summary for Scala Language, Part 01Java 008 - Mission: Stream2018-06-20T00:00:00+00:002018-06-20T00:00:00+00:00http://www.devocative.org/article/tech/java8-stream<p>In this part, a summary of <code class="language-plaintext highlighter-rouge">Stream</code> API in Java 8 is introduced. Although there are various tutorials in the Web,
this text attempts to highlight the important points. The previous part, <a href="/article/tech/java8-fp">Functional Programming</a>,
addressed functional programming and Lambda paradigm in Java 8.</p>
<h2 id="stream">Stream</h2>
<p><code class="language-plaintext highlighter-rouge">Stream</code> class, itself, is a Java interface with some static methods to create a stream. Some other classes
like collections has a specific method to return a Stream. The following sample demonstrates some of Stream API,
and the following sections describe those APIs.</p>
<p>The APIs can be divided into three groups</p>
<ul>
<li>Creating a Stream (<a href="#creation">Creation</a>)</li>
<li>Applying an intermediate processing operation on the elements and returning a new Stream (<a href="#transformation-&-resizing">Transformation & Resizing</a>)</li>
<li>Returning a proper result, also called <em>terminal operations</em> (<a href="#reduction-&-collection">Reduction & Collection</a>)</li>
</ul>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
</pre></td><td class="rouge-code"><pre><span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.io.IOException</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.nio.file.Files</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.nio.file.Path</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.*</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.function.Function</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.regex.Pattern</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.stream.Collectors</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.stream.LongStream</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.stream.Stream</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">static</span> <span class="n">org</span><span class="o">.</span><span class="na">junit</span><span class="o">.</span><span class="na">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">TestStream</span> <span class="o">{</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testFibonacci</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">Stream</span><span class="o"><</span><span class="kt">int</span><span class="o">[]></span> <span class="n">iterate</span><span class="o">;</span>
<span class="n">iterate</span> <span class="o">=</span> <span class="nc">Stream</span><span class="o">.</span><span class="na">iterate</span><span class="o">(</span><span class="k">new</span> <span class="kt">int</span><span class="o">[]{</span><span class="mi">1</span><span class="o">,</span> <span class="mi">1</span><span class="o">},</span> <span class="n">n</span> <span class="o">-></span> <span class="k">new</span> <span class="kt">int</span><span class="o">[]{</span><span class="n">n</span><span class="o">[</span><span class="mi">1</span><span class="o">],</span> <span class="n">n</span><span class="o">[</span><span class="mi">0</span><span class="o">]</span> <span class="o">+</span> <span class="n">n</span><span class="o">[</span><span class="mi">1</span><span class="o">]});</span>
<span class="kt">int</span> <span class="n">nth</span> <span class="o">=</span> <span class="n">iterate</span>
<span class="o">.</span><span class="na">peek</span><span class="o">(</span><span class="n">n</span> <span class="o">-></span> <span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">printf</span><span class="o">(</span><span class="s">"Debug: %s \n"</span><span class="o">,</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">toString</span><span class="o">(</span><span class="n">n</span><span class="o">)))</span>
<span class="o">.</span><span class="na">limit</span><span class="o">(</span><span class="mi">5</span><span class="o">)</span>
<span class="o">.</span><span class="na">reduce</span><span class="o">((</span><span class="n">a</span><span class="o">,</span> <span class="n">b</span><span class="o">)</span> <span class="o">-></span> <span class="n">b</span><span class="o">)</span>
<span class="o">.</span><span class="na">orElse</span><span class="o">(</span><span class="k">new</span> <span class="kt">int</span><span class="o">[]{</span><span class="mi">0</span><span class="o">,</span> <span class="mi">0</span><span class="o">})[</span><span class="mi">1</span><span class="o">];</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">8</span><span class="o">,</span> <span class="n">nth</span><span class="o">);</span>
<span class="n">iterate</span> <span class="o">=</span> <span class="nc">Stream</span><span class="o">.</span><span class="na">iterate</span><span class="o">(</span><span class="k">new</span> <span class="kt">int</span><span class="o">[]{</span><span class="mi">1</span><span class="o">,</span> <span class="mi">1</span><span class="o">},</span> <span class="n">n</span> <span class="o">-></span> <span class="k">new</span> <span class="kt">int</span><span class="o">[]{</span><span class="n">n</span><span class="o">[</span><span class="mi">1</span><span class="o">],</span> <span class="n">n</span><span class="o">[</span><span class="mi">0</span><span class="o">]</span> <span class="o">+</span> <span class="n">n</span><span class="o">[</span><span class="mi">1</span><span class="o">]});</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">list</span> <span class="o">=</span> <span class="n">iterate</span>
<span class="o">.</span><span class="na">limit</span><span class="o">(</span><span class="mi">5</span><span class="o">)</span>
<span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="n">n</span> <span class="o">-></span> <span class="n">n</span><span class="o">[</span><span class="mi">1</span><span class="o">])</span>
<span class="c1">//.collect(ArrayList::new, ArrayList::add, ArrayList::addAll)</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">list</span><span class="o">,</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">2</span><span class="o">,</span> <span class="mi">3</span><span class="o">,</span> <span class="mi">5</span><span class="o">,</span> <span class="mi">8</span><span class="o">));</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">test_Files_FlatMap_Distinct_Sorted_Reduction</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">IOException</span> <span class="o">{</span>
<span class="kd">final</span> <span class="nc">String</span> <span class="n">content</span> <span class="o">=</span> <span class="s">"test01 passed\ntest02 passed\ntest11 failed"</span><span class="o">;</span>
<span class="kd">final</span> <span class="nc">String</span> <span class="n">grepped</span> <span class="o">=</span> <span class="s">"test01 passed\ntest11 failed"</span><span class="o">;</span>
<span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">words</span> <span class="o">=</span>
<span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"test01"</span><span class="o">,</span> <span class="s">"passed"</span><span class="o">,</span> <span class="s">"test02"</span><span class="o">,</span> <span class="s">"passed"</span><span class="o">,</span> <span class="s">"test11"</span><span class="o">,</span> <span class="s">"failed"</span><span class="o">);</span>
<span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">distinctWords</span> <span class="o">=</span>
<span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"test01"</span><span class="o">,</span> <span class="s">"passed"</span><span class="o">,</span> <span class="s">"test02"</span><span class="o">,</span> <span class="s">"test11"</span><span class="o">,</span> <span class="s">"failed"</span><span class="o">);</span>
<span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">sortedDistinctWords</span> <span class="o">=</span>
<span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"test11"</span><span class="o">,</span> <span class="s">"test02"</span><span class="o">,</span> <span class="s">"test01"</span><span class="o">,</span> <span class="s">"passed"</span><span class="o">,</span> <span class="s">"failed"</span><span class="o">);</span>
<span class="kd">final</span> <span class="nc">Path</span> <span class="n">file</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">createTempFile</span><span class="o">(</span><span class="s">"__"</span><span class="o">,</span> <span class="s">"__"</span><span class="o">);</span>
<span class="nc">Files</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">file</span><span class="o">,</span> <span class="n">content</span><span class="o">.</span><span class="na">getBytes</span><span class="o">());</span>
<span class="c1">// Grepping lines containing '1'</span>
<span class="k">try</span> <span class="o">(</span><span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">lines</span><span class="o">(</span><span class="n">file</span><span class="o">))</span> <span class="o">{</span>
<span class="nc">String</span> <span class="n">result</span> <span class="o">=</span> <span class="n">lines</span>
<span class="o">.</span><span class="na">filter</span><span class="o">(</span><span class="n">line</span> <span class="o">-></span> <span class="n">line</span><span class="o">.</span><span class="na">contains</span><span class="o">(</span><span class="s">"1"</span><span class="o">))</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">joining</span><span class="o">(</span><span class="s">"\n"</span><span class="o">));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">grepped</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// List of words</span>
<span class="k">try</span> <span class="o">(</span><span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">lines</span><span class="o">(</span><span class="n">file</span><span class="o">))</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">result</span> <span class="o">=</span> <span class="n">lines</span>
<span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">line</span> <span class="o">-></span> <span class="nc">Stream</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">line</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)))</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">words</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// List of distinct words</span>
<span class="k">try</span> <span class="o">(</span><span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">lines</span><span class="o">(</span><span class="n">file</span><span class="o">))</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">result</span> <span class="o">=</span> <span class="n">lines</span>
<span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">line</span> <span class="o">-></span> <span class="nc">Stream</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">line</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)))</span>
<span class="o">.</span><span class="na">distinct</span><span class="o">()</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">distinctWords</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// List of distinct & descending-sorted words</span>
<span class="k">try</span> <span class="o">(</span><span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">lines</span><span class="o">(</span><span class="n">file</span><span class="o">))</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">result</span> <span class="o">=</span> <span class="n">lines</span>
<span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">line</span> <span class="o">-></span> <span class="nc">Stream</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">line</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)))</span>
<span class="o">.</span><span class="na">distinct</span><span class="o">()</span>
<span class="o">.</span><span class="na">sorted</span><span class="o">(</span><span class="nc">Comparator</span><span class="o">.</span><span class="na">reverseOrder</span><span class="o">())</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">sortedDistinctWords</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// List of distinct & descending-sorted words</span>
<span class="k">try</span> <span class="o">(</span><span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">lines</span><span class="o">(</span><span class="n">file</span><span class="o">))</span> <span class="o">{</span>
<span class="nc">String</span> <span class="n">result</span> <span class="o">=</span> <span class="n">lines</span>
<span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">line</span> <span class="o">-></span> <span class="nc">Stream</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">line</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)))</span>
<span class="o">.</span><span class="na">distinct</span><span class="o">()</span>
<span class="o">.</span><span class="na">sorted</span><span class="o">(</span><span class="nc">Comparator</span><span class="o">.</span><span class="na">reverseOrder</span><span class="o">())</span>
<span class="o">.</span><span class="na">findFirst</span><span class="o">()</span> <span class="c1">// min(Comparator.reverseOrder()) instead of sorted() & findFirst()</span>
<span class="o">.</span><span class="na">get</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"test11"</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// Count number of words</span>
<span class="k">try</span> <span class="o">(</span><span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="nc">Files</span><span class="o">.</span><span class="na">lines</span><span class="o">(</span><span class="n">file</span><span class="o">))</span> <span class="o">{</span>
<span class="kt">long</span> <span class="n">result</span> <span class="o">=</span> <span class="n">lines</span>
<span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">line</span> <span class="o">-></span> <span class="nc">Stream</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="n">line</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)))</span>
<span class="o">.</span><span class="na">count</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">words</span><span class="o">.</span><span class="na">size</span><span class="o">(),</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// Count number of characters of words (1/2)</span>
<span class="nc">String</span> <span class="n">fileAsStr</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">String</span><span class="o">(</span><span class="nc">Files</span><span class="o">.</span><span class="na">readAllBytes</span><span class="o">(</span><span class="n">file</span><span class="o">));</span>
<span class="kt">long</span> <span class="n">result</span> <span class="o">=</span> <span class="nc">Pattern</span><span class="o">.</span><span class="na">compile</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)</span>
<span class="o">.</span><span class="na">splitAsStream</span><span class="o">(</span><span class="n">fileAsStr</span><span class="o">)</span>
<span class="o">.</span><span class="na">mapToLong</span><span class="o">(</span><span class="nl">String:</span><span class="o">:</span><span class="n">length</span><span class="o">)</span>
<span class="o">.</span><span class="na">sum</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">36</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="c1">// Count number of characters of words (2/2)</span>
<span class="n">fileAsStr</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">String</span><span class="o">(</span><span class="nc">Files</span><span class="o">.</span><span class="na">readAllBytes</span><span class="o">(</span><span class="n">file</span><span class="o">));</span>
<span class="n">result</span> <span class="o">=</span> <span class="nc">Pattern</span><span class="o">.</span><span class="na">compile</span><span class="o">(</span><span class="s">"\\s"</span><span class="o">)</span>
<span class="o">.</span><span class="na">splitAsStream</span><span class="o">(</span><span class="n">fileAsStr</span><span class="o">)</span>
<span class="o">.</span><span class="na">reduce</span><span class="o">(</span><span class="mi">0L</span><span class="o">,</span>
<span class="o">(</span><span class="n">total</span><span class="o">,</span> <span class="n">word</span><span class="o">)</span> <span class="o">-></span> <span class="n">total</span> <span class="o">+</span> <span class="n">word</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span>
<span class="o">(</span><span class="n">total1</span><span class="o">,</span> <span class="n">total2</span><span class="o">)</span> <span class="o">-></span> <span class="n">total1</span> <span class="o">+</span> <span class="n">total2</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">36</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testFactorial</span><span class="o">()</span> <span class="o">{</span>
<span class="kt">long</span> <span class="n">result</span> <span class="o">=</span> <span class="nc">LongStream</span>
<span class="c1">//.range(1, 5) [1, 5)</span>
<span class="o">.</span><span class="na">rangeClosed</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">5</span><span class="o">)</span> <span class="c1">// [1, 5]</span>
<span class="o">.</span><span class="na">reduce</span><span class="o">((</span><span class="n">left</span><span class="o">,</span> <span class="n">right</span><span class="o">)</span> <span class="o">-></span> <span class="n">left</span> <span class="o">*</span> <span class="n">right</span><span class="o">)</span>
<span class="o">.</span><span class="na">orElse</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">120</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="n">result</span> <span class="o">=</span> <span class="nc">LongStream</span>
<span class="c1">//.range(1, 5) [1, 5)</span>
<span class="o">.</span><span class="na">rangeClosed</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="mi">5</span><span class="o">)</span> <span class="c1">// [1, 5]</span>
<span class="o">.</span><span class="na">reduce</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="o">(</span><span class="n">left</span><span class="o">,</span> <span class="n">right</span><span class="o">)</span> <span class="o">-></span> <span class="n">left</span> <span class="o">*</span> <span class="n">right</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">120</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testNumberStream</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Integer</span><span class="o">></span> <span class="n">integers</span> <span class="o">=</span> <span class="nc">DoubleStream</span>
<span class="o">.</span><span class="na">generate</span><span class="o">(</span><span class="nl">Math:</span><span class="o">:</span><span class="n">random</span><span class="o">)</span> <span class="c1">// generate unlimited by calling Math.random</span>
<span class="o">.</span><span class="na">mapToInt</span><span class="o">(</span><span class="n">d</span> <span class="o">-></span> <span class="o">(</span><span class="kt">int</span><span class="o">)</span> <span class="o">(</span><span class="n">d</span> <span class="o">*</span> <span class="mi">10</span><span class="o">))</span> <span class="c1">// convert to IntStream</span>
<span class="o">.</span><span class="na">limit</span><span class="o">(</span><span class="mi">10</span><span class="o">)</span> <span class="c1">// limit to first 10 elements</span>
<span class="o">.</span><span class="na">boxed</span><span class="o">()</span> <span class="c1">// convert to Stream<Integer></span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">());</span> <span class="c1">// collect to a list</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">10</span><span class="o">,</span> <span class="n">integers</span><span class="o">.</span><span class="na">size</span><span class="o">());</span>
<span class="nc">StringBuilder</span> <span class="n">password</span> <span class="o">=</span> <span class="nc">DoubleStream</span>
<span class="o">.</span><span class="na">generate</span><span class="o">(</span><span class="nl">Math:</span><span class="o">:</span><span class="n">random</span><span class="o">)</span>
<span class="o">.</span><span class="na">mapToInt</span><span class="o">(</span><span class="n">d</span> <span class="o">-></span> <span class="o">(</span><span class="kt">int</span><span class="o">)</span> <span class="o">(</span><span class="n">d</span> <span class="o">*</span> <span class="mi">1000</span><span class="o">))</span>
<span class="o">.</span><span class="na">filter</span><span class="o">(</span><span class="n">value</span> <span class="o">-></span> <span class="o">(</span><span class="n">value</span> <span class="o">>=</span> <span class="sc">'A'</span> <span class="o">&&</span> <span class="n">value</span> <span class="o"><=</span> <span class="sc">'Z'</span><span class="o">)</span> <span class="o">||</span>
<span class="o">(</span><span class="n">value</span> <span class="o">>=</span> <span class="sc">'a'</span> <span class="o">&&</span> <span class="n">value</span> <span class="o"><=</span> <span class="sc">'z'</span><span class="o">)</span> <span class="o">||</span>
<span class="o">(</span><span class="n">value</span> <span class="o">>=</span> <span class="sc">'0'</span> <span class="o">&&</span> <span class="n">value</span> <span class="o"><=</span> <span class="sc">'9'</span><span class="o">))</span>
<span class="o">.</span><span class="na">limit</span><span class="o">(</span><span class="mi">10</span><span class="o">)</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nl">StringBuilder:</span><span class="o">:</span><span class="k">new</span><span class="o">,</span> <span class="nl">StringBuilder:</span><span class="o">:</span><span class="n">appendCodePoint</span><span class="o">,</span> <span class="nl">StringBuilder:</span><span class="o">:</span><span class="n">append</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">10</span><span class="o">,</span> <span class="n">password</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">length</span><span class="o">());</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testCollectors</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Employee</span><span class="o">></span> <span class="n">list</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"John"</span><span class="o">,</span> <span class="mi">5000</span><span class="o">),</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"Jack"</span><span class="o">,</span> <span class="mi">6000</span><span class="o">),</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"Jack"</span><span class="o">,</span> <span class="mi">7000</span><span class="o">),</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"Bill"</span><span class="o">,</span> <span class="mi">3000</span><span class="o">));</span>
<span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span> <span class="nc">Employee</span><span class="o">></span> <span class="n">name2employee</span> <span class="o">=</span> <span class="n">list</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toMap</span><span class="o">(</span><span class="nl">Employee:</span><span class="o">:</span><span class="n">getName</span><span class="o">,</span> <span class="nc">Function</span><span class="o">.</span><span class="na">identity</span><span class="o">(),</span> <span class="o">(</span><span class="n">curV</span><span class="o">,</span> <span class="n">newV</span><span class="o">)</span> <span class="o">-></span> <span class="n">newV</span><span class="o">));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">3</span><span class="o">,</span> <span class="n">name2employee</span><span class="o">.</span><span class="na">size</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">7000</span><span class="o">,</span> <span class="n">name2employee</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"Jack"</span><span class="o">).</span><span class="na">getSalary</span><span class="o">().</span><span class="na">intValue</span><span class="o">());</span>
<span class="kd">final</span> <span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Employee</span><span class="o">>></span> <span class="n">name2employees</span> <span class="o">=</span> <span class="n">list</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">groupingBy</span><span class="o">(</span><span class="nl">Employee:</span><span class="o">:</span><span class="n">getName</span><span class="o">,</span> <span class="nl">LinkedHashMap:</span><span class="o">:</span><span class="k">new</span><span class="o">,</span> <span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">()));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"John"</span><span class="o">,</span> <span class="n">name2employees</span><span class="o">.</span><span class="na">keySet</span><span class="o">().</span><span class="na">stream</span><span class="o">().</span><span class="na">findFirst</span><span class="o">().</span><span class="na">get</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">3</span><span class="o">,</span> <span class="n">name2employees</span><span class="o">.</span><span class="na">size</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="n">name2employees</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"Bill"</span><span class="o">).</span><span class="na">size</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="n">name2employees</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"Jack"</span><span class="o">).</span><span class="na">size</span><span class="o">());</span>
<span class="kd">final</span> <span class="kt">int</span> <span class="n">averageSalary</span> <span class="o">=</span> <span class="o">(</span><span class="kt">int</span><span class="o">)</span> <span class="n">list</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">mapToInt</span><span class="o">(</span><span class="nl">Employee:</span><span class="o">:</span><span class="n">getSalary</span><span class="o">)</span>
<span class="o">.</span><span class="na">average</span><span class="o">()</span>
<span class="o">.</span><span class="na">orElse</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">5250</span><span class="o">,</span> <span class="n">averageSalary</span><span class="o">);</span>
<span class="kd">final</span> <span class="nc">Map</span><span class="o"><</span><span class="nc">Boolean</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Employee</span><span class="o">>></span> <span class="n">highSalaryEmployees</span> <span class="o">=</span> <span class="n">list</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">partitioningBy</span><span class="o">(</span><span class="n">emp</span><span class="o">-></span> <span class="n">emp</span><span class="o">.</span><span class="na">getSalary</span><span class="o">()</span> <span class="o">></span> <span class="n">averageSalary</span><span class="o">));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="n">highSalaryEmployees</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="kc">true</span><span class="o">).</span><span class="na">size</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="n">highSalaryEmployees</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="kc">false</span><span class="o">).</span><span class="na">size</span><span class="o">());</span>
<span class="o">}</span>
<span class="c1">// ------------------------------</span>
<span class="kd">class</span> <span class="nc">Employee</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">String</span> <span class="n">name</span><span class="o">;</span>
<span class="kd">private</span> <span class="nc">Integer</span> <span class="n">salary</span><span class="o">;</span>
<span class="nc">Employee</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">Integer</span> <span class="n">salary</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">name</span> <span class="o">=</span> <span class="n">name</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">salary</span> <span class="o">=</span> <span class="n">salary</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">name</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">Integer</span> <span class="nf">getSalary</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">salary</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">+</span> <span class="s">", "</span> <span class="o">+</span> <span class="n">getSalary</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="creation">Creation</h3>
<p>The following methods return a <code class="language-plaintext highlighter-rouge">Stream</code> object.</p>
<table>
<tr>
<th>API</th>
<th>Description</th>
</tr>
<tr>
<td colspan="2" align="center" style="background-color:#aaaa00">
<i>
Interface Static Methods (also valid for <em>IntStream</em>, <em>LongStream</em> and <em>DoubleStream</em>)
</i>
</td>
</tr>
<tr>
<td>
<code>Stream.of(T... varg)</code>
</td>
<td>
<code>Stream<String> stream = Stream.of("1", "5", "7")</code>
</td>
</tr>
<tr>
<td>
<code>Stream.generate(Supplier)</code>
</td>
<td>
<p><code>Stream<Double> randoms = Stream.generate(Math::random)</code></p>
<ul>
<li>Returns an infinite sequential unordered stream</li>
<li>Each element is generated by the provided <code>Supplier</code></li>
<li>Suitable for generating constant streams, streams of random elements, etc</li>
</ul>
</td>
</tr>
<tr>
<td>
<pre>
Stream.iterate(
T seed,
UnaryOperator<T>)</pre>
</td>
<td>
<ul>
<li>Returns an infinite sequential ordered Stream</li>
<li>Produced by iterative application of a function <code>f</code> to an initial element <code>seed</code></li>
<li>Consisting of <code>seed</code>, <code>f(seed)</code>, <code>f(f(seed))</code>, etc.</li>
</ul>
</td>
</tr>
<tr>
<td colspan="2" align="center" style="background-color:#aaaa00">
<i>Object's Method Returning a Stream</i>
</td>
</tr>
<tr>
<td colspan="2">
<code>COLLECTION_VAR.stream()</code> or
<code>COLLECTION_VAR.parallelStream()</code><br />
</td>
</tr>
<tr>
<td colspan="2">
<code>Stream<String> words = Pattern.compile("\\s").splitAsStream(CONTENT_VAR)</code>
</td>
</tr>
<tr>
<td colspan="2">
<code>Stream<String> lines = Files.lines(PATH_VAR)</code>
<figure class="highlight"><pre><code class="language-java" data-lang="java"><span class="nc">BufferedReader</span> <span class="n">reader</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BufferedReader</span><span class="o">(...)</span>
<span class="cm">/*
Note: new BufferedReader(new InputStreamReader(INPUT_STREAM_OBJ))
so any InputStream can be converted to Stream<String>
*/</span>
<span class="nc">Stream</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">lines</span> <span class="o">=</span> <span class="n">reader</span><span class="o">.</span><span class="na">lines</span><span class="o">()</span></code></pre></figure>
</td>
</tr>
<tr>
<td colspan="2">
<code>IntStream chars = "جاوا".codePoints()</code>
</td>
</tr>
<tr>
<td colspan="2">
<code>IntStream stream = new Random().ints()</code><br />
<code>LongStream stream = new Random().longs()</code><br />
<code>DoubleStream stream = new Random().doubles()</code>
</td>
</tr>
<tr>
<td colspan="2">
<code>Stream<T> s = Arrays.stream(T[] array)</code><br />
<code>Stream<T> s = Arrays.stream(T[] array, int startInclusive, int endExclusive)</code>
</td>
</tr>
</table>
<h3 id="transformation--resizing">Transformation & Resizing</h3>
<p>The following methods return an <code class="language-plaintext highlighter-rouge">Stream</code> or Stream-based object.</p>
<table>
<tr>
<th>API</th>
<th>Description</th>
</tr>
<tr>
<td><code>filter(Predicate<? super T> p)</code></td>
<td>
<pre>
// T t -> boolean
public interface Predicate<T> {
boolean test(T t);
}</pre>
so
<code>filter(n -> n > 12)</code>
</td>
</tr>
<tr>
<td><code>map(Function<? super T, ? extends R> mapper)</code></td>
<td>
<pre>
// T t -> R
public interface Function<T, R> {
R apply(T t);
}</pre>
so
<code>map(s -> s.length())</code>
</td>
</tr>
<tr>
<td><code>mapToInt(ToIntFunction<? super T> mapper): <b>IntStream</b></code></td>
<td>
<pre>
// T t -> int
public interface ToIntFunction<T> {
int applyAsInt(T value);
}</pre>
so
<code>mapToInt(s -> s.length())</code>
</td>
</tr>
<tr>
<td colspan="2">
<code>mapToLong()</code> and <code>mapToDouble()</code> are similar to the above.
</td>
</tr>
<tr>
<td>
<pre>
flatMap(Function<? super T,
? extends Stream<? extends R>> mapper)</pre>
</td>
<td>
<pre>
Stream<String> lines = Files.lines(path)
Stream<String> words = lines.flatMap(
line -> Stream.of(line.split(" +")))</pre>
</td>
</tr>
<tr>
<td><code>limit(long n)</code></td>
<td>
Returns a stream consisting of the <em>first n</em> elements in the encounter order,
so it can be quite expensive on ordered parallel pipelines.
<br />
<blockquote>
<b>Note</b>: If ordered elements is required, and there is poor performance or memory utilization with
limit() in parallel pipelines, switching to sequential execution with sequential() may improve performance.
</blockquote>
</td>
</tr>
<tr>
<td><code>skip(long n)</code></td>
<td>
Returns a stream remaining of the elements after discarding the <em>first n</em> elements in the encounter order,
so it can be quite expensive on ordered parallel pipelines.
<br />
If this stream contains fewer than <em>n</em> elements then an empty stream will be returned.
<br />
<b>Note</b>: the note in limit()
</td>
</tr>
<tr>
<td><code>distinct()</code></td>
<td>
Returns a stream consisting of the distinct elements (according to equals()).
<br />
For ordered streams, the selection of distinct elements is stable, however for unordered streams no stability guarantees are made.
<br />
<b>Note</b>: the note in limit()
</td>
</tr>
<tr>
<td>
<code>sorted()</code>
<br />
<code>sorted(Comparator<? super T> comparator)</code>
</td>
<td>
Returns a stream of sorted elements according to natural order or given <em>comparator</em>.
<br />
For ordered streams, the sort is stable, however for unordered streams no stability guarantees are made.
</td>
</tr>
</table>
<h3 id="reduction--collection">Reduction & Collection</h3>
<ul>
<li>These methods are <em>terminal operations</em> and get the final answer from a <code class="language-plaintext highlighter-rouge">Stream</code>.</li>
<li>Reduction ones mostly return an <code class="language-plaintext highlighter-rouge">Optional</code> object</li>
<li>Collection ones mostly return a <code class="language-plaintext highlighter-rouge">Collection</code></li>
<li>After calling these methods, the <code class="language-plaintext highlighter-rouge">Stream</code> object is closed</li>
</ul>
<table>
<tr>
<th>API</th>
<th>Description</th>
</tr>
<tr>
<td colspan="2" align="center" style="background-color:#aaaa00">
<i>
Reduction
</i>
</td>
</tr>
<tr>
<td><code>findFirst(): Optional</code></td>
<td>
Returns an Optional describing the first element
</td>
</tr>
<tr>
<td><code>findAny(): Optional</code></td>
<td>
Returns an Optional element of the stream.
<br />
The behavior is nondeterministic, so it is effective when you parallelize the stream
and the first match in any of the examined segments will complete the computation.
</td>
</tr>
<tr>
<td><code>anyMatch(Predicate): boolean</code></td>
<td>
Returns whenever any elements of this stream match the provided predicate
</td>
</tr>
<tr>
<td colspan="2">There are <code>allMatch()</code> and <code>noneMatch()</code>, the same syntax as above,
that return true if all or no elements match a predicate</td>
</tr>
<tr>
<td><code>reduce(BinaryOperator<T> accumulator): Optional</code></td>
<td>
<pre>
// T t, T t -> T
public interface BinaryOperator<T> {
T apply(T t, T t);
}</pre>
Performs a reduction on the elements of this stream, using an <b>associative accumulation function</b> (e.g.
sum and product, string concatenation, maximum and minimum, set union and intersection),
and returns an Optional describing the reduced value.
<br />
<b>Note</b>: in fact <code>BinaryOperator<T> extends BiFunction<T, T, T></code>
</td>
</tr>
<tr>
<td><code>count(): long</code></td>
<td>
Count the elements in this stream.
<br />
This is a special case of reduction equivalent to:
<br />
<code>return mapToLong(e -> 1L).sum();</code>
</td>
</tr>
<tr>
<td>
<code>max()</code>
<code>min()</code>
<code>sum()</code>
<code>average()</code>
<code>summaryStatistics()</code>
</td>
<td>
The related mathematical function is applied on the numerical elements of the Stream.
So the stream must be <code>IntStream</code>, <code>LongStream</code>, or <code>DoubleStream</code>.
</td>
</tr>
<tr>
<td colspan="2" align="center" style="background-color:#aaaa00">
<i>
Collection
</i>
</td>
</tr>
<tr>
<td>
<pre>
<R> R collect(
Supplier<R> supplier,
BiConsumer<R, ? super T> accumulator,
BiConsumer<R, R> combiner)</pre>
</td>
<td>
<pre>stream.collect(
ArrayList::new,
ArrayList::add,
ArrayList::addAll)</pre>
</td>
</tr>
<tr>
<td>
<code>collect(Collector collector)</code>
</td>
<td>
<code>collect(Collectors.asList())</code>
<br />
<code>collect(Collectors.asSet())</code>
<br />
<code>collect(Collectors.toCollection(LinkedHashSet::new))</code>
<br />
<br />
<code>collect(Collectors.joining())</code>
<br />
<code>collect(Collectors.joining(","))</code>
<br />
If your stream contains objects other than strings, you need to first convert them to strings, like this:
<code>stream.map(Object::toString).collect(Collectors.joining(","))</code>
<br />
<br />
<pre>
// map name to salary
collect(Collectors.toMap(
Employee::getName,
Employee::getSalary))</pre>
<pre>
// map name to employee
collect(Collectors.toMap(
Employee::getName,
Function.identity()))</pre>
Note: Duplicate key results in exception in previous two toMap(), however the following trie to handle it!
<pre>
// map name to employee, on duplicate key use first one
collect(Collectors.toMap(
Employee::getName,
Function.identity(),
(curVal, newVal) -> curVal))</pre>
<br />
<pre>collect(Collectors.groupingBy(
Employee::getName): Map<String, List<Employee>></pre>
<pre>collect(Collectors.groupingBy(
Employee::getName,
LinkedHashMap::new,
Collectors.toList())): Map<String, List<Employee>></pre>
</td>
</tr>
<tr>
<td>
<code><A> A[] toArray(IntFunction<A[]> generator)</code>
</td>
<td>
<pre>
// int v -> R
public interface IntFunction<R> {
R apply(int value);
}</pre>
The most common call is <code>toArray(TYPE[]::new)</code> (constructor reference)
</td>
</tr>
</table>
<h3 id="another-example">Another Example</h3>
<p>This a simplified version of a real example. Suppose there are two entities: <code class="language-plaintext highlighter-rouge">Report</code> & <code class="language-plaintext highlighter-rouge">Group</code>. These two entities has a
many-to-many association, navigable from <code class="language-plaintext highlighter-rouge">Report</code> to <code class="language-plaintext highlighter-rouge">Group</code> (following code). Now, when a list of <code class="language-plaintext highlighter-rouge">Report</code> objects is
fetched from database, each <code class="language-plaintext highlighter-rouge">Report</code> knows its <code class="language-plaintext highlighter-rouge">Group</code>, but for presentation, a reverse relation is needed,
and it is required to group <code class="language-plaintext highlighter-rouge">Report</code>s by <code class="language-plaintext highlighter-rouge">Group</code>. The following code shows the solution:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
</pre></td><td class="rouge-code"><pre><span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.*</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.stream.Collectors</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">static</span> <span class="n">org</span><span class="o">.</span><span class="na">junit</span><span class="o">.</span><span class="na">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">TestManyToMany</span> <span class="o">{</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">test</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">Group</span> <span class="n">scm</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Group</span><span class="o">(</span><span class="s">"SCM"</span><span class="o">);</span>
<span class="nc">Group</span> <span class="n">mtc</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Group</span><span class="o">(</span><span class="s">"MTC"</span><span class="o">);</span>
<span class="nc">Group</span> <span class="n">hse</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Group</span><span class="o">(</span><span class="s">"HSE"</span><span class="o">);</span>
<span class="nc">Report</span> <span class="n">stock</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Report</span><span class="o">(</span><span class="s">"Stock Inventory"</span><span class="o">,</span> <span class="n">scm</span><span class="o">,</span> <span class="n">mtc</span><span class="o">);</span>
<span class="nc">Report</span> <span class="n">incid</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Report</span><span class="o">(</span><span class="s">"Incidents"</span><span class="o">,</span> <span class="n">hse</span><span class="o">);</span>
<span class="nc">Report</span> <span class="n">artcl</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Report</span><span class="o">(</span><span class="s">"Accounting Articles"</span><span class="o">,</span> <span class="n">scm</span><span class="o">,</span> <span class="n">mtc</span><span class="o">,</span> <span class="n">hse</span><span class="o">);</span>
<span class="nc">Report</span> <span class="n">mttr</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Report</span><span class="o">(</span><span class="s">"MTTR"</span><span class="o">,</span> <span class="n">mtc</span><span class="o">);</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Report</span><span class="o">></span> <span class="n">reports</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">stock</span><span class="o">,</span> <span class="n">incid</span><span class="o">,</span> <span class="n">artcl</span><span class="o">,</span> <span class="n">mttr</span><span class="o">);</span>
<span class="nc">Map</span><span class="o"><</span><span class="nc">Group</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Report</span><span class="o">>></span> <span class="n">expected</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TreeMap</span><span class="o"><>();</span>
<span class="n">expected</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">scm</span><span class="o">,</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">stock</span><span class="o">,</span> <span class="n">artcl</span><span class="o">));</span>
<span class="n">expected</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">mtc</span><span class="o">,</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">stock</span><span class="o">,</span> <span class="n">artcl</span><span class="o">,</span> <span class="n">mttr</span><span class="o">));</span>
<span class="n">expected</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">hse</span><span class="o">,</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">incid</span><span class="o">,</span> <span class="n">artcl</span><span class="o">));</span>
<span class="nc">Map</span><span class="o"><</span><span class="nc">Group</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Report</span><span class="o">>></span> <span class="n">result</span> <span class="o">=</span> <span class="n">reports</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span class="n">report</span> <span class="o">-></span>
<span class="n">report</span><span class="o">.</span><span class="na">getGroups</span><span class="o">().</span><span class="na">stream</span><span class="o">().</span><span class="na">map</span><span class="o">(</span><span class="n">dataGroup</span> <span class="o">-></span>
<span class="k">new</span> <span class="nc">AbstractMap</span><span class="o">.</span><span class="na">SimpleEntry</span><span class="o"><>(</span><span class="n">dataGroup</span><span class="o">,</span> <span class="n">report</span><span class="o">)</span>
<span class="o">)</span>
<span class="o">)</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">groupingBy</span><span class="o">(</span>
<span class="nc">AbstractMap</span><span class="o">.</span><span class="na">SimpleEntry</span><span class="o">::</span><span class="n">getKey</span><span class="o">,</span>
<span class="nl">TreeMap:</span><span class="o">:</span><span class="k">new</span><span class="o">,</span>
<span class="nc">Collectors</span><span class="o">.</span><span class="na">mapping</span><span class="o">(</span>
<span class="nc">AbstractMap</span><span class="o">.</span><span class="na">SimpleEntry</span><span class="o">::</span><span class="n">getValue</span><span class="o">,</span>
<span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">()))</span>
<span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">expected</span><span class="o">,</span> <span class="n">result</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// ------------------------------</span>
<span class="kd">private</span> <span class="kd">class</span> <span class="nc">Report</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">String</span> <span class="n">name</span><span class="o">;</span>
<span class="kd">private</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Group</span><span class="o">></span> <span class="n">groups</span><span class="o">;</span>
<span class="nc">Report</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">Group</span><span class="o">...</span> <span class="n">groups</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">name</span> <span class="o">=</span> <span class="n">name</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">groups</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">groups</span><span class="o">);</span>
<span class="o">}</span>
<span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">name</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Group</span><span class="o">></span> <span class="nf">getGroups</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">groups</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">getName</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// ------------------------------</span>
<span class="kd">private</span> <span class="kd">class</span> <span class="nc">Group</span> <span class="kd">implements</span> <span class="nc">Comparable</span><span class="o"><</span><span class="nc">Group</span><span class="o">></span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">String</span> <span class="n">name</span><span class="o">;</span>
<span class="nc">Group</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">name</span> <span class="o">=</span> <span class="n">name</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">name</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">getName</span><span class="o">();</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">int</span> <span class="nf">compareTo</span><span class="o">(</span><span class="nc">Group</span> <span class="n">o</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">getName</span><span class="o">().</span><span class="na">compareTo</span><span class="o">(</span><span class="n">o</span><span class="o">.</span><span class="na">getName</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>Lines 30 to 34 act like a <code class="language-plaintext highlighter-rouge">Cartesian Product</code> and the result is tuples of <code class="language-plaintext highlighter-rouge">(Group, Report)</code></li>
<li><code class="language-plaintext highlighter-rouge">AbstractMap.SimpleEntry</code> is used as the data structure for tuple</li>
<li>Lines 38 to 40 create a <code class="language-plaintext highlighter-rouge">mapping</code> from <code class="language-plaintext highlighter-rouge">AbstractMap.SimpleEntry<Group, Report></code> to <code class="language-plaintext highlighter-rouge">Report</code></li>
</ul>Mehdi BizhaniAn introduction to Java 8, StreamJava 008 - Mission: FP2018-06-06T00:00:00+00:002018-06-06T00:00:00+00:00http://www.devocative.org/article/tech/java8-fp<p>This part tries to summarise general paradigm of <code class="language-plaintext highlighter-rouge">Functional Programming</code> in Java 8. Future parts cover other aspects of Java 8.</p>
<h2 id="functional-programming">Functional Programming</h2>
<ul>
<li>Block of code with parameters</li>
<li>Block of code executed at a later point (i.e. callback)</li>
<li>Well suited for concurrent and event-driven (or “reactive”) programming</li>
<li>Method and constructor references refer to methods or constructors without invoking them</li>
<li>Add default and static methods to interfaces
<ul>
<li>Any conflicts between default methods from multiple interfaces must be resolved manually</li>
</ul>
</li>
<li>Lambda expression
<ul>
<li>More efficient than using traditional inner classes</li>
<li>Access <em>effectively final variables</em> from the enclosing scope</li>
<li>Applicable on an object of an <strong>interface</strong> with a single abstract method! Such an interface is called a <strong><code class="language-plaintext highlighter-rouge">functional interface</code></strong>
<ul>
<li>Such interface can be tagged by <code class="language-plaintext highlighter-rouge">@FunctionalInterface</code></li>
<li>The <em>compiler</em> asserts that the annotated entity is an interface with a single abstract method</li>
</ul>
</li>
</ul>
</li>
</ul>
<h3 id="general-syntax">General Syntax</h3>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
</pre></td><td class="rouge-code"><pre><span class="cm">/* General Syntax (similar to other functional declaration in other languages,
(v1, v2, ...) -> return_type) */</span>
<span class="o">([[</span><span class="no">TYPE1</span><span class="o">]</span> <span class="no">VAR1</span><span class="o">,</span> <span class="o">[</span><span class="no">TYPE2</span><span class="o">]</span> <span class="no">VAR2</span><span class="o">,</span> <span class="o">...])</span> <span class="o">-></span> <span class="no">TYPE</span>
<span class="c1">// So it can be </span>
<span class="o">([[</span><span class="no">TYPE1</span><span class="o">]</span> <span class="no">VAR1</span><span class="o">,</span> <span class="o">[</span><span class="no">TYPE2</span><span class="o">]</span> <span class="no">VAR2</span><span class="o">,</span> <span class="o">...])</span> <span class="o">-></span> <span class="no">EXPRESSION</span> <span class="o">|</span> <span class="no">STATEMENT</span>
<span class="c1">// or</span>
<span class="o">([[</span><span class="no">TYPE1</span><span class="o">]</span> <span class="no">VAR1</span><span class="o">,</span> <span class="o">[</span><span class="no">TYPE2</span><span class="o">]</span> <span class="no">VAR2</span><span class="o">,</span> <span class="o">...])</span> <span class="o">-></span> <span class="o">{</span>
<span class="no">STATEMENT_1</span><span class="o">;</span>
<span class="no">STATEMENT_2</span><span class="o">;</span>
<span class="o">...</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>A lambda can have no parameter (e.g. <code class="language-plaintext highlighter-rouge">Runnable</code>)</li>
<li>Parameter types of a lambda expression can be inferred
<ul>
<li>In this case, all parameters’ type are inferred and must declared without explicit type name!</li>
</ul>
</li>
<li>Parentheses can be omitted for single & inferred-type-parameter lambda
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre><span class="nc">Button</span> <span class="n">b</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Button</span><span class="o">();</span>
<span class="n">b</span><span class="o">.</span><span class="na">addActionListener</span><span class="o">(</span><span class="n">event</span> <span class="o">-></span> <span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"button clicked"</span><span class="o">));</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>In spite of other languages, function types such as <code class="language-plaintext highlighter-rouge">(String, String) -> int</code> must be assigned to a specific type in Java!
So a lambda expression must be assigned to a variable of <em>functional interface type</em>:
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="nc">Comparator</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">comparator</span> <span class="o">=</span>
<span class="o">(</span><span class="n">str1</span><span class="o">,</span> <span class="n">str2</span><span class="o">)</span> <span class="o">-></span> <span class="nc">Integer</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">str1</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">str2</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
<span class="nc">ToIntBiFunction</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span> <span class="nc">String</span><span class="o">></span> <span class="n">handler</span> <span class="o">=</span>
<span class="o">(</span><span class="n">str1</span><span class="o">,</span> <span class="n">str2</span><span class="o">)</span> <span class="o">-></span> <span class="nc">Integer</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">str1</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">str2</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>A lambda expression has three ingredients:
<ul>
<li>A block of code</li>
<li>Parameters</li>
<li>Values for the <em>free variables</em>: the variables that are not parameters and not defined inside the code.
<blockquote>
<p>Let’s suppose an example for implementation detail: one can translate a lambda expression into an object with a single method,
so that the values of the free variables are copied into instance <em>final</em> variables of that object.</p>
</blockquote>
<p><strong>Note</strong>: The technical term for a block of code together with the values of the free variables is a <strong><em>closure</em></strong>.</p>
<blockquote>
<p>In fact, inner classes have been closures all along. Java 8 gives it an attractive syntax.</p>
</blockquote>
</li>
</ul>
</li>
<li>The body of a lambda expression has <strong>the same scope as a nested block</strong>.</li>
</ul>
<p>Example:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
</pre></td><td class="rouge-code"><pre><span class="kn">import</span> <span class="nn">org.junit.Before</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.math.BigDecimal</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Arrays</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Collections</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Comparator</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.List</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.concurrent.atomic.AtomicInteger</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">static</span> <span class="n">org</span><span class="o">.</span><span class="na">junit</span><span class="o">.</span><span class="na">Assert</span><span class="o">.</span><span class="na">assertArrayEquals</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">static</span> <span class="n">org</span><span class="o">.</span><span class="na">junit</span><span class="o">.</span><span class="na">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">TestFunP</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="no">EXPECTED_SORT</span> <span class="o">=</span>
<span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"A"</span><span class="o">,</span> <span class="s">"car"</span><span class="o">,</span> <span class="s">"Bar"</span><span class="o">,</span> <span class="s">"Bill"</span><span class="o">);</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">BigDecimal</span><span class="o">[]</span> <span class="no">EXPECTED_LEN</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BigDecimal</span><span class="o">[]{</span>
<span class="nc">BigDecimal</span><span class="o">.</span><span class="na">valueOf</span><span class="o">(</span><span class="mi">3</span><span class="o">),</span>
<span class="nc">BigDecimal</span><span class="o">.</span><span class="na">valueOf</span><span class="o">(</span><span class="mi">1</span><span class="o">),</span>
<span class="nc">BigDecimal</span><span class="o">.</span><span class="na">valueOf</span><span class="o">(</span><span class="mi">4</span><span class="o">),</span>
<span class="nc">BigDecimal</span><span class="o">.</span><span class="na">valueOf</span><span class="o">(</span><span class="mi">3</span><span class="o">)</span>
<span class="o">};</span>
<span class="kd">private</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">strings</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="nc">AtomicInteger</span> <span class="n">counter</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">AtomicInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="c1">// ------------------------------</span>
<span class="nd">@Before</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">before</span><span class="o">()</span> <span class="o">{</span>
<span class="n">strings</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="s">"car"</span><span class="o">,</span> <span class="s">"A"</span><span class="o">,</span> <span class="s">"Bill"</span><span class="o">,</span> <span class="s">"Bar"</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testLambdaFull</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">Collections</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="n">strings</span><span class="o">,</span>
<span class="o">(</span><span class="nc">String</span> <span class="n">str1</span><span class="o">,</span> <span class="nc">String</span> <span class="n">str2</span><span class="o">)</span> <span class="o">-></span> <span class="nc">Integer</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">str1</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">str2</span><span class="o">.</span><span class="na">length</span><span class="o">()));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="no">EXPECTED_SORT</span><span class="o">,</span> <span class="n">strings</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testLambdaInferred</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">Collections</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="n">strings</span><span class="o">,</span>
<span class="o">(</span><span class="n">str1</span><span class="o">,</span> <span class="n">str2</span><span class="o">)</span> <span class="o">-></span> <span class="nc">Integer</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">str1</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">str2</span><span class="o">.</span><span class="na">length</span><span class="o">()));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="no">EXPECTED_SORT</span><span class="o">,</span> <span class="n">strings</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testComparatorLambda</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">Collections</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="n">strings</span><span class="o">,</span> <span class="nc">Comparator</span><span class="o">.</span><span class="na">comparingInt</span><span class="o">(</span><span class="n">str</span> <span class="o">-></span> <span class="n">str</span><span class="o">.</span><span class="na">length</span><span class="o">()));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="no">EXPECTED_SORT</span><span class="o">,</span> <span class="n">strings</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testComparatorMethodRefClassInst</span><span class="o">()</span> <span class="o">{</span>
<span class="c1">// ** Class::instanceMethod **</span>
<span class="nc">Collections</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="n">strings</span><span class="o">,</span> <span class="nc">Comparator</span><span class="o">.</span><span class="na">comparingInt</span><span class="o">(</span><span class="nl">String:</span><span class="o">:</span><span class="n">length</span><span class="o">));</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="no">EXPECTED_SORT</span><span class="o">,</span> <span class="n">strings</span><span class="o">);</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testComparatorMethodRefObjectInst</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">Exception</span> <span class="o">{</span>
<span class="nc">Thread</span><span class="o">.</span><span class="na">currentThread</span><span class="o">().</span><span class="na">setName</span><span class="o">(</span><span class="s">"MAIN"</span><span class="o">);</span>
<span class="c1">// object::instanceMethod & lambda no param</span>
<span class="nc">Thread</span> <span class="n">th</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Thread</span><span class="o">(()</span> <span class="o">-></span> <span class="nc">Collections</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="n">strings</span><span class="o">,</span> <span class="k">this</span><span class="o">::</span><span class="n">instComp</span><span class="o">));</span>
<span class="n">th</span><span class="o">.</span><span class="na">start</span><span class="o">();</span>
<span class="n">th</span><span class="o">.</span><span class="na">join</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="no">EXPECTED_SORT</span><span class="o">,</span> <span class="n">strings</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"MAIN"</span><span class="o">,</span> <span class="nc">Thread</span><span class="o">.</span><span class="na">currentThread</span><span class="o">().</span><span class="na">getName</span><span class="o">());</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testComparatorMethodRefEnclosingClassInst</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">Exception</span> <span class="o">{</span>
<span class="nc">Thread</span> <span class="n">th</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Thread</span><span class="o">(</span><span class="k">new</span> <span class="nc">MyRunner</span><span class="o">(</span><span class="n">strings</span><span class="o">));</span>
<span class="n">th</span><span class="o">.</span><span class="na">start</span><span class="o">();</span>
<span class="n">th</span><span class="o">.</span><span class="na">join</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="no">EXPECTED_SORT</span><span class="o">,</span> <span class="n">strings</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="n">counter</span><span class="o">.</span><span class="na">get</span><span class="o">());</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testConstructorReference</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">BigDecimal</span><span class="o">[]</span> <span class="n">allLength</span> <span class="o">=</span> <span class="n">strings</span>
<span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="nl">String:</span><span class="o">:</span><span class="n">length</span><span class="o">)</span> <span class="c1">// Class::instanceMethod</span>
<span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="nl">BigDecimal:</span><span class="o">:</span><span class="k">new</span><span class="o">)</span> <span class="c1">// Class::constructor</span>
<span class="o">.</span><span class="na">toArray</span><span class="o">(</span><span class="nc">BigDecimal</span><span class="o">[]::</span><span class="k">new</span><span class="o">);</span> <span class="c1">// Array::constructor</span>
<span class="n">assertArrayEquals</span><span class="o">(</span><span class="no">EXPECTED_LEN</span><span class="o">,</span> <span class="n">allLength</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// ------------------------------</span>
<span class="kd">private</span> <span class="kt">int</span> <span class="nf">instComp</span><span class="o">(</span><span class="nc">String</span> <span class="n">a</span><span class="o">,</span> <span class="nc">String</span> <span class="n">b</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">Thread</span><span class="o">.</span><span class="na">currentThread</span><span class="o">().</span><span class="na">setName</span><span class="o">(</span><span class="s">"INST_COMP"</span><span class="o">);</span>
<span class="k">return</span> <span class="nc">Integer</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">a</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">b</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
<span class="o">}</span>
<span class="c1">// ------------------------------</span>
<span class="kd">class</span> <span class="nc">MyRunner</span> <span class="kd">implements</span> <span class="nc">Runnable</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">list</span><span class="o">;</span>
<span class="nc">MyRunner</span><span class="o">(</span><span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">list</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">list</span> <span class="o">=</span> <span class="n">list</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">run</span><span class="o">()</span> <span class="o">{</span>
<span class="c1">// EnclosingClass.this::instanceMethod</span>
<span class="n">list</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="nc">TestFunP</span><span class="o">.</span><span class="na">this</span><span class="o">::</span><span class="n">instComp</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"INST_COMP"</span><span class="o">,</span> <span class="nc">Thread</span><span class="o">.</span><span class="na">currentThread</span><span class="o">().</span><span class="na">getName</span><span class="o">());</span>
<span class="n">counter</span><span class="o">.</span><span class="na">incrementAndGet</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong>Notes</strong></p>
<ul>
<li>Since Java 8, we can use <code class="language-plaintext highlighter-rouge">LIST_VAR.sort()</code> instead of <code class="language-plaintext highlighter-rouge">Collections.sort(LIST_VAR)</code>.</li>
<li>For number comparison use <code class="language-plaintext highlighter-rouge"><NUMBER CLASS>.compare()</code> (e.g. Integer.compare())
<blockquote>
<p>Don’t compute x - y for comparison since computation can overflow for large operands of opposite sign!</p>
</blockquote>
</li>
</ul>
<h3 id="method--constructor-reference">Method & Constructor Reference</h3>
<p>Consider lambda as <em><code class="language-plaintext highlighter-rouge">(v1, v2, ...) -> return_type</code></em> in general functional programming idiom.
Now, instead of an in-line code, a method <strong>with the same syntax</strong> (<code class="language-plaintext highlighter-rouge">return_type F(v1, v2, ...)</code>)can be referred conformed to one of the following syntax:</p>
<ul>
<li><em>object</em><strong>::</strong><em>instanceMethod</em></li>
<li><em>Class</em><strong>::</strong><em>staticMethod</em></li>
<li><em>Class</em><strong>::</strong><em>instanceMethod</em>
<blockquote>
<p>the first parameter becomes the target of the method, e.g. <code class="language-plaintext highlighter-rouge">String::compareToIgnoreCase</code> is the same as <code class="language-plaintext highlighter-rouge">(x, y) -> x.compareToIgnoreCase(y)</code>
or in above example <code class="language-plaintext highlighter-rouge">String::length</code> is the same as <code class="language-plaintext highlighter-rouge">x -> x.length()</code></p>
</blockquote>
</li>
<li><strong>this::</strong><em>instanceMethod</em></li>
<li><strong>super::</strong><em>instanceMethod</em></li>
<li><em>EnclosingClass</em><strong>.this::</strong><em>instanceMethod</em></li>
<li><em>EnclosingClass</em><strong>.super::</strong><em>instanceMethod</em></li>
<li><em>Class</em><strong>::new</strong>
<blockquote>
<p>constructor reference</p>
</blockquote>
</li>
<li><em>Class</em><strong>[]::new</strong>
<blockquote>
<p>constructor reference with array type (<code class="language-plaintext highlighter-rouge">int[]::new</code> is equivalent to the lambda expression <code class="language-plaintext highlighter-rouge">x -> new int[x]</code>)
this is useful for libraries with method returning array such as <code class="language-plaintext highlighter-rouge">stream.toArrey()</code>, so you can call <code class="language-plaintext highlighter-rouge">stream.toArray(CLASS[]::new)</code></p>
</blockquote>
</li>
</ul>
<p>The <code class="language-plaintext highlighter-rouge">::</code> operator separates the method name.</p>
<h4 id="more-of-comparator-class">More of <code class="language-plaintext highlighter-rouge">Comparator</code> class</h4>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
</pre></td><td class="rouge-code"><pre><span class="kn">import</span> <span class="nn">org.junit.Test</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Arrays</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.Comparator</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.List</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.stream.Collectors</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">static</span> <span class="n">org</span><span class="o">.</span><span class="na">junit</span><span class="o">.</span><span class="na">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">TestComparator</span> <span class="o">{</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testComparator</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Employee</span><span class="o">></span> <span class="n">list</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"John"</span><span class="o">,</span> <span class="mi">5000</span><span class="o">),</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"Jack"</span><span class="o">,</span> <span class="mi">5000</span><span class="o">),</span>
<span class="k">new</span> <span class="nf">Employee</span><span class="o">(</span><span class="s">"Bill"</span><span class="o">,</span> <span class="mi">3000</span><span class="o">));</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">expected</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span>
<span class="s">"Jack, 5000"</span><span class="o">,</span>
<span class="s">"John, 5000"</span><span class="o">,</span>
<span class="s">"Bill, 3000"</span><span class="o">);</span>
<span class="c1">// try to sort: first by salary descending, and then by name ascending</span>
<span class="n">list</span><span class="o">.</span><span class="na">sort</span><span class="o">(</span><span class="nc">Comparator</span>
<span class="o">.</span><span class="na">comparing</span><span class="o">(</span><span class="nl">Employee:</span><span class="o">:</span><span class="n">getSalary</span><span class="o">,</span> <span class="o">(</span><span class="n">o1</span><span class="o">,</span> <span class="n">o2</span><span class="o">)</span> <span class="o">-></span> <span class="nc">Double</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">o2</span><span class="o">,</span> <span class="n">o1</span><span class="o">))</span>
<span class="o">.</span><span class="na">thenComparing</span><span class="o">(</span><span class="nl">Employee:</span><span class="o">:</span><span class="n">getName</span><span class="o">));</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">actual</span> <span class="o">=</span> <span class="n">list</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="nl">Employee:</span><span class="o">:</span><span class="n">toString</span><span class="o">)</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toList</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">expected</span><span class="o">,</span> <span class="n">actual</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// ------------------------------</span>
<span class="kd">class</span> <span class="nc">Employee</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">String</span> <span class="n">name</span><span class="o">;</span>
<span class="kd">private</span> <span class="nc">Integer</span> <span class="n">salary</span><span class="o">;</span>
<span class="nc">Employee</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">Integer</span> <span class="n">salary</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">name</span> <span class="o">=</span> <span class="n">name</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">salary</span> <span class="o">=</span> <span class="n">salary</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">name</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">Integer</span> <span class="nf">getSalary</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">salary</span><span class="o">;</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">+</span> <span class="s">", "</span> <span class="o">+</span> <span class="n">getSalary</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h4 id="working-with-files">Working with <code class="language-plaintext highlighter-rouge">Files</code></h4>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
</pre></td><td class="rouge-code"><pre><span class="kd">final</span> <span class="nc">Consumer</span><span class="o"><</span><span class="nc">File</span><span class="o">></span> <span class="n">printFileName</span> <span class="o">=</span>
<span class="n">f</span> <span class="o">-></span> <span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">f</span><span class="o">.</span><span class="na">isDirectory</span><span class="o">()</span> <span class="o">?</span>
<span class="n">f</span><span class="o">.</span><span class="na">getName</span><span class="o">()</span> <span class="o">+</span> <span class="s">"/"</span> <span class="o">:</span>
<span class="n">f</span><span class="o">.</span><span class="na">getName</span><span class="o">()</span>
<span class="o">);</span>
<span class="nc">File</span> <span class="n">file</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">File</span><span class="o">(</span><span class="s">"<A_DIR>"</span><span class="o">);</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">File</span><span class="o">></span> <span class="n">files</span> <span class="o">=</span> <span class="nc">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">file</span><span class="o">.</span><span class="na">listFiles</span><span class="o">());</span>
<span class="n">files</span><span class="o">.</span><span class="na">forEach</span><span class="o">(</span><span class="n">printFileName</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"---"</span><span class="o">);</span>
<span class="c1">// sort files, directories first, and then by name in each group</span>
<span class="n">files</span><span class="o">.</span><span class="na">sort</span><span class="o">((</span><span class="n">o1</span><span class="o">,</span> <span class="n">o2</span><span class="o">)</span> <span class="o">-></span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">byDir</span> <span class="o">=</span> <span class="nc">Boolean</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">o2</span><span class="o">.</span><span class="na">isDirectory</span><span class="o">(),</span> <span class="n">o1</span><span class="o">.</span><span class="na">isDirectory</span><span class="o">());</span>
<span class="k">if</span> <span class="o">(</span><span class="n">byDir</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">o1</span><span class="o">.</span><span class="na">getName</span><span class="o">().</span><span class="na">compareToIgnoreCase</span><span class="o">(</span><span class="n">o2</span><span class="o">.</span><span class="na">getName</span><span class="o">());</span>
<span class="o">}</span>
<span class="k">return</span> <span class="n">byDir</span><span class="o">;</span>
<span class="o">});</span>
<span class="n">files</span><span class="o">.</span><span class="na">forEach</span><span class="o">(</span><span class="n">printFileName</span><span class="o">);</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h3 id="interfaces-with-default-and-static-methods">Interfaces with <code class="language-plaintext highlighter-rouge">default</code> and <code class="language-plaintext highlighter-rouge">static</code> methods</h3>
<ul>
<li>From Java 8</li>
<li>An interface can have multiple static methods. Now it is possible to put the factory method in the interface!</li>
<li>Any methods in an interface can have <code class="language-plaintext highlighter-rouge">default</code> implementation, e.g. suppose the default implementation for <code class="language-plaintext highlighter-rouge">isEmpty</code> in <code class="language-plaintext highlighter-rouge">Collection</code> interface:
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre><span class="kd">public</span> <span class="kd">interface</span> <span class="nc">Collection</span> <span class="o">{</span>
<span class="kt">int</span> <span class="nf">size</span><span class="o">();</span>
<span class="k">default</span> <span class="kt">boolean</span> <span class="n">isEmpty</span> <span class="o">{</span> <span class="k">return</span> <span class="n">size</span><span class="o">()</span> <span class="o">==</span> <span class="mi">0</span><span class="o">;</span> <span class="o">}</span>
<span class="o">...</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>In case of default method conflicts, it must be resolved manually:
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="kd">interface</span> <span class="nc">Person</span> <span class="o">{</span> <span class="k">default</span> <span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{...}</span> <span class="o">}</span>
<span class="kd">interface</span> <span class="nc">Named</span> <span class="o">{</span> <span class="k">default</span> <span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{...}</span> <span class="o">}</span>
<span class="kd">class</span> <span class="nc">Employee</span> <span class="kd">implements</span> <span class="nc">Person</span><span class="o">,</span> <span class="nc">Named</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nc">Person</span><span class="o">.</span><span class="na">super</span><span class="o">.</span><span class="na">getName</span><span class="o">();</span> <span class="c1">// call the default method explicitly</span>
<span class="o">}</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
<p>Note: Concrete superclass methods mask default methods</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
</pre></td><td class="rouge-code"><pre><span class="kd">interface</span> <span class="nc">Named</span> <span class="o">{</span> <span class="k">default</span> <span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{...}</span> <span class="o">}</span>
<span class="kd">class</span> <span class="nc">Employee</span> <span class="o">{</span> <span class="kd">public</span> <span class="nc">String</span> <span class="nf">getName</span><span class="o">()</span> <span class="o">{...}</span> <span class="o">}</span>
<span class="kd">class</span> <span class="nc">Manager</span> <span class="kd">extends</span> <span class="nc">Employee</span> <span class="kd">implements</span> <span class="nc">Named</span> <span class="o">{</span> <span class="c1">// </span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">toString</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">getName</span><span class="o">();</span> <span class="c1">// Employee.getName() is called!</span>
<span class="o">}</span>
<span class="o">...</span>
<span class="o">}</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>Mehdi BizhaniAn introduction to Java 8, Functional ProgrammingCurious case of Jekyll and GitHub2018-05-25T00:00:00+00:002018-05-25T00:00:00+00:00http://www.devocative.org/article/tech/jekyll<blockquote>
<p><strong>Notes on Jekyll 4.0</strong></p>
<p>Today, I went through my instructions here and I got some problems due to new Jekyll 4.0.0 version.
After some searches, I found following notes to solve the problem:</p>
<ul>
<li>It seems that the current <code class="language-plaintext highlighter-rouge">github-pages</code> is not compatible with Jekyll 4.0.0. So in the <a href="#installation-ubuntudebian">section</a>,
I mentioned the version of <code class="language-plaintext highlighter-rouge">jekyll</code> and <code class="language-plaintext highlighter-rouge">bundle</code> explicitly.</li>
<li>After calling <code class="language-plaintext highlighter-rouge">bundle install</code>, I got a weired error for <code class="language-plaintext highlighter-rouge">nokogiri</code> gem. Due to <a href="https://github.com/sergiokopplin/indigo/issues/81">link</a>,
I installed <code class="language-plaintext highlighter-rouge">zlib1g-dev</code> package in my Debian box, so the error is resolved.</li>
</ul>
</blockquote>
<h2 id="creating-a-simple-blog-site">Creating a Simple Blog Site</h2>
<p>What you need:</p>
<ul>
<li>A GitHub account
<ul>
<li>A repository with the name of <code class="language-plaintext highlighter-rouge"><USERNAME>.github.io</code></li>
<li>A pushed project of <code class="language-plaintext highlighter-rouge">Jekyll</code></li>
</ul>
</li>
<li>A DNS name for your blog (Optional): <a href="https://help.github.com/articles/using-a-custom-domain-with-github-pages">More Info</a></li>
</ul>
<h2 id="installation-ubuntudebian">Installation (Ubuntu/Debian)</h2>
<p>The main installation guide: <a href="https://jekyllrb.com/docs/installation">Jekyll Site</a></p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
</pre></td><td class="rouge-code"><pre><span class="nb">sudo </span>apt <span class="nb">install </span>ruby ruby-dev build-essential zlib1g-dev
</pre></td></tr></tbody></table></code></pre></div></div>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
</pre></td><td class="rouge-code"><pre><span class="nb">echo</span> <span class="s1">'export GEM_HOME=$HOME/App/Gems'</span> <span class="o">>></span> ~/.bashrc
<span class="nb">echo</span> <span class="s1">'export PATH=$GEM_HOME/bin:$PATH'</span> <span class="o">>></span> ~/.bashrc
<span class="nb">source</span> ~/.bashrc
gem <span class="nb">install </span>jekyll <span class="nt">-v</span> <span class="s1">'3.8.2'</span> <span class="c"># - Installing jekyll via Ruby Gems</span>
gem <span class="nb">install </span>bundler <span class="nt">-v</span> <span class="s1">'1.16.2'</span> <span class="c"># - Installing bundler via Ruby Gems</span>
jekyll <span class="nt">-v</span> <span class="c"># - My Version: 3.8.2</span>
jekyll new myblog <span class="c"># - Create a project based on Minema template</span>
<span class="nb">cd </span>myblog
bundle <span class="nb">exec </span>jekyll serve <span class="c"># - Build & Execute a local web server </span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p>The last command starts a local webserver with address <a href="http://127.0.0.1:4000">http://127.0.0.1:4000</a> to review the blog.
To stop it, just press <code class="language-plaintext highlighter-rouge">ctrl+c</code>.</p>
<p>The contents in folder <code class="language-plaintext highlighter-rouge">myblog</code> are:</p>
<ul>
<li>_posts/</li>
<li>_config.yml</li>
<li>.gitignore</li>
<li>404.html</li>
<li>about.md</li>
<li>Gemfile</li>
<li>Gemfile.lock</li>
<li>index.md</li>
</ul>
<h3 id="on-ruby-update">On Ruby Update</h3>
<p>There are times when you update your system and ruby packages, and your gems won’t work anymore. In that case
execute following commands in root of your project to reinstall your required gems:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
</pre></td><td class="rouge-code"><pre>gem <span class="nb">install </span>bundler <span class="nt">-v</span> <span class="s1">'1.16.2'</span>
bundler <span class="nb">install</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<h2 id="configuration">Configuration</h2>
<h3 id="enable-github-pages">Enable Github Pages</h3>
<ul>
<li>Modify <code class="language-plaintext highlighter-rouge">Gemfile</code> to enable <code class="language-plaintext highlighter-rouge">github-pages</code> (instructions included in the file)</li>
<li>Execute <code class="language-plaintext highlighter-rouge">bundle update</code></li>
<li>If you encounter version incompatibility with Jekyll 4 or about <code class="language-plaintext highlighter-rouge">nokogiri</code>, refer to <a href="#notes-on-jekyll-40">link</a></li>
</ul>
<h3 id="preferred-setting">Preferred Setting</h3>
<p>File <code class="language-plaintext highlighter-rouge">_config.yml</code> is the configuration of the site based on YAML format (mostly <code class="language-plaintext highlighter-rouge">key: value</code> pairs).
Preferred altered config entries are:</p>
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
</pre></td><td class="rouge-code"><pre><span class="na">title</span><span class="pi">:</span> <span class="s"><SITE TITLE></span>
<span class="na">email</span><span class="pi">:</span> <span class="s"><EMAIL></span>
<span class="na">description</span><span class="pi">:</span> <span class="s"><DESCRIPTION></span>
<span class="na">author</span><span class="pi">:</span> <span class="s"><DEFAULT_SITE_AUTHOR></span>
<span class="na">url</span><span class="pi">:</span> <span class="s2">"</span><span class="s"><DNS_URL_NAME</span><span class="nv"> </span><span class="s">or</span><span class="nv"> </span><span class="s"><USERNAME>.github.io>"</span>
<span class="na">permalink</span><span class="pi">:</span> <span class="s">/:categories/:title</span>
<span class="na">show_excerpts</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">github_username</span><span class="pi">:</span> <span class="s"><G_USERNAME></span>
<span class="na">twitter_username</span><span class="pi">:</span> <span class="s"><T_USERNAME></span>
<span class="na">linkedin_username</span><span class="pi">:</span> <span class="s"><L_USERNAME></span>
<span class="na">markdown</span><span class="pi">:</span> <span class="s">kramdown</span>
<span class="na">kramdown</span><span class="pi">:</span>
<span class="na">syntax_highlighter_opts</span><span class="pi">:</span>
<span class="na">block</span><span class="pi">:</span>
<span class="na">line_numbers</span><span class="pi">:</span> <span class="no">true</span>
<span class="na">theme</span><span class="pi">:</span> <span class="s">minima</span>
<span class="na">plugins</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">jekyll-feed</span>
<span class="na">defaults</span><span class="pi">:</span>
<span class="pi">-</span>
<span class="na">scope</span><span class="pi">:</span>
<span class="na">path</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">posts</span>
<span class="na">values</span><span class="pi">:</span>
<span class="na">author</span><span class="pi">:</span> <span class="s"><DEFAULT_BLOG_ENTRIES_AUTHOR></span>
</pre></td></tr></tbody></table></code></pre></div></div>
<p><strong>Note</strong>: Lines from 15 to 18 is set to show line numbers generally in all code blocks.</p>
<h3 id="style-customization">Style Customization</h3>
<p>Since showing line numbers is set in the <code class="language-plaintext highlighter-rouge">_config.yml</code>, the rendered result is not well and needs to be altered.
So the main style of site must be modified.
To handle this issue, the file <code class="language-plaintext highlighter-rouge"><PRJ_DIR>/assets/main.scss</code> must be created with following content:</p>
<div class="language-scss highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
</pre></td><td class="rouge-code"><pre><span class="na">---</span><span class="err">
# </span><span class="na">Only</span><span class="err"> </span><span class="na">the</span><span class="err"> </span><span class="na">main</span><span class="err"> </span><span class="na">Sass</span><span class="err"> </span><span class="na">file</span><span class="err"> </span><span class="na">needs</span><span class="err"> </span><span class="na">front</span><span class="err"> </span><span class="na">matter</span><span class="err"> (</span><span class="na">the</span><span class="err"> </span><span class="na">dashes</span><span class="err"> </span><span class="na">are</span><span class="err"> </span><span class="na">enough</span><span class="err">)
</span><span class="na">---</span><span class="err">
$</span><span class="na">content-width</span><span class="p">:</span> <span class="m">1000px</span> <span class="o">!</span><span class="nb">default</span><span class="p">;</span>
<span class="k">@import</span> <span class="s2">"minima"</span><span class="p">;</span>
<span class="nc">.highlight</span> <span class="p">{</span>
<span class="nt">table</span> <span class="p">{</span>
<span class="nl">border</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nl">margin</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nt">th</span><span class="o">,</span> <span class="nt">td</span> <span class="p">{</span>
<span class="nl">padding</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nl">border</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nt">pre</span><span class="o">,</span> <span class="nt">code</span> <span class="p">{</span>
<span class="nl">border</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nl">margin</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nl">font-size</span><span class="p">:</span> <span class="m">14px</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></td></tr></tbody></table></code></pre></div></div>
<ul>
<li>The <code class="language-plaintext highlighter-rouge">highlight</code> css class here overrides the default definition in minima theme.</li>
<li>Line 5 set the content width to 1000px. The variable is defined in file <code class="language-plaintext highlighter-rouge">minima.scss</code>
in minima’s GEM directory (<code class="language-plaintext highlighter-rouge">$GEM_HOME/gems/minima-2.4.1/_sass/minima.scss</code>)</li>
</ul>
<h3 id="create-a-post">Create a Post</h3>
<ul>
<li>Must create a file in the <code class="language-plaintext highlighter-rouge">_posts</code> directory with name of pattern: <code class="language-plaintext highlighter-rouge"><YEAR>-<MONTH>-<DAY>-<NAME>.md</code></li>
<li>Must starts the file with following content at top (called <a href="#front-matter">Front Matter</a>):
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
</pre></td><td class="rouge-code"><pre><span class="nn">---</span>
<span class="na">layout</span><span class="pi">:</span> <span class="s">post</span>
<span class="na">title</span><span class="pi">:</span> <span class="s2">"</span><span class="s"><TITLE>"</span>
<span class="na">categories</span><span class="pi">:</span> <span class="s"><CAT1> [, <CAT2>, ...]</span>
<span class="na">excerpt</span><span class="pi">:</span> <span class="s"><EXCERPT></span>
<span class="nn">---</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Now continute the content based on <a href="https://www.markdownguide.org">Markdown</a> format</li>
</ul>
<h3 id="create-a-page">Create a Page</h3>
<ul>
<li>A page is a link that is shown at the top header of your blog (menu bar)</li>
<li>By default, an <code class="language-plaintext highlighter-rouge">about.md</code> page is created</li>
<li>To add a page, just create a <code class="language-plaintext highlighter-rouge"><NAME>.md</code> file in the root of your project with following content at top
(called <a href="#front-matter">Front Matter</a>):
<div class="language-yml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre><span class="nn">---</span>
<span class="na">layout</span><span class="pi">:</span> <span class="s">page</span>
<span class="na">title</span><span class="pi">:</span> <span class="s2">"</span><span class="s"><TITLE>"</span>
<span class="na">permalink</span><span class="pi">:</span> <span class="s">/<LINK>/</span>
<span class="nn">---</span>
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
<li>Now continue the content based on <a href="https://www.markdownguide.org">Markdown</a> format</li>
</ul>
<p><strong>Note</strong>: In my case, I created a <code class="language-plaintext highlighter-rouge">pages</code> directory in the root of the project, and I saved all my pages in it, even <code class="language-plaintext highlighter-rouge">about.md</code>.
Jekyll automatically finds all the pages in all normal directory (not started with <code class="language-plaintext highlighter-rouge">_</code>) and generates them in the root of your blog.</p>
<h2 id="basics">Basics</h2>
<ul>
<li>After building the site, <code class="language-plaintext highlighter-rouge">_site/</code> is created.</li>
<li>All regular files are copied to <code class="language-plaintext highlighter-rouge">_site/</code>, unless it starts with <code class="language-plaintext highlighter-rouge">_</code>.</li>
<li>After <code class="language-plaintext highlighter-rouge">Gemfile</code> modification, execute <code class="language-plaintext highlighter-rouge">bundle update</code></li>
<li>For GitHub Pages Gem (set in <code class="language-plaintext highlighter-rouge">Gemfile</code>), execute <code class="language-plaintext highlighter-rouge">apt install zlib1g-dev</code> by root user</li>
<li>The default theme is <code class="language-plaintext highlighter-rouge">minema</code></li>
<li>Execute <code class="language-plaintext highlighter-rouge">bundle show minima</code> in project root directory: it shows the home directory of theme gem <code class="language-plaintext highlighter-rouge">minima</code> (i.e. <code class="language-plaintext highlighter-rouge">$GEM_HOME/gems/minima-2.4.1</code>).
In spite of older version, the following directories are in theme home directory:
<ul>
<li><code class="language-plaintext highlighter-rouge">_includes/</code></li>
<li><code class="language-plaintext highlighter-rouge">_layouts/</code></li>
<li><code class="language-plaintext highlighter-rouge">_sass/</code></li>
</ul>
<p><strong>Note</strong>: you can copy any above directory to the site to change the default generation behavior.</p>
</li>
</ul>
<h3 id="front-matter">Front Matter</h3>
<ul>
<li>The content between two <code class="language-plaintext highlighter-rouge">---</code> at the above of an <code class="language-plaintext highlighter-rouge">.md</code> file.</li>
<li>It is in JSON or YAML format, each row is a <code class="language-plaintext highlighter-rouge">key: value</code> pair. In fact, they are <strong>meta-data</strong> for a post!</li>
</ul>
<h4 id="post">Post</h4>
<ul>
<li>Except the <code class="language-plaintext highlighter-rouge">layout</code>, other keys are optional!
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
6
7
</pre></td><td class="rouge-code"><pre>---
layout: post
title: "<TITLE>"
date: <YEAR>-<MONTH>-<DAY> <HOUR>:<MINUTE>:<SECOND> <TIME_ZONE>
categories: <CAT1> [, <CAT2>, ...]
excerpt: <EXCERPT>
---
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<h4 id="page">Page</h4>
<ul>
<li>Except the <code class="language-plaintext highlighter-rouge">layout</code> and <code class="language-plaintext highlighter-rouge">title</code>, other keys are optional!
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><table class="rouge-table"><tbody><tr><td class="rouge-gutter gl"><pre class="lineno">1
2
3
4
5
</pre></td><td class="rouge-code"><pre>---
layout: page
title: "<TITLE>"
permalink: /<LINK>/
---
</pre></td></tr></tbody></table></code></pre></div> </div>
</li>
</ul>
<h3 id="creating-drafts">Creating Drafts</h3>
<p>Drafts are posts which has not been published yet. So they must be in another directory. Jekyll has a standard for it,
and you can create <code class="language-plaintext highlighter-rouge">_drafts</code> in your repository’s root and put your drafts in it.</p>
<p><strong>Note</strong>: To preview your drafts in your local environment, you can execute Jekyll by <code class="language-plaintext highlighter-rouge">bundle exec jekyll serve --drafts</code>.</p>Mehdi BizhaniAn introduction to Jekyll for GitHub Pages