Thursday, October 24, 2019

Convert tab delimited file into comma delimited on Mac


Need to convert tab delimited file to comma delimited on Mac.


Use sed utility, note that standard version doesn't understand \t, so you can either generate tab symbol by pressing Ctrl+V and Tab or install posix sed.

sed 's/ /,/g' <source file> > <target file>

Tuesday, October 22, 2019

Simple build script that works in Windows and Linux


Have two development environments:

MacOS, Visual Studio Professional for Mac, .NET Core 3.0

Windows 10, Visual Studio Professional for Windows, .NET Core 3.0

Need to write a script that will be working on Windows (.bat file) and MacOS (bash). 

For example sake, scripting languages or msbuild conditions should not be used.


Never thought that it's possible, definitely not an option for complex scenarios, but for basic commands go following (assuming that script build_webpack.cmd).

echo off

NODE=`which node`
cd "$(dirname "$0")/.."
$NODE ./node_modules/webpack/bin/webpack.js --config webpack.config.vendor.js $@

goto $@

SET mypath=%~dp0
CD "%mypath:~0,-1%\\.."
node node_modules\\webpack\\bin\\webpack.js --config webpack.config.vendor.js %*

Now, if we need to invoke this script during project build, add the following line:

    <Exec Command="ClientApp/scripts/build_webpack.cmd" />

Wednesday, June 5, 2019

How to find out what processes were killed by OOM on Linux


Services with high memory utilization are dying sporadically.


Run the following command to get a list of processes killed by oom.

dmesg | egrep -i 'killed process'

Useful articles:

1. How to troubleshoot OOM

2. How to add swap file to Linux Azure VM

vi /etc/waagent.conf
service walinuxagent restart

How to find out how long the process has been running on Linux


Need to find out how long process is running


In order to get time how long process is running, execute following command;

ps -o etime= -p <your pid>

Thursday, May 30, 2019

Azure VM is not accessible /proc/net/route contains no routes


Oversudden Azure VMs running Ubuntu 18LTS became unreachable (WestUS region)
/proc/net/route contains no routes


Connected to Azure VM via Console and got following running lines of error messages

2019/05/30 18:24:20.747552 ERROR ExtHandler /proc/net/route contains no routes

Restart waagent service produced the following:

INFO Daemon Azure Linux Agent Version:
2019/05/30 18:24:18.242022 INFO Daemon OS: ubuntu 18.04
2019/05/30 18:24:18.246632 INFO Daemon Python: 3.6.7
2019/05/30 18:24:18.251476 INFO Daemon Add daemon process pid 2330 to walinuxagent systemd cgroup
2019/05/30 18:24:18.258645 INFO Daemon CGroups: ok
2019/05/30 18:24:18.263173 INFO Daemon Run daemon
2019/05/30 18:24:18.268142 INFO Daemon Clean protocol
2019/05/30 18:24:18.272685 INFO Daemon Provisioning already completed, skipping.
2019/05/30 18:24:18.277567 INFO Daemon RDMA capabilities are not enabled, skipping
2019/05/30 18:24:18.285862 INFO Daemon Determined Agent WALinuxAgent-2.2.40 to be the latest agent
2019/05/30 18:24:18.613217 INFO ExtHandler Agent WALinuxAgent-2.2.40 is running as the goal state agent
2019/05/30 18:24:18.635339 INFO ExtHandler Detect protocol endpoints
2019/05/30 18:24:18.642625 INFO ExtHandler Clean protocol
2019/05/30 18:24:18.651472 INFO ExtHandler WireServer endpoint is not found. Rerun dhcp handler
2019/05/30 18:24:18.658883 INFO ExtHandler Test for route to
2019/05/30 18:24:18.665334 WARNING ExtHandler No route exists to
2019/05/30 18:24:18.670932 INFO ExtHandler Checking for dhcp lease cache
2019/05/30 18:24:18.675843 INFO ExtHandler looking for leases in path [/var/lib/dhcp/dhclient.*.leases]
2019/05/30 18:24:18.684096 INFO ExtHandler cached endpoint not found
2019/05/30 18:24:18.690504 INFO ExtHandler Cache exists [False]
2019/05/30 18:24:18.695994 INFO ExtHandler Send dhcp request
2019/05/30 18:24:18.700569 INFO ExtHandler Examine /proc/net/route for primary interface
2019/05/30 18:24:18.705882 ERROR ExtHandler /proc/net/route contains no routes
2019/05/30 18:24:18.710763 WARNING ExtHandler Could not determine primary interface, please ensure /proc/net/route is correct
2019/05/30 18:24:18.717827 WARNING ExtHandler Contents of /proc/net/route:
Iface   Destination     Gateway         Flags   RefCnt  Use     Metric  Mask            MTU     Window  IRTT

2019/05/30 18:24:18.732775 WARNING ExtHandler Primary interface examination will retry silently
2019/05/30 18:24:20.747552 ERROR ExtHandler /proc/net/route contains no routes
2019/05/30 18:24:22.755975 ERROR ExtHandler /proc/net/route contains no routes

VM reboot didn't help, but VM instance stop/start helped.


Tuesday, May 14, 2019

Tibco Jaspersoft throws error java.lang.NoClassDefFoundError: Could not initialize class java.awt.Color


Tibco Jaspersoft Report server throws java.lang.NoClassDefFoundError: Could not initialize class java.awt.Color when attempting to render report containing graphical elements.


Check missing dependencies using following command:

ldd /opt/jasperreports-server-7.1.1/java/lib/amd64/

Results: (0x00007ffff17f4000) => /lib/x86_64-linux-gnu/ (0x00007f346bc2f000) => /lib/x86_64-linux-gnu/ (0x00007f346b891000) => /opt/jasperreports-server-7.1.1/java/lib/amd64/ (0x00007f346b5bf000) => /usr/lib/x86_64-linux-gnu/ (0x00007f346b3ad000) => /usr/lib/x86_64-linux-gnu/ (0x00007f346b075000) => not found => /lib/x86_64-linux-gnu/ (0x00007f346ae71000) => not found => not found => /opt/jasperreports-server-7.1.1/java/lib/amd64/ (0x00007f346ac45000) => not found => /lib/x86_64-linux-gnu/ (0x00007f346a854000)
        /lib64/ (0x00007f346c0a7000) => not found => /usr/lib/x86_64-linux-gnu/ (0x00007f346a62c000) => not found => /opt/jasperreports-server-7.1.1/java/lib/amd64/ (0x00007f346a41d000) => /usr/lib/x86_64-linux-gnu/ (0x00007f346a219000) => /usr/lib/x86_64-linux-gnu/ (0x00007f346a013000) => not found => /lib/x86_64-linux-gnu/ (0x00007f3469dfe000) => /lib/x86_64-linux-gnu/ (0x00007f3469bf6000)

Install missing packages using following command, can be ignored.

sudo apt-get install libfontconfig1 libxrender1 libxi6 libxtst6

Restart Jasper Report server using commands

sudo /opt/jasperreports-server-7.1.1/ stop
sudo /opt/jasperreports-server-7.1.1/ start

Wednesday, May 1, 2019

Create GitLab backup without repositories


Need to create GitLab backup skipping git repositories themselves. GitLab installed using Omnibus package.


According GitLab Omnibus Documentation and Backing up and restoring GitLab, required backup can be created using command below:

gitlab-rake gitlab:backup:create BACKUP=gitlab_20190420 GZIP_RSYNCABLE=yes SKIP=repositories

Please note, that following files are not included to backup and must be backed up separately.



Add user to sudoers


Need to add existing user to sudoers.



usermod -aG sudo <username>

Install and configure Domain Controller in Microsoft Azure


Need to install and configure domain controller in MS Azure.


There is excellent article how to setup DC in Azure.

Configure Azure Firewall ports required to join Domain Controller


There are two virtual networks, Network 1 (contains Domain Controller, Windows Server 2016) and Network 2 (contains workload VMs, Windows Server 2016). Traffic between them goes through Azure Firewall and User Defined Routes.


After some experiments came up with following categories of rules:

Name Proto Src Dest Ports
tcp-to-dc tcp * <dc server> 53,88,135,139,389,445,464,3268,3269,49152-64000
udp-to-dc udp * <dc server> 53,88,123,135,137,138,464,389,49152-64000


Sync files using rsync with sudo privileges


Need to copy files from one Linux VM to another (Ubuntu 18). Root account is not available, have accounts with sudo privileges.


Sample command listed below:

sudo rsync -e "ssh" --rsync-path="sudo rsync" -Pav remoteuser@<remote server>:/mnt/disk01/folder_to_sync/ /mnt/disk01/folder_to_sync/

When executing, the system will ask you for password to existing user and then password for remoteuser.

Note, that remoteuser should be able to execute sudo command without prompting to type password.

Refer to for instructions.

Run commands with sudo without having to enter a password


Need to temporarily allow access to user to run commands with sudo without typing password (for batch jobs)


Add following entry to /etc/sudoers to allow remoteuser to execute commands with sudo without entering password.



Tuesday, April 30, 2019

Azure icon sets


Need get PNG, SVG icons for Microsoft Azure services, components, etc. Who struggled with presentations, will understand..


Excellent article from Microsoft MVP Chris Pietschmann - Microsoft Azure Icon Set Download – Visio stencil, PowerPoint, PNG, SVG

Icon set can be found at:

ORA-01102: cannot mount database in EXCLUSIVE mode


ORA-01102: cannot mount database in EXCLUSIVE mode during database instance startup

sqlplus / as sysdba
QL*Plus: Release Production on Tue Apr 30 21:26:57 2019

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup;
ORACLE instance started.

Total System Global Area 6174015488 bytes
Fixed Size                  8634320 bytes
Variable Size            1241514032 bytes
Database Buffers         4915724288 bytes
Redo Buffers                8142848 bytes
ORA-01102: cannot mount database in EXCLUSIVE mode


Same as, seems this problem appeared because database was not stopped properly.

Follow great article


Verify that the database was shutdown cleanly by doing the following:

1. Verify that there is not a "sgadef<sid>.dbf" file in the directory

% ls $ORACLE_HOME/dbs/sgadef<sid>.dbf

If this file does exist, remove it.

% rm $ORACLE_HOME/dbs/sgadef<sid>.dbf

2. Verify that there are no background processes owned by "oracle"

% ps -ef | grep ora_ | grep $ORACLE_SID

If background processes exist, remove them by using the Unix
command "kill". For example:

% kill -9 <Process_ID_Number>

3. Verify that no shared memory segments and semaphores that are owned
by "oracle" still exist

% ipcs -b

If there are shared memory segments and semaphores owned by "oracle",
remove the shared memory segments

% ipcrm -m <Shared_Memory_ID_Number>

and remove the semaphores

% ipcrm -s <Semaphore_ID_Number>

NOTE: The example shown above assumes that you only have one
database on this machine. If you have more than one
database, you will need to shutdown all other databases
before proceeding with Step 4.

4. Verify that the "$ORACLE_HOME/dbs/lk<sid>" file does not exist. This is what caused issue in our case. Simple removal of this file did the trick.

5. Startup the instance

Related issues


ORA-01012: not logged on startup failed


Getting ORA-01012: no logged on startup failed during database startup


This behavior in our case caused by "disgraceful" database shutdown.. Seems like zombie Oracle processes existed that prevented database instance from starting up.

sysresv utility allows to diagnose orphaned shared memory segments.

> sysresv

IPC Resources for ORACLE_SID "<your SID>" :
Maximum shared memory segment size (shmmax): 8589934592 bytes
Total system shared memory (shmall): 6871949312 bytes
Total system shared memory count (shmmni): 4096
*********************** Dumping ipcs output ********************
------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages
------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x00000000 0          oracle     600        8634368    891
0x00000000 32769      oracle     600        6157238272 446
0x00000000 65538      oracle     600        8142848    446
0x9ab568e0 98307      oracle     600        28672      446

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0x1d625978 98304      oracle     600        227
0x1d625979 131073     oracle     600        227
0x1d62597a 163842     oracle     600        227

*********************** End of ipcs commanddump **************

***************** Dumping Resource Limits(s/h) *****************
core file size                         0 KB/UNLIMITED
data seg size                     UNLIMITED/UNLIMITED
scheduling priority                    0 KB/0 KB
file size                         UNLIMITED/UNLIMITED
pending signals                       63 KB/63 KB
max locked memory                     14 GB/14 GB
max memory size                   UNLIMITED/UNLIMITED
open files                            64 KB/64 KB
POSIX message queues                 800 KB/800 KB
real-time priority                     0 KB/0 KB
stack size                            32 MB/32 MB
cpu time                          UNLIMITED/UNLIMITED
max user processes                    16 KB/16 KB
virtual memory                    UNLIMITED/UNLIMITED
file locks                        UNLIMITED/UNLIMITED

***************** End of Resource Limits Dup ******************

Maximum map count configured per process:  65530
Total /dev/shm size: 8403361792 bytes, used: 0 bytes
Shared Memory:
ID              KEY
32769           0x00000000
65538           0x00000000
0               0x00000000
98307           0x9ab568e0
ID              KEY
98304           0x1d625978
131073          0x1d625979

Kill memory segments

ipcrm -m <Shared Memory ID>

 Kill semaphores

ipcrm -s <Semaphore>

Try to startup database instance again.

ORA-27125: unable to create shared memory segment during startup


After reboot, unable to startup Oracle 12c database instance (Red Hat Enterprise Server 7.6)

ORA-27125: unable to create shared memory segment
Linux-x86_64 Error: 28: No space left on device
Additional information: 3822
Additional information: 6157238272


Verify OS kernel.shmall memory setting.

1. Get current value 
cat /proc/sys/kernel/shmall

This seems to be too high..

2. Determine page size
getconf PAGE_SIZE

3. Calculate recommended value for shmall

shmall = <total size of SGA>/<page size>

In our case, we have 16GB RAM, so, shmall = 16 * 1024 * 1024 * 1024 / 4096 = 4194304

4. Update /etc/sysctl.conf
vi /etc/sysctl.conf
sudo sysctl -p

5. Verify  kernel.shmall again
cat /proc/sys/kernel/shmall

6. Start Oracle instance
sudo su - oracle
sqlplus / as sysdba

This of course leads to another error..


Formula to set proper values for max processes, sessions and transactions in Oracle


Need to adjust max number of sessions in Oracle DB.


Standard formula looks like:

PROCESSES = Operating System Dependant

alter system set sessions=1000 scope=spfile;
alter system set processes=905 scope=spfile;
alter system set transactions=1100 scope=spfile;

shutdown immediate;

Wednesday, April 24, 2019

How to get external IP for Linux VMs on Azure


How to get external IP for Linux VMs on Azure


dig +short 

Change mount point for /mnt folder in Ubuntu Linux on Azure


When provisioning Linux VMs on Azure, /mnt folder is automatically created and pointed to Resource Disk. In some case, we need to have more fine-grained control over /mnt folder (for instance for backward compatibility) and map Resource Disk to its sub-folder, e.g. /mnt/tmp


Azure provides fantastic support for Linux VMs called Microsoft Azure Linux Agent.

sudo service waagent restart
vi /etc/waagent.conf 
# Mount point for the resource disk

 Restart waagent service

 sudo service waagent restart

Install telnet client from command line on Windows Server


Need to install telnet client on Windows Server 2012, 2016


pkgmgr /iu:”TelnetClient”