BitmapCake!

I have been doing a lot of work with Postgresql recently including replication, clustering, automatic failovers and so on. During one of my tests, I came through this situation where I tried to restart Postgresql server and it shows the server is started, but if I check the status of it, it is always shown as down, and I was unable to connect to it.

After doing everything I could, I felt this could be something related to the start up process, and remembered I disabled the starting of the Postgresql server while setting up the pacemaker cluster. So that was it. I just changed "disabled" to "auto" and it started the service :)

Just run the following.

echo auto > /etc/postgresql/9.6/main/start.conf

and start the server.

service postgresql start

One other thing you could try is, see if you have enough space left on your disks. That could also be a reason for Postgresql to not start, although it wasn't the case with me.

Check the disk space with following command.

df -h

I hope this will be useful for someone :)

If you are using Ubuntu 15 or any higher version, you can use the systemctl command to enable or disable services. But before Ubuntu 15, it is not available.

So here is an alternate way to do the same in earlier versions like Ubuntu 14.04. I recently wanted the same and thought of writing it down here for my own reference as well as a help for those who looking for it.

Enter the following command along with the service name you want.

update-rc.d -f <service> remove
You can use the following command to get a list of services running in your server.

service --status-all
I hope this will be useful to someone :) Please don't forget to leave a comment if you know any better way.

I had this problem with Ubuntu for a long time and I've been looking for a solution that works. After trying out many different things, I finally found something that really worked and I thought I should share it in my blog for my future reference as well as a help for those who looking for it.

If your running a x64 bit Ubuntu or other Linux and find USB transfers hang at the end apply this fix:

echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes
echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes

I suggest you edit your /etc/rc.local file to make this change persistant across reboots.

sudo nano /etc/rc.local

Go to the bottom of the file and leave a space then paste in those two lines.

Save the file with ctrl + x then press y.

To revert the changes enter this in console and remove the lines in /etc/rc.local

echo 0 > /proc/sys/vm/dirty_background_bytes
echo 0 > /proc/sys/vm/dirty_bytes

More info and references: https://unix.stackexchange.com/questions/107703/why-is-my-pc-freezing-while-im-copying-a-file-to-a-pendrive/107722#107722


Source: https://gist.github.com/2E0PGS/f63544f8abe69acc5caaa54f56efe52f

I directly copied it from the above github gist.

Honestly I don't have any idea about the properties which are set above, and I didn't even do any research on them. However it seems working and shows the real copying progess, and doesn't hang at the end of the copying.

You can list the users with the following command:
\du
Edit the /etc/postgresql/9.6/main/postgresql.conf file and do the following changes.

listen_addresses = '*'
wal_level = replica
max_wal_senders = 5
wal_keep_segments = 32
archive_mode = on
archive_command = 'test ! -f /var/lib/postgresql/9.6/main/archive/%f && cp %p /var/lib/postgresql/9.6/main/archive/%f''
Now we need to edit the pg_hba.conf file and add a record for the standy server so that it can connect to the primary.

host replication replication 192.168.3.169/32 md5

Note: 192.168.3.169 is the IP address of standby server.

Now create a new directory inside of the 'main' directory for the archive configuration - run the below command as postgres user:

mkdir -p /var/lib/postgresql/9.6/main/archive/

On your Standby Server
===================

service postgresql stop
We shall keep a backup of the postgres data directory.

mv /var/lib/postgresql/9.6/main /var/lib/postgresql/9.6/main_backup
Now we should migrate the data dir from the master server using the pg_basebackup tool.

pg_basebackup -h 192.168.4.168 -D /var/lib/postgresql/9.6/main -U replication -v -P

Note: 192.168.4.168 is the IP address of the Master server.

After the base backup is complete, we should create a recovery.conf file.

/var/lib/postgresql/9.6/main/recovery.conf

standby_mode = 'on'
primary_conninfo = 'host=192.168.4.168 port=5432 user=replication password=123'
restore_command = 'cp /var/lib/postgresql/9.6/main/archive/%f %p'
trigger_file = '/tmp/postgresql.trigger'
Switch back to the root user.

exit
Start the Postgres service

service postgresql start
Verification
==========

Now we have finished setting up streaming replication between the two servers. To verify that both servers are working as expected, switch to the postgres user and run the following command in the Master Server.

su - postgres
psql -x -c "select * from pg_stat_replication;"
You should see the client_address property pointed to standy server's IP address and the state as streaming.

Note: I took the above restore and archive commands from the Postgresql documentation here: https://www.postgresql.org/docs/9.6/continuous-archiving.html

That's it. I tried my best to keep it as simple as possible. This is only a basic setup to give you an idea on how to setup streaming replication. For a production environment, you might need to set proper values depending on the requirements.

I hope this post will be helpful for someone :) Don't forget to leave a comment if it helped.

I was setting up streaming replication between two Postgresql servers in active passive mode. After the setup was done, I tried to connect to the slave node to see if everything works fine and received the following error:

FATAL:  the database system is starting up

I was thinking for a while before I realized that I didn't set the slave node to hot_standby mode. It seems like a common mistake everyone does, so I thought of writing it down here hoping someone would find it useful.

If you get this error, this is what you have to do:

Open your postgresql.conf file.

vim /etc/postgresql/9.6/main/postgresql.conf
Locate the following

hot_standby = off
and change it as follows:

hot_standby = on

Save the file, then:

service postgresql restart

Now you can query your slave server ;)

Today I was installing Postgres 9.6 in Ubuntu 14 server and while adding the key, I got this error.

gpg: no valid OpenPGP data found error

This is the command I entered:

wget -q -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

I tried several solutions and found a way to get past this error. To solve, all you have to do is download the key separately and add it.

Here we basically the run the above in two seperate commands. This is how to do it.

First get the key.

wget https://www.postgresql.org/media/keys/ACCC4CF8.asc

Then add it:

sudo apt-key add ACCC4CF8.asc

That's it. Hope it will help someone ;)

I have been using Tmux for some time with Ubuntu 16.04 and it was all working so well until I upgraded my system to Ubuntu 18.04. When I tried to run tmux via the command, it interrupted with an error as shown below:

tmux: error while loading shared libraries: libevent-2.0.so

So I checked a little bit deeper and found the solution to this. It seems like tmux in Ubuntu 18.04 requires the shown version of libevent and since I installed tmux under Ubuntu 16.04, I'm missing it. That's pretty obvious.

You should also note that Tmux comes with all the required libraries in Ubuntu repositories if you use Ubuntu 18.04. So in order to fix it, all you have to do is remove the version of tmux you already have and reinstall it.

First let's remove tmux.

sudo apt purge tmux*
This will remove all tmux related packages from your system. Now if you enter the command tmux in your terminal, you should get a message saying it's not installed. If it still shows the same error, then you have to manually remove tmux.

To see which tmux binary is used in your system, run the following command.

which tmux

It will display the path to tmux. You can remove it manually by running:

sudo rm $(which tmux)

Fine, now tmux is removed from your system. Now let's install it using apt.

sudo apt install tmux

After installing, you can start tmux by entering the tmux command in your terminal. If you get some error saying tmux is not found, then its probably because your system is still using the old path to tmux, which we deleted before.

To verify that, enter the following command.

type tmux

If you get a response something to similar to this :

tmux is hashed (/usr/local/bin/tmux)

Then it means you have to clear the old hash, so it will identify the new tmux path.

You can remove the hash by running the following command.

hash -d tmux

This will remove the hash entry and now if you again try to run tmux, it should work as expected.

I hope this will help somebody :) Feel free to comment and ask any questions.

Vim by default adds two tabs when you indent something. It also messes up the whole thing if you paste a large code block with tabs into it.

Here's how you can set the tabs to 4 space width, and fix that messing up issue.

Edit your vim config file and add this at the bottom of it.

vim ~/.vimrc

and add the following.

set tabstop=4

Now save and quit. Start vim to see that it now indents your code only by a single tab :)

Today, a friend asked me to login to a Linux server and he sent me a PPK file which he uses with his Putty client in Windows. I had to generate a Public / Private key pair from that PPK file in order to use it with my SSH client in Ubuntu. It was really easy and took me only a few minutes. I thought I should post this in my blog so it would be helpful to somebody, and it would also be useful to me as a quick reference in future whenever I need it again.

This is how to convert your PPK file to an OpenSSH keypair.

1. First, if you don't have putty tools installed in your computer, you have to install it. This will help us when dealing with PPK files.
   
   sudo apt install putty-tools
   
2. Now we can use the puttygen utility to generate an OpenSSH private key from this ppk key file. To do that, run the following command.
   
   puttygen <your-key-file>.ppk -O private-openssh -o <new-key-file>
Ex: puttygen key.ppk -O private-openssh -o id_rsa
   Here the -O option stands for "output-type" and it specifies the type of file we want to generate. By giving it as "private-openssh", we tell it to generate an OpenSSH private key file. You can see the manual page by running man puttygen if you want to learn more about the other output types. (But they aren't required now).
   


3. Okay, now we have the private key. We can use ssh-keygen to generate the relavent public key for this private key file.
   
   ssh-keygen -y -f <new-key-file> > <new-key-file>.pub

Ex: ssh-keygen -y -f id_rsa > id_rsa.pub
   
   Here the -y and -f options mean the following: (taken from the Manual page)
   

   
   -y = This option will read a private OpenSSH format file and print an OpenSSH public key to stdout.
   -f filename = Specifies the filename of the key file.

   
   
   I hope it's clear. We are almost done.
   


4. Last step is to set the right permissions for the new key pair. Usually, the SSH client expects the Key files to have permission as 400 or 600. So let's set it.
   
   chmod 400 id_rsa
chmod 400 id_rsa.pub
   Now we should be able to login to our server using the new key pair we generated.
   
   ssh -i .ssh/do_server/id_rsa root@xxx.xxx.xxx.xxx

I hope this will be useful to somebody in future :) If this post helped you, don't forget to leave a comment and let me know !

I worked on something interesting today. That was "creating a simple email client for Gmail", using the PHP's built-in IMAP library. I know there is an official PHP library provided by Google for that, and I have even wrote a blog post about it sometime back (explaining about a bug I found in it). But this time I wanted to do it with the Imap library instead.

So soon after finishing the Inbox, I started working on listing the "Sent Items". But to my suprise, I had a bit hard time finding the right mailbox name for the sent items. I assumed a few names like "Sent", "Sent Items", "Sent Messages", but none of them worked. So I turned to Google and did some search, and then I found this page on the Google Developers website.

https://developers.google.com/gmail/imap/

It explains everything about Gmail and the IMAP protocol, and how to actually use IMAP with Gmail. In the IMAP Extensions page, they have listed all the mailbox names and it was just a matter of using the right name.

Here's the list:


a004 LIST "" "*"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\Noselect \HasChildren) "/" "[Gmail]"
* LIST (\HasNoChildren \All) "/" "[Gmail]/All Mail"
* LIST (\HasNoChildren \Drafts) "/" "[Gmail]/Drafts"
* LIST (\HasNoChildren \Important) "/" "[Gmail]/Important"
* LIST (\HasNoChildren \Sent) "/" "[Gmail]/Sent Mail"
* LIST (\HasNoChildren \Junk) "/" "[Gmail]/Spam"
* LIST (\HasNoChildren \Flagged) "/" "[Gmail]/Starred"
* LIST (\HasNoChildren \Trash) "/" "[Gmail]/Trash"


https://developers.google.com/gmail/imap/imap-extensions

The mailbox name should be the name you see inside the square brackets. ex: [Gmail]/Sent Mail

Here's how to use it:

$hostname = '{imap.gmail.com:993/imap/ssl}[Gmail]/Sent Mail';

I thought I should post this in my blog, hoping that it would save someone else's time :) Please don't forget to leave a comment if it helped you. :)

Today I installed MariaDB on my laptop and faced this weird issue. After the installation, I tried to login to the DB shell using the user 'root' and the blank password. Then it returned me this error message stating:

ERROR 1698 (28000): Access denied for user 'root'@'localhost'
My first guess was that they might have changed the default password and set it to something else. So I did some research and realized it was not the case here. The actual problem was related to the authentication process of MariaDB.

MariaDB uses a plugin called UNIX_SOCKET to authenticate users, which allows them to use the operting system credentials to login. If I wanted to login to MariaDB using the root user, it first requires me to logged into my system as root (which I can temporarily take access using the sudo command.)

But since I'm logged in as the user nimeshka, I cannot request MariaDB to login as another user (in this case, root). So if I enter the command,

mysql -uroot -p
it would tell me that I have no permissions to login. Why? because I'm authenticated to the Ubuntu system as nimeshka, so MariaDB won't trust me when I ask it to login as root :P So it is clear that if I enter the following command, it would allow me to log in.

sudo mysql
You should also note that If I had created a user in mysql with the username nimeshka, I could have logged in to the shell by just entering the mysql command (it will take my current system username and will log me in as the particular mysql user)

Another thing to note: As I was concerned why I didn't face this issue when I installed MariaDB earlier, I looked for a bit more details in their documentation and found a satisfying answer. It says that they use the UNIX_SOCKET plugin by default in new installations of Ubuntu 15 and later (I used to install MariaDB in Ubuntu 14 earlier and today I tried it in Ubuntu 16, which makes sense!).

So here is how to disable the UNIX_SOCKET plugin for root user.

# First we will login with sudo (means root)
$ sudo mysql -u root

# Then switch to the mysql DB.
MariaDB [(none)]> use mysql

# Then we will update the authentication plugin for root user by running the following query.

MariaDB [(none)]> update user set plugin='' where User='root';

# We need to flush privilages and exit the session.
MariaDB [(none)]> flush privileges;
MariaDB [(none)]> quit


That's it guys, hope this will be useful for someone :)

By default Ubuntu shows file results and recently accessed files in the dash. Although it's useful, there are times it becomes annoying to the point that you wish if you have an option to disable / hide these files from the dash.

Luckily, it's something very easy to achieve. Here's how to do it.

Just press your super key (Windows key) and type privacy into the dash search box. You will see a result similar to the image shown below. (Icons could be different depending on your theme. However, you need to select the "Security and Privacy" app.



Then once you open it, click on the "Files & Applications" tab (second tab). Here you can disable the "Record file and application usage" option to ask Ubuntu to not remember your actions. If you still want it, and just want to ignore specific file types, you just have to uncheck the file types you want from the list shown below in this window. 




You can also clear all your recorded usage data so your dash will not have any past data :)

That's it guys. I hope it's clear and helpful. Please don't forget to leave a comment if this was helpful to you!!

Good news folks, Gmail launched it's new user interface with a cool material design a few days ago and I promptly switched to it. Like every other web developer who questioned "why Google doesn't introduce the material design UI to Gmail?", I too was concerned on the same, and I knew it would be there someday soon. Now it's released and I'm happy with the new version so far. It looks good, user friendly and not so heavy on the browser (surprisingly).

However, reminding us all that nothing is perfect, I found that a few elements of the new UI are not up to my tastes (Seems a bit difficult to read, specially the bold subjects and the font size). So I changed it to make it look how I like. If you too want to make some changes to it, this is how I did it. It's really easy, but I did this in Firefox <3 only.

So what we are going to do here is apply a custom style sheet on top of the Gmail interface. You can do it easily in Firefox. Remember, we apply the changes to the browser, not to the web page itself. So we need to change the browser's style sheets, which will be applied to the websites we ask on it.

First select Help -> Troubleshooting information from Firefox tool bar. It will open you a page with some configuration values table. In the table, under the Profile Directory row, you will see a button named "Open Directory".





Click on it and you will be taken to your Firefox profile directory. Now create a new directory and name it as "chrome" (If this directory already exists, you can skip this step.)

Additional Info: For those who wonder, "chrome? in Firefox? why?". Here's the answer: In GUI terms, a window of an interface is usually known as a chrome. So this is not making any references to Google Chrome ;)

Now, inside the new directory you created, make a new file and call it as "userContent.css". You can add the following styles to it to make any changes you wish to see in Gmail interface.


@-moz-document domain('mail.google.com') {
    .yW {
        font-size: 15px !important;
    }
   
    .zA > .a4W {
            font-size: 15px !important;
    }

    .zF, .bqe, .bq3 {
            font-weight: 500 !important;
    }
}

What's important here is that, you have to write all your styles for Gmail within the outer css rule. That is: @-moz-document domain('mail.google.com') {}.

Now if you restart Firefox and access Gmail, you will see your fonts are a little bit expanded and very readable. I also changed the font-weight to 500 to reduce the boldness.



If you are used to the old Gmail interface and it's fonts, I recommend to change the fonts to "Sans Serif", but if you do so, remember to change the font-weight ;)

As you might have already guessed, you can use this same method to apply custom styles to any website you want :) Just change "mail.google.com" to  whatever the domain name you wish.

That's it guys! I hope this post is clear. Please feel free to ask anything on comments and don't be hesitant to point out if I have done anything in a wrong way :)

Today I'm going to share something interesting which will help you to download a large number of files at once. For one of my projects, I had to find images from Google, and upload them one by one to a particular website and create posts. At first, this was not much difficult as the number of images I needed was very little. But it kept growing and growing and I had to manually do the same thing repeatedly. Since I'm a lazy person by nature, I wanted to automate the process and here is a part of it, which I did to download the images quickly.

Unfortunately I couldn't automate the Googling part because it needed some human eyes to find quality images (I didn't want to go to the level of automation using computer vision ;) ). So however I searched for images, then whenever I found an interesting image, I right clicked on it and opened the image in a new tab (using ctrl + click). I kept doing this until I had enough images.

Here's the interesting part:

I installed this awesome Firefox extension in my browser, which allowed me to copy all of the URLS which were currently open in my brower. That was very convenient given that I had to copy each URL one by one when I began this task.

So now I have a list of URLS of the images I wanted to download. Now what? I pasted the copied list to a text document and opened another text document to write my shell (bash) script. So here it is.

#!/bin/bash

file="urls.txt"

while IFS= read line
do
    echo "$line"
    wget "$line"   
done <"$file"

Explanation: First I assign the name of the file I want to read into the file variable. Then I simply read the file line by line and run a wget command with the url to download the file.

To run this script, all I had to do was fire up a terminal, grant the execute permissions to the file and run it using the following command.

$ ./download.sh

download.sh is the name of your shell script.

That's it folks. I hope this will help someone :) Please leave a comment if you think this helped you or if you think I miss anything.

How to exit a telnet session?

It's funny, but I was fumbling with the terminal to exit a telnet session today. I rarely have to use telnet so I'm not much experienced with it. But if someone ever want to exit from a telnet session this is what you have to do.

Basically when you start a telnet session, this is what you would see most of the times:

~$ telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Now what? Pressing Ctrl + c, z, x, d will not do anything. I was too lazy that I didn't pay much attention to what was printed in the console. If I had read that, I would have easily exit the session.

It says:  Escape character is '^]'.

So that's it. You need to press Ctrl + ] keys and it will take you back to the telnet prompt where you can either press Ctrl + D or enter the command "quit" and press enter.

~$ telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

One thing to note is that I use Terminator as my terminal emulator and this method doesn't work in it. I Still can't exit from that (: But this definitely works in the gnome terminal.

I hope this will help someone :) Please don't forget to leave a comment if you ever come across this post. Feel free to correct me or add any methods you know other than this.

A few days ago, I was looking for a Redis client for Ubuntu and found that there is one called Redis Desktop Manager. Although it's free and open source, it only provides builds for Windows. Ubuntu and Mac OS users have to subscribe by making a monthly payment or they can build it by themselves using the source (: So I opted for the second option and built and installed it in my Ubuntu box since it was for a small task.

Then today, when I tried to run the RDM again, I got this error and I was wondering what might have changed in the system. In fact there was nothing changed so I had to dig a little bit more into the error and found why I got this error. So here's how I fixed it.

If you haven't built the Redis Desktop Manager (RDM) by source before, this is how to do it. (It's all well documented)

~$ git clone --recursive https://github.com/uglide/RedisDesktopManager.git -b 0.9 rdm && cd ./rdm

~$ cd src/
~$ ./configure
~$ source /opt/qt59/bin/qt59-env.sh && qmake && make && sudo make install
~$ cd /usr/share/redis-desktop-manager/bin
~$ sudo mv qt.conf qt.backup
Now it's done and you can simply run RDM by entering the command:

~$ cd /usr/share/redis-desktop-manager/bin/
~$ ./rdm
or simply by using:

~$ sh /usr/share/redis-desktop-manager/bin/rdm.sh
I believe that you are reading this blog post means you got either of the following errors when you run the above command:

/usr/share/redis-desktop-manager/bin/rdm.sh: 2: /usr/share/redis-desktop-manager/bin/rdm.sh: source: not found
/usr/share/redis-desktop-manager/bin/rdm: error while loading shared libraries: libQt5Charts.so.5: cannot open shared object file: No such file or directory
or just:

./rdm: error while loading shared libraries: libQt5Charts.so.5: cannot open shared object file: No such file or directory
It says it's unable to load the libQt5Charts.so library. So if you are sure that you have installed (I bet you have) and still get this error, you need to do a few things before running RDM.

First you need to tell it where you have the QT libraries. When you build from the source, the QT libraries are installed at /opt/qt59 directory.

So just enter the command

~$ source /opt/qt59/bin/qt59-env.sh
Now if try to run RDM again, it should work fine :) Since this seems reduntant for us to enter this command everytime we want to run the Redis Desktop Manager, you can edit the rdm.sh file and add the above line on top of it.

Note: If you paid enough attention to the build process, you may have noticed that we ran the above command during building it from the source. That's the reason why it worked a few days ago when I installed it.

So let's update /usr/share/redis-desktop-manager/bin/rdm.sh file.

~$ vim /usr/share/redis-desktop-manager/bin/rdm.sh
#!/bin/bash
source /opt/qt59/bin/qt59-env.sh
DIR=$(dirname "$(readlink -f "$0")")
export LD_LIBRARY_PATH="$DIR/../lib":$LD_LIBRARY_PATH
$DIR/rdm
Notice that we added the line: source /opt/qt59/bin/qt59-env.sh after the shebang. Now save it by pressing ESC then :wq in vim and RDM should load fine :)

I also found that having to enter long paths to run RDM is overkill. So I just added a softlink to my /usr/bin directory using the below command.

ln -s /usr/share/redis-desktop-manager/bin/rdm.sh /usr/bin/redis-desktop
Now I can run Redis Desktop Manager using the redis-desktop command.

That's it folks. I hope this will be helpful to someone. Please don't forget to let me know if I have missed anything, and leave a comment if you know a better way to do this.

I was using PyCharm today and I suddenly got this weird error when I started the debugger. The debugger was working fine earlier so I was a bit clueluess as to why it happened. The error I recieved was something similar to what I have shown below.

Connected to pydev debugger (build ....)
Could not connect to 127.0.0.1: 60695
Traceback (most recent call last):
File "/Applications/PyCharm 2.app/Contents/helpers/pydev/pydevd.py", line 1572, in <module>
debugger.connect(host, port)
File "/Applications/PyCharm 2.app/Contents/helpers/pydev/pydevd.py", line 319, in connect
self.initialize_network(s)
File "/Applications/PyCharm 2.app/Contents/helpers/pydev/pydevd.py", line 311, in initialize_network
time.sleep(0.1) # give threads time to start
KeyboardInterrupt
This is not the exact error, but it was something similar. Anyway, if you get this error and you aren't able to debug your lovely app, here is a simple fix for that.

Just remove all your hit points in the debugger and restart the debug process. It will work fine :)

That's it!! I hope this will help someone. If it helped you, don't forget to leave a comment. If you also know any other fix, let everyone know that too by leaving a comment.