I will consider as a prerequisite having the QGIS and Pycharm installed and knowing how to use them. Since we want to use Pycharm's remote debugging function we need to install the Professional version of Pycharm (as described by https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html).

We install the pydevd-pycharm library by opening the Pycharm QGIS plug-in project. Select Edit Configurations... from Run/Debug Configurations to open the edit page as shown below:

Use the + sign to add a new configuration (or Alt + Insert) and select Python Debug Server. In our case, we created a new Python Debug Server configuration called qgisDebugger in our case.

In order to have the debugger running, we need to install the pydevd-pycharm library as described by the message displayed on the right side of the window. For doing this you either use directly execute while in Python:
pip install pydevd-pycharm~=211.7628.24

Or directly in Pycharm by using the interpreter settings as shown in the following picture:

After the pydevd-pycharm library was installed, we need to set-up:

• IDE host name as 127.0.0.1,
• Port as 53100,
• and have “Redirect output to console” and “Suspend after connect” selected.

Once we have the new Python Debug Server configuration saved, we need to perform one more action before being able to work with the debugger. Import the pydevd-pycharm package in the python module that needs to be debugged and copy the following statement:
pydevd_pycharm.settrace('127.0.0.1', port=53100, stdoutToServer=True, stderrToServer=True)

In the place where you want to stop first. The statement is equivalent to a breakpoint thus when the debugger is executed it will first stop at its position. Save the modifications and click on the debug button after yous elected the new configuration as shown in the below figure:

Once you click on the debug button in Pycharm, the server will wait for a connection (also shown in the terminal). Open the QGIS and (re)load the plug-in. First the operation will stop where we introduced the pydevdpycharm.settrace() command and after we continut at any breakpoint introduced by use in Pycharm.

If you want to run the plug-in code without the debugger, you need to comment out the pydevdpycharm.settrace(). Otherwise, QGIS will crash loading the plug-in without having the Debug Server started.

The debugger is an extension of PostgreSQL installation and it needs to be installed and configured in order to be able to use it.

Once installed, it can be very useful to debug PL/pgSQL functions in PostgreSQL, EDB-SPL functions, or stored procedures.

I thereby explain the necessary steps to install and enable the debugger. In my case I have already installed PostgreSQL version 13.3 and pgAdmin version 5.4 so I will not cover how to install these two.

First install the PostgreSql pldebugger:
sudo apt-get install postgresql-13-pldebugger

In case that the package is not available it can be also compiled:
sudo apt-get install build-essential
sudo apt-get install postgresql-server-dev-13
cd /usr/local/src
git clone git://git.postgresql.org/git/pldebugger.git
cd pldebugger
export USE_PGXS=1
make
make install

Once installed, you need to edit the postgresql.conf by:
sudo nano /etc/postgresql/13/main/postgresql.conf

Use Ctrl+W to search for shared_preload_libraries, replace it with:
shared_preload_libraries = 'plugin_debugger'

Save the changes (Ctrl+X and press Y followed by Enter). Once this part is done you need to restart PostgreSQL service by:
sudo service postgresql stop
sudo service postgresql start

In order to check if everything is up and running, you need to check the PostgreSQL log:
sudo tail -f /var/log/postgresql/postgresql-13-main.log

The last step is to enable the debugger in a PostgreSQL database that you want to debug by installing the extension with:
CREATE EXTENSION pldbgapi;

You can do this either from the PostgreSQLcommand line or from pgAdmin directly.

Now let us assume that we want to debug the get_film_count (len_from integer, len_to integer) function that returns the number of films that have the length between len_from and len_to.

In pgAdmin navigate to the same database and right click the PL/pgSQL function get_film_count(integer,integer). We have now two new tools, Debug and Set BreakPont, as shown in the following figure:

In the context menu choose Debugging > Debug and a Debugger window will open to prompt for any parameters (as seen in the below figure).

Once some values are given (len_from and len_to in our case), it will then break on the first line of the function after the Debug button is clicked.

In the other context menu, if you choose Debugging -> Set breakpoint the following window opens:

Waiting for the next function call. When a function call happens, the debugger waits for you to add breakpoints, or step through it line-by-line.

I found the debugger “a must have” for developing and debugging PL/pgSQL functions in PostgreSQL. More information about how to use the pgAdmin debugger can be found in the official documentation

There are lots of ways of speeding your QGIS build times, some of them described in the INSTALL file, other on blog posts like Nathan's or on the QGIS Developers Mailing list.

But everyone wants more speed, right? So, I'm going to present one solution that I haven't encountered until two days ago so I thought of letting everybody know. Maybe all of you do it and I'm the last one to the party :).

I was planning to switch to the new quad core mobile processors (either Intel or AMD) and digging around I settled on a Thinkpad T480s. As I like to do my homework, the finest article I could find is from a laptop review site that I can wholeheartedly recommend: notebookcheck.net

From the article in the link above, I found out that I can do some fine tuning and utilize the full performance potential of my CPU by keeping the core at much higher frequencies for prolongued periods of time. You never get this out of the box because all the chips are shipped from the factory with a safety margin in their default operating voltage. This is all because there is a level of variation in every chip although their naming is the same (your i5-8250U is different then mine).

Knowing this, you can fine tune your chip and do undervolting with some specific tools. As a result, you can either:

• get some performance benefits without sacrificing temperature or power consumption
• get even more performance at the expense of some increase in temperature and in power consumption
• decrease your battery consumption and the chip temperature.

Your choice!

If your laptop cooling solution is good (the T480s has a nifty cooling system, as it was setup to support also a dedicated NVIDIA card) you can go with the "far more performance" mode as the increase in temperature is noticeable only when you look at the temperature metrics.
Does it bother you if you consume 10 W more while you laptop is plugged in? I am surely not bothered!

Out of the box, my QGIS building time under Linux was 39 minutes 12 s, with each core running stable at 2400MHz after few minutes.

Now I get 29 minutes 15 seconds, each core stabilizing at around 3200MHz, so I shaved around 10 minutes of my building time, which is quite a lot!

Look at my poor MacBook Pro from 2016.. running Linux of course.
Now I'm more than 50 Minutes faster. Can't believe it!

If you want to compare your build time, I used the following parameters under Debian Buster:

cmake -G Ninja -D CMAKE_BUILD_TYPE=Debug -D CMAKE_INSTALL_PREFIX=/home/web/qgis-master -D WITH_ASTYLE=ON -D WITH_APIDOC=ON -D WITH_CUSTOM_WIDGETS=ON -D WITH_QWTPOLAR=ON -D WITH_INTERNAL_QWTPOLAR=ON -D WITH_SERVER=ON -D SERVER_SKIP_ECW=ON -D ENABLE_TESTS=ON -D ENABLE_PGTEST=ON -D WITH_3D=False ..

My ThinkPad T480s with its 15W TDP i5-8250U even surpasses some units with the 45 W chip Core i7-7700HQ. Now that's really cool, ain't it?

To quote from the article above:

Undervolting: We managed to reduce the core voltage of our processor by 125 mV, which stabilizes the performance and reduce both the temperature as well as the consumption at the same time.

If you want another nice article of undervolting the T480s and the X1 Carbon 6th generation you can check this one.

How did I specifically did it?

I installed Windows on a stick, and with XTU I determined the stable undervolting parameters for my CPU:

• 'CPU' -90 mV (fate has it that my CPU was not as good as the one the guys from notebookcheck got as I didn't manage to go to 125mV like they did)
• 'CPU Cache' -90 mV
• 'GPU' -90 mV

I immediately noticed benchmark improvements under Windows which I won't post here as my target was and is to have the improvements under Linux.

For my Linux machine (Debian) things are not as easy to do as on Windows. Those bad boys from Intel didn't do a XTU for Linux.

I found out of this reverse engineered project and I'm using it despite the Disclaimer:

This tool may damage your hardware since it uses reverse engineered methods of MSR usage. Use it on your own risk.

My /etc/intel-undervolt.conf config looks like this:

# CPU Undervolting
# Usage: apply ${index} ${display_name} ${undervolt_value}
# Example: apply 2 'CPU Cache' -25.84

apply 0 'CPU' -90
apply 1 'GPU' -90
apply 2 'CPU Cache' -90
apply 3 'System Agent' 0
apply 4 'Analog I/O' 0

# TDP Alteration
# Usage: tdp ${short_term} ${long_term}
# Example: tdp 45 35
tdp 45 45
# Critical Temperature Offset Alteration
# Usage: tjoffset ${temperature_offset}
# Example: tjoffset -20
tjoffset -15

Notice that I made the temperature offset to -15. This means that on my specific chip with 100 degrees C limit I enforced the upper temperature limit at 85 degrees as I like to keep my notebook surface cool enough so it doesn't bother me.

There is a trick (learned after quite some time of trial and error) in that it won't work until you set your CPU scaling Governor in performance mode.

You can use cpupower to do it after installing it with apt-get install linux-cpupower.

Check first what's your current situation with cpupower frequency-info which should output something like:

analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 400 MHz - 3.40 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 400 MHz and 3.40 GHz.
          The governor "powersave" may decide which speed to use within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 1.52 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes

Notice the The governor "powersave" may decide which speed to use within this range. part which tells you that you're in powersave mode.

By doing cpupower frequency-set -g performance you can put it in performace mode and your CPU will fly over the performance realms.

I've used the sensors utility by doing watch -n 1 sensors plus the watch -n 1 grep \"cpu MHz\" /proc/cpuinfo command to check the improvements and to see the CPU temperature.

The commands output:

Every 1.0s: sensors          thinkpad: Fri May 25 12:23:18 2018

iwlwifi-virtual-0
Adapter: Virtual device
temp1:            N/A  

pch_skylake-virtual-0
Adapter: Virtual device
temp1:        +66.0°C  

acpitz-virtual-0
Adapter: Virtual device
temp1:        +84.0°C  (crit = +128.0°C)

thinkpad-isa-0000
Adapter: ISA adapter
fan1:        4291 RPM

coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +85.0°C  (high = +100.0°C, crit = +100.0°C)
Core 0:        +85.0°C  (high = +100.0°C, crit = +100.0°C)
Core 1:        +84.0°C  (high = +100.0°C, crit = +100.0°C)
Core 2:        +81.0°C  (high = +100.0°C, crit = +100.0°C)
Core 3:        +83.0°C  (high = +100.0°C, crit = +100.0°C)

Every 1.0s: grep "cpu ...  thinkpad: Fri May 25 12:22:57 2018

cpu MHz         : 3217.363
cpu MHz         : 3217.202
cpu MHz         : 3217.075
cpu MHz         : 3216.951
cpu MHz         : 3216.628
cpu MHz         : 3216.522
cpu MHz         : 3216.501
cpu MHz         : 3216.220

As I usually keep my notebook plugged in I created a systemd service to set it in performance mode when it boots with:

cat << EOF | tee /etc/systemd/system/cpupower.service
[Unit]
Description=CPU powersave

[Service]
Type=oneshot
ExecStart=/usr/bin/cpupower -c all frequency-set -g performance

[Install]
WantedBy=multi-user.target
EOF

systemctl enable cpupower.service

When I'm not plugged in, I simply do a cpupower frequency-set -g powersave to prolong the battery life.

Or.. better yet, you could install acpi with apt-get install acpi -y, add this to a bash script /root/switch_governor.sh with the following content:

plugged_in=$(acpi -a | cut -d' ' -f3 | cut -d- -f1)

if [ "$plugged_in" = "on" ]; then
    cpupower frequency-set -g performance
else
    cpupower frequency-set -g powersave
fi

Make it executable with chmod +x /root/switch_governor.sh and add to your root crontab (crontab -e -u root):

*/1 * * * * /root/switch_governor.sh

This checks each minute if your laptop is plugged in and sets the governor appropiately.

Here's how to build QGIS with Oracle support on Debian based systems (this means also Ubuntu).

I'm using Debian Testing which at the time of writing this article has the codename Buster.

First, you need to install the QGIS build dependencies. One way of doing that is to get the dependencies that are written in the https://github.com/qgis/QGIS/blob/master/INSTALL.

Sometimes, there's an updating lag for the dependencies written in the INSTALL document so I tend to use the apt-get build-dep command.

In order to do that, you need to configure the QGIS software repositories on your linux box if you haven't done it already. See https://qgis.org/en/site/forusers/alldownloads.html#debian-ubuntu

Depending on what version of QGIS you want to compile, add your corresponding repositories.

For QGIS 3 (master) on my box (Buster is not available yet in the QGIS repos, but Stretch works).
So I added to the /etc/apt/sources.list the following lines:

deb http://qgis.org/debian-nightly stretch main
deb-src http://qgis.org/debian-nightly stretch main

Don't forget to add the qgis.org repository public key to your apt keyring as specified in the above qgis.org link.

After you add the repositories do an sudo apt-get update to get the latest lists of packages.

Now you can install the dependencies by doing sudo apt-get build-dep qgis. For the oracle build you also need to do a sudo apt-get install qtbase5-private-dev qt5-default

Now let's install the Oracle dependencies:

mkdir -p ~/dev/oracle && cd ~/dev/oracle

In order to download the dependencies from Oracle (http://www.oracle.com/technetwork/database/features/instant-client/index-097480.html) you need to register an account.
After you made your account you need to download the following files in the ~/dev/oracle directory:

http://download.oracle.com/otn/linux/instantclient/122010/oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm

http://download.oracle.com/otn/linux/instantclient/122010/oracle-instantclient12.2-jdbc-12.2.0.1.0-1.x86_64.rpm

http://download.oracle.com/otn/linux/instantclient/122010/oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm

http://download.oracle.com/otn/linux/instantclient/122010/oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm

http://download.oracle.com/otn/linux/instantclient/122010/oracle-instantclient12.2-odbc-12.2.0.1.0-2.x86_64.rpm

http://download.oracle.com/otn/linux/instantclient/122010/oracle-instantclient12.2-tools-12.2.0.1.0-1.x86_64.rpm

From what I can recall, not all of these are needed but at the time when I took the notes I wanted to do some extra stuff, you have it all.

To install the rpm files on a debian system:

sudo apt-get install alien libaio1 -y
# Create the deb files from the rpm ones:
sudo alien oracle-instantclient12.2-*
# install the deb files
sudo dpkg -i oracle-instantclient12.2-*.deb
# setup the libraries
sudo sh -c 'echo "/usr/lib/oracle/12.2/client64/lib" > /etc/ld.so.conf.d/oracle.conf'
sudo ldconfig
# check that the libraries are loaded with
sudo ldconfig -p | grep oracle

Setup the Oracle variables that many programs lool for by adding to the /etc/profile.d/oracle.sh file:

export ORACLE_HOME=/usr/lib/oracle/12.2/client64
export PATH=$PATH:$ORACLE_HOME/bin
export ORACLE_INCLUDEDIR=${ORACLE_HOME}/include
export ORACLE_LIBDIR=${ORACLE_HOME}/lib

Fix SDK (.h files) finding if there is no include directory under ORACLE_HOME, and it is located over in /usr/include/oracle/, by creating a symbolic link to assist packages looking for these header files:

sudo ln -s /usr/include/oracle/12.2/client64 /usr/lib/oracle/12.2/client64/include
sudo mount --bind /usr/include/oracle/12.2/client64 /usr/lib/oracle/12.2/client64/include

See also https://help.ubuntu.com/community/Oracle Instant Client for more info.

mkdir -p ~/dev/cpp && cd ~/dev/cpp
git clone https://github.com/qgis/QGIS.git qgis
mkdir qgis/build-master && cd qgis/build-master

If you want to look at the compile options graphically you should first do a sudo apt-get install cmake-curses-gui.
Now that the ccmake command is installed you can play around with it by doing a ccmake -G Ninja .. that should first output something like this:

EMPTY CACHE

EMPTY CACHE:
Press [enter] to edit option Press [d] to delete an entry       

CMake Version 3.9.5
Press [c] to configure
Press [h] for help           Press [q] to quit without generating
Press [t] to toggle advanced mode (Currently Off)

By pressing c at least one time you can get an automatic configuration. Most of the options are automatically configured but some need to be specified.

Keep in mind that some of the options appear after configuring others and pressing c again. If it looks ok to you, and you have the press g option, press it and then start the build.

I personally don't use ccmake anymore since I'm using the cmake arguments to configure my build:

cmake \
-G Ninja \
-D CMAKE_BUILD_TYPE=Debug \
-D CMAKE_INSTALL_PREFIX=$HOME/qgis-master \
-D WITH_ASTYLE=ON \
-D WITH_APIDOC=ON \
-D WITH_CUSTOM_WIDGETS=ON \
-D WITH_QWTPOLAR=ON -D WITH_INTERNAL_QWTPOLAR=ON \
-D WITH_SERVER=ON -D SERVER_SKIP_ECW=ON \
-D WITH_ORACLE=ON \
-D OCI_INCLUDE_DIR=/usr/lib/oracle/12.2/client64/include \
-D OCI_LIBRARY=/usr/lib/oracle/12.2/client64/lib/libclntsh.so.12.1 \
..

If you want to install it instead of running QGIS from the output directory you need to do:

sudo ninja install
sudo chown -R $USER:$USER $HOME/qgis-master

You need root privileges for the install if you have built WITH_CUSTOM_WIDGETS, otherwise, the installer won't have installing rights in the QT folders. The above command restores the rights in the $HOME folder to its user.

In order to run QGIS from the install dir, we create the $HOME/qgis-master/bin/start-qgis-master.sh file with the following content:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HOME}/qgis-master/lib/:${HOME}/dev/cpp/oracle-client/instantclient_12_1

${HOME}/qgis-master/bin/./qgis

Rnning sh $HOME/qgis-master/bin/start-qgis-master.sh should start QGIS with ORACLE support.W

Now that pgAdmin 4 is out you may want to install it on your linux machine.

As for debian/ubuntu at least there aren't any packages build yet here's a way to do it.

Create a Python 3 virtual environment

You can do without a virtual environment but that's how I like to do things.

Install the needed packages (you can also build Python from source if you fancy another Python version that what you already have in your OS) with sudo apt-get install python3-dev python3-venv wget

Create a virtual environment and activate it with:
python3 -m venv pgadmin4 && source pgadmin4/bin/activate

Download, configure and run

wget https://ftp.postgresql.org/pub/pgadmin/pgadmin4/v6.11/pip/pgadmin4-6.11-py3-none-any.whl
pip install wheel
pip install pgadmin4-6.11-py3-none-any.whl

As I want to use the Desktop mode I created the pgadmin4/lib/python3.9/site-packages/pgadmin4/config_local.py file and I added according to my usual login user, qgis:

SERVER_MODE = False
LOG_FILE = '/home/qgis/.pgadmin/pgadmin4.log'
SQLITE_PATH = '/home/qgis/.pgadmin/pgadmin4.db'
SESSION_DB_PATH = '/home/qgis/.pgadmin/sessions'
STORAGE_DIR = '/home/qgis/.pgadmin/storage'

Run python pgadmin4/lib/python3.9/site-packages/pgadmin4/setup.py and the setup should be completed.
Depending on your setup, if you encounter permissions errors you should re-run the above command after you fix them.

You can test that everything works as it should by doing python pgadmin4/lib/python3.9/site-packages/pgadmin4/pgAdmin4.py and by opening your web browser to http://localhost:5050

Create a service

Create a /etc/systemd/system/pgadmin4.service service file containing:

[Unit]
Description=Pgadmin4 Service
After=network.target

[Service]
User=qgis
Group=qgis
WorkingDirectory=/home/qgis/Downloads/pgadmin4/
Environment="PATH=/home/qgis/Downloads/pgadmin4/bin"
ExecStart=/home/qgis/Downloads/pgadmin4/bin/python /home/web/Downloads/pgadmin4/lib/python3.9/site-packages/pgadmin4/pgAdmin4.py
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Run the service with sudo systemctl start pgadmin4.

Test if everything is fine by doing sudo systemctl status pgadmin4.

Set it up to autostart when linux boots by doing a sudo systemctl enable pgadmin4.

That's it!

As QtWebKit Python bindings were removed from Debian Testing/Stretch QGIS is unable to start with Python support and consequently with Python plugins.

Yes, there aren't many Debian Testing users in the GIS world, but there's a specific case for some with newer hardware.

If you have an Intel Skylake GPU in use like I do and you're using Linux then you probably already know preliminary support in the kernel has been only added since kernel version 4.3 with 4.4 supporting it fully.

Thus, you cannot use Debian Jessie or Ubuntu Trusty. Several months ago when I bought my current laptop the only viable option was Debian Testing and until this issue appeared it has served me without flaws. Now that Ubuntu Xenial (16.04), which is based on Debian Stretch, is in the open you can also work with it but it probably suffers from the same caveat.

The only solution is to build python-qt4 with WebKit support.

The steps are:

• sudo apt-get install bison cmake dh-python doxygen flex gdal-bin git graphviz grass-dev libexpat1-dev libfcgi-dev libgdal-dev libgeos-dev libgsl-dev libopenscenegraph-dev libosgearth-dev libpq-dev libproj-dev libqca2-dev libqca2-plugin-ossl libqjson-dev libqscintilla2-dev libqt4-dev libqt4-opengl-dev libqt4-sql-sqlite libqwt-dev libspatialindex-dev libspatialite-dev libsqlite3-dev lighttpd locales pkg-config poppler-utils python-all python-all-dev python-future python-gdal python-mock python-nose2 python-psycopg2 python-pyspatialite python-sip python-sip-dev python-yaml qt4-dev-tools qt4-doc-html spawn-fcgi txt2tags xauth xfonts-100dpi xfonts-75dpi xfonts-base xfonts-scalable xvfb cmake-curses-gui libqtwebkit-dev

Note: From the normal qgis dependencies (see INSTALL document) I removed the pyqt4-dev-tools pyqt4.qsci-dev python-qscintilla2 python-qt4 python-qt4-dev python-qt4-sql packages and I added the libqtwebkit-dev but you don't need to remove anything from the dependencies as they will simply be overwritten.

• Download PyQt
• Extract the archive and cd to the directory with tar -xvf PyQt-x11-gpl-4.11.4.tar.gz && cd PyQt-x11-gpl-4.11.4
• Configure with the option to also build the QSCIntilla api with python configure-ng.py --qmake=/usr/bin/qmake-qt4 --qsci-api
• make && sudo make install
• Download QScintilla
• Extract the archive with tar -xvf QScintilla_gpl-2.9.3.tar.gz
• Cd to the Qt4Qt5 subdirectory with cd QScintilla_gpl-2.9.3/Qt4Qt5/
• qmake-qt4 qscintilla.pro
• make && sudo make install
• the Python bindings for QScintilla are in the QScintilla_gpl-2.9.3/Python directory so cd ../Python should get you there
• python configure.py --qmake=/usr/bin/qmake-qt4
• make && sudo make install should do the trick

Congratulations, you have built python-qt4 with QtWebKit support + QScintilla with python bindings.

You can now build QGIS from source as you basically you have build from source what comes in the pyqt4-dev-tools pyqt4.qsci-dev python-qscintilla2 python-qt4 python-qt4-dev python-qt4-sql packages. All this with QtWebKit support, of course.

When running python commands in the QGIS console, you have some neat features: attribute auto complete, syntax highlighting etc.

Naturally, everyone wants to have these features in their IDE of choice.

Let's see how it looks in PyCharm under Debian/Ubuntu.

After you create a new project:

and create a test.py file with
from qgis.core import QgsDataSourceURI, QgsVectorLayer
you may find out like I did that PyCharm may not recognize it:

You can see from the picture above that we have some red highlights that show that PyCharm is not so happy with what's written there.

By checking the console log of PyCharm I noticed the following:
.. WARN - .skeletons.PySkeletonRefresher - qgis._analysis .. WARN - .skeletons.PySkeletonRefresher - qgis._core .. WARN - .skeletons.PySkeletonRefresher - qgis._gui .. WARN - .skeletons.PySkeletonRefresher - qgis._networkanalysis .. WARN - .skeletons.PySkeletonRefresher - qgis._server
I thought of solving it quickly so I went to File --> Settings --> Project Interpreter (you can also open this up by doing CTRL + ALT + S) and add to the interpreter paths the location of the QGIS Python modules which for me is: /home/web/apps/share/qgis/python.

Which should look finally like this:

Trying to run the simple import script outputted:

/usr/bin/python2.7 /home/web/PycharmProjects/untitled/test.py
Traceback (most recent call last):
  File "/home/web/PycharmProjects/untitled/test.py", line 1, in <module>
    from qgis.core import QgsDataSourceURI, QgsVectorLayer
  File "/home/web/apps/share/qgis/python/qgis/__init__.py", line 36, in <module>
    from qgis.core import QgsFeature, QgsGeometry
  File "/home/web/apps/share/qgis/python/qgis/core/__init__.py", line 34, in <module>
    from qgis._core import *
ImportError: libqgis_core.so.2.14.3: cannot open shared object file: No such file or directory

Hmm...

As I was using a QGIS built from source but I hadn't setup globally or locally the path to the QGIS libraries the following step was doing just that.

So, moments later, after closing PyCharm, doing export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/web/apps/lib and reopening it with ./pycharm.sh I ran the script again and the import was successful.

However, for some reason, PyCharm still was not able to build the skeletons of the QGIS modules so the main goal was still unmet.

I tried so many things to no avail among which I can recall:

• setting the PYTHONPATH outside PyCharm
• installing older versions of PyCharm
• clearing the PyCharm cache to force rebuild of skeletons by clicking File --> Invalidate Caches / Restart.. or by deleting the ~/.PyCharm2016.1/ and even the ~/PycharmProjects/
• even went on Windows and see if I encounter the same problem which I did but I solved it rather quickly by following the blogs of Nathan and Gary which I'll sum up towards the end of this page.

If you encounter this kind of problems you better begin and checkout the PyCharm idea.log file.

You can find the its location by going to Help --> Show Log in Files.
A tail -f ~/.PyCharm2016.1/system/log/idea.log proved useful for me as I could see what skeletons are updating and whether the QGIS modules are parsed or not.

Debian/Ubuntu solution

Open the Python console from QGIS and run the following:

import sys
print(sys.path)

It should output something like:

['/home/web/apps/share/qgis/python/plugins/processing', '/home/web/apps/share/qgis/python', u'/home/web/.qgis2/python', u'/home/web/.qgis2/python/plugins', '/home/web/apps/share/qgis/python/plugins', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/web/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', u'/home/web/.qgis2//python', '/home/web/apps/share/qgis/python/plugins/fTools/tools']

Clean the output by:

• replacing ', ' and ', u' with :
• delete the encompassing [' and ']

Now put it in a PYTHONPATH variable.

The final command should look like

export PYTHONPATH=$PYTHONPATH:/home/web/apps/share/qgis/python/plugins/processing:/home/web/apps/share/qgis/python:/home/web/.qgis2/python:/home/web/.qgis2/python/plugins:/home/web/apps/share/qgis/python/plugins:/usr/lib/python2.7:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/home/web/.local/lib/python2.7/site-packages:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PILcompat:/usr/lib/python2.7/dist-packages/gtk-2.0:/home/web/.qgis2//python:/home/web/apps/share/qgis/python/plugins/fTools/tools

If you built your QGIS from source don't forget to point to the libraries if there's nothing setup by doing export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/web/apps/lib

IMPORTANT!!!

Before testing if it works don't forget to remove whatever skeletons etc. PyCharm did before by removing the ~/.PyCharm2016.1 directory as File --> Invalidate Caches / Restart.. in PyCharm did jack squat for me.

You can see it works now:

Groovy!

Now, depending on what you want:

• you can make a script that sets the variables and then runs Pycharm
• add the variables to the ~/.bashrc
• other ..

As I want to have a PyCharm aware QGIS icon on desktop I created a file called pycharm-qgis-lts.sh in the PyCharm directory with the content described above:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/web/apps/lib
export PYTHONPATH=$PYTHONPATH:/home/web/apps/share/qgis/python/plugins/processing:/home/web/apps/share/qgis/python:/home/web/.qgis2/python:/home/web/.qgis2/python/plugins:/home/web/apps/share/qgis/python/plugins:/usr/lib/python2.7:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload:/home/web/.local/lib/python2.7/site-packages:/usr/local/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages:/usr/lib/python2.7/dist-packages/PILcompat:/usr/lib/python2.7/dist-packages/gtk-2.0:/home/web/.qgis2//python:/home/web/apps/share/qgis/python/plugins/fTools/tools
sh /home/web/Downloads/pycharm/bin/pycharm.sh

The last line executes the pycharm.sh script.
I made the file executable for the owning user with chmod u+x pycharm-qgis-lts.sh.

In order to also see it as a Desktop application I followed the steps outlined here and I created a pycharm-qgis-lts.desktop file in the ~/.local/share/applications directory with the content:

[Desktop Entry]
Version=1.0
Type=Application
Encoding=UTF-8
Name=PyCharm QGIS LTS
Icon=/home/web/Downloads/pycharm/bin/pycharm.png
Exec="/home/web/Downloads/pycharm/bin/pycharm-qgis-lts.sh"
Comment=Develop with pleasure!
Categories=Development;IDE;
Terminal=false
StartupWMClass=pycharm-qgis-lts

That's it!

Windows Solution

Create a qgis_pycharm.bat file with the following content:

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W64
SET PYCHARM="C:\Program Files (x86)\JetBrains\PyCharm 2016.1.4\bin\pycharm.bat"

call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-7.0.4\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-7.0.4\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis

 
start "PyCharm aware of QGIS" /B %PYCHARM% %*

You may need to change your paths accordingly but this certainly works. I won't go into details but you can check Nathan's blog or Gary's blog from which I got the main ideas.

Just running the file ensures you'll have a QGIS-modules aware PyCharm.

I've been using PostgreSQL on CentOS for several years without a glitch.
Since upgrading to CentOS 7.2 I experienced a crash for the first time.

PANIC: semop(id=..) failed: Invalid argument

While digging into the problem I noticed that the crashed happened all the time after a crontab script ran.
Doing an Internet search revealed the problem is due to systemd 212 that came with Centos 7.2 as it has the feature of removing all IPC objects owned by a user if she or he fully logs out.

The solution is to set RemoveIPC=no in /etc/systemd/logind.conf and to restart systemd-logind service to disable this.

This is why it's good to do LVM or VM snapshots before upgrading, to ensure you have a way to revert when things go wrong.

• reference 1
• reference 2

Like every software, QGIS has bugs and one of the many ways that people can contribute to QGIS is bug reporting.

Actually, I think that bug reporting and documentation translation and writing are one of the best ways to join the QGIS effort and contribute back to this nice community.

This post gives some hints on bug reporting and I would like to start it by simply saying that you should take your time when doing a bug report.
By this I mean that you should give specific details and be as explicit as you can as in what conditions that bug appears.

You can checkout https://hub.qgis.org/wiki/17/Bugreports on more info about the bug reporting procedure.

The best way to present a QGIS bug is by accompanying your report with data and the exact steps that are needed to be done so that the issue can be easily reproduced.

I strongly believe that a good report (with data attached) makes a very good invite for any developer to check out the report. As developer time is as precious to him/her as everyone else's time the fact that you (the bug reporter) invested time so that the developer doesn't waste his/hers will probably get your bug checked out faster than others that are not detailed enough and can give headaches when trying to reproduce.

For vectors, you can save your data in shapefile or sqlite format.

However, keep in mind that it's best that you don't use the shapefile format for anything, including bug reporting.
Although you can easily search for articles that say why I will just paste some of the why nots I could easily find:

• shapefiles have serious problems with attributes. For example, they cannot store null values, they round up numbers, they have poor support for Unicode character strings, they do not allow field names longer than 10 characters, and they cannot store both a date and time in a field. These are just the main issues
• shapefiles do not work well for storing information in a language other than English

So, although the shapefile format is widely used (inertia), I strongly recommend saving your portable data in the sqlite format instead of shapefiles. For anything!

You can use the Create SpatiaLite Layer tool to do it:

The created sqlite file is over 5.6MB in size as it contains all the coordinate systems definitions in it.

If you want to shrink it you can simple delete the CRS's that you don't use for your data by connecting with DB Manager to the database and by running the following query:

DELETE FROM spatial_ref_sys WHERE srid <> 3844;
DELETE FROM spatial_ref_sys_aux WHERE srid <> 3844;
VACUUM;

Basically, the above queries delete all the CRS definitions other than the one that I am usually using, that has the srid of 3844.
Keep in mind that the Vacuum command is needed in order to shrink the database after the data is deleted.

You can see from the terminal output below to what amount the data shrinked:

web@thinkpad:~/$ ls -al -h water.sqlite 
-rw-r--r-- 1 web web 5.6M Jan 24 16:05 water.sqlite

web@thinkpad:~/$ ls -al -h water.sqlite 
-rw-r--r-- 1 web web 667K Jan 24 16:07 water.sqlite

web@thinkpad:~/$ ls -al -h water.sqlite 
-rw-r--r-- 1 web web 128K Jan 24 16:11 water.sqlite

I'm back from my first QGIS hackfest and it was wonderful.

It was really interesting as I met a lot of nice people and we did good networking.

I didn't knew first as what to expect as I hadn't met anyone in person before and I was the only Romanian attending (actually the first to a QGIS hackfest). Nothing further from the truth as the community is great and the majority if not all the community members like to get in touch with newcomers.

Basically, we had great communication and we went out each night to grab a beer and we stood quite late each time.
I think that a good connection was also enabled by the nice setup in Gran Canaria, with lots of things to do, and warm weather although it was the beginning of November. Having the possibility to staying the nights in t-shirts (more that 20 degrees) certainly contributed to a nice setup.

What really amazed me is that there were lots of people attending that were involved with GIS/Hydraulic modeling in the water/wastewater industry but somehow, they hadn't connected before and they didn't knew about each others interests.
I already knew of course about the common interests of Denis Rouzaud and Matthias Kuhn whom with we're working along in the QWAT and QGEP projects.
I met David Erill and Carlos López Quintanilla that had strong ties with the giswater project but there were also others that had similar interests.
Even the organizers from the Office of Free Software of the University of Las Palmas, represented by Pablo Fernández Moniz, were working along with students on practical projects for the local water company, which is a tremendous thing.

On a side node, it's a pity that we don't have tight collaboration between Romanian universities and companies that are involved in the matching industries. Some collaborations may exist, but to the best of my knowledge, they're the exception.

We did talk about combining our efforts and about the possibility of merging projects, data-models and so on. It would be very nice and productive for everyone to build a bigger water community around QGIS so that the knowledge, coding and all the efforts are centralized.