How to find a file on Linux

There are zillions of small little Linux commands that make life easy. Those tiny little commands may save you hours of time and tons of frustration. Wheneever I look as the terminal of one of my colleagues, I discover yet another trick that they use to make life easy. May I present a little command… Read More »

Add data on Big Data in Hive and Impala

This post provides info on how data may be added on a Big Data platform with help of Hive and Impala. We start with a dataset that is stored on a Linux platform. We will show how these data can be stored on a HDFS/ Hadoop platform. After that, we will show how these data… Read More »

Dropping a table in Oracle

To drop a table is straightforward in Oracle. One might simply issue a drop table statement. Let us assume we have table HH. When “drop table HH” is fired, the table is removed. However, an error is returned if the stable doesn’t exist. Again: if table HH doesn’t exist and a SQL “drop table HH”… Read More »

transpose a record in Oracle

Transpose a record in Oracle isn’t easy. I had a small table with several records and one value in a record. I wanted to transpose that table into one record with the values adjacent to each other. The question: how to accomplish this? Recently, Oracle introduced the pivot facility that allowed this procedure. The code… Read More »

SQL Loader

The SQL Loader is a facility that allows you to load data files blazingly fast. It is able to do as data files are directly written to disk without any overhead. It needs two files: a control file and a data file. The process generates a log file, that provides information on whether process has… Read More »

Calculate distances in Oracle

Oracle allows you to calculate a distance between two points. Such calculation is not trivial as one must take into account that distances are calculated over the globe and the points are indicated on a longitude – latitude base. If we calculate the distance between longitude, latitude =(0,0) and (1,1), one has about 156 kilometers.… Read More »

Transpose in Oracle

Transposing data means changing data from a row into a column. Starting from version 11, this is possible in Oracle as well. It is possible to translate some values that appear in rows into columns. Doing so, a new table can be created that has an additional set of columns with column names being derived… Read More »

Reading and writing in Java

Reading and writing from and to files is not easy in Java. This can already be seen if one simply googles on “Java Filereader problem”. This generated 477000 hits. Apparently, reading (and writing) is not trivial. I wrote a small program that is able to read a file and copy its contents to another file.… Read More »

Another set of keys and values

Another example on how mapper and reducers are used in a Hadoop context is given below. This programme is created as three classes. One class is an overall class that calls two other classes: a mapper class and a reducer. The mapper classer reads a file and creates a series of words. In the first… Read More »

Map and reduce – what happens?

In Big Data, the concept of mapping and reducing plays a huge role. The idea is that a a massive dataset is split over several servers. On each server, a part of the data is investigated. This part is called a mapper. In a subsequent part, these parts are merged into an outcome. This latter… Read More »