Tuesday, 17 July 2018

Open Data KIT tutorial and install with postgres DB

Open Data Kit basic tutorial



Opend Data Kit (ODK) is a free opensource java applications tools for data capturing and aggregate the data and store it in a server. Uses of ODK is for digital surveys instead of the physical paper survey.

Before ODK was introduced, manual surveys have a long process to implement.
The physical paper survey involves decide the questions on the paper survey form, Collecting the survey data and finally aggregate survey data results. 

Advantages of ODK 

  1.  Provides Java server for collecting data
  2.  Provides Mobile
  3. Eliminate paper survey and make a digital survey

ODK modules


Odk is not a single tool but it has different following modules

  1. ODK Form
  2. ODK Collect
  3. ODK Aggregate

ODK installation on amazon web services


First, generate war file using odk agggregate packe on windows.
the following  are required to setup ODK on windows

  • Java installation 
  • ODK aggregate package
  • Tomcat
  • Postgres

   Download ODK package

Go to the official opendatakit website and download ODK package for corresponding operating system Downloaded ODK Aggregate v1.4.3 windows-installer.exe and click on it for follow the steps.

odk setup



open data kit install


odk install


The above screenshot explains about the folder where odk application.war is deployed 

ODK platform amazon


odk on tomcat amazon

odk ssl configuration

odk tomcat configuration

Above ip address is to provide Amazon public IP address


odk postgres setup

odk database configuration

This IP address is the location of an instance where Postgres is installed. ODK and Postgres are installed on the same machine, so providing 127.0.0.1 as Postgres instance
odk aggregate postgres database settings

odk aggregate database settings

This screen talks about  odk application database, user details information


odk instance name

odk aggregate email configuration

odk aggregate configuration

setup odk aggregate configuration

odk aggregate configuration success


The above setup creates following files as per below screenshots.

create_db_and_user.sql :- This is SQL script for  creating database odk_prod , user :- odk_user, password: odk_user and granting all privileges for odk_user
ODKAggregate.war :- This is odk aggregate application need to deploy in tomcat server.

odk SQL and war files

Postgres Installation

Please install postgres. Once postgres is installed, please follow below steps to create odk database, odk users

1.     Execute the Postgres SQL shell (psql) commandline client using the postgres username and password (the password you chose during the installation of the database), and supply the create_db_and_user.sql to it. To open the psql commandline client, open a cmd window (Windows), terminal window (MacOS) or term window (Linux) and 'cd' to the directory containing the psql client (psql.exe on Windows). Then, please type:
2.  psql --username postgres
And enter the PostgreSQL server password. You will then be within the psql commandline client. You should see a 'postgres=#' prompt. Type:
\cd D:/odksetup/ODKAGG~1
\i create_db_and_user.sql
\q
Here is what this looks like on Windows, with what you must type in red:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Windows\System32>cd "\Program Files\PostgreSQL\9.0\bin"

C:\Program Files\PostgreSQL\9.0\bin>psql.exe --username postgres
Password for user postgres:*******
psql (9.0.4)
WARNING: Console code page (437) differs from Windows code page (1252)
         8-bit characters might no work correctly. See psql reference
          page "Notes for Windows users" for details.
Type "help" for help.

postgres=# \cd D:/odksetup/ODKAGG~1
postgres=# \i create_db_and_user.sql
CREATE DATABASE
CREATE ROLE
GRANT
ALTER DATABASE
WARNING: Console code page (437) differs from Windows code page (1252)
         8-bit characters might no work correctly. See psql reference
          page "Notes for Windows users" for details.
You are now connected to database "odk_prod".
CREATE SCHEMA
GRANT
odk_prod=# \q
C:\Program Files\PostgreSQL\9.0\bin>



Next step is to install java and tomcat. once installed deploy war to tomcat

 Odk application deploy to tomcat


copy the ODKAggregate.war to the /var/lib/tomcat6/webapps/ODKAggregate directory of the Apache Tomcat installation.

Self SSL security generation on Ubuntu 

Please generate the certificate using the following command

sudo keytool -genkey -alias admin -keypass adminpass -keystore certificate.jks -storepass adminpass

The above command generates certificate.jks in the current folder. Please copy certificate.jks file to /var/lib/tomcat6/conf and add/modify below entries in /var/lib/tomcat6/conf/server.xml

<?xml version="1.0" encoding="UTF-8"?>
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true" maxThreads="150" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" keystoreFile="conf/certificate.jks" keystorePass="adminpass" />

8443 is SSL port for tomcat and 8080 is for normal HTTP request
Next step is to start Apache Tomcat server(if not already started)

visit http://ipaddress:8080/ODKAggregate/ and go to the Management / Permissions tab to complete configuring the access rights for ODK Aggregate.


Read More

Learn node red tutorial with examples

why node-red tools required?


Learn nodered  tutorials with examples
From 2000 onwards, We have computers and desktop that runs web and desktop applications.  A few years back, mobiles and tablets came into existences and popular still continues. The industry started developing mobile apps that run on these devices. overall programmers write a software code for web, desktop and mobile applications. With recent advancements in technology. a lot of devices came into existence. different devices are sensors in cars/motors/home and electronic devices. Internet of things is a concept where these group of devices connected to the internet to exchange the data. We have to write a program/embed a code on this hardware devices to share data across cloud and DB servers.

A lot of Devices types introduced and a lot of data.  there are many things to dealing with devices.

It is very difficult to write an embed program or code and test it on each device type as it takes too much time to get it running and these devices produce/consumes a lot of data which needs Service API interaction with external services. To simplify this, we need to rapid application development to deal with devices. Because of all these things, Node-red visual flow tool Editor was introduced to link/integrate with different services without complex coding knowledge. It just needs to connect wires/flows node points in the editor and deploy the code easily. Mainly Node-red is useful for design, develop and testing  Internet of things  applications

These devices already have support for programming services via different devices like Raspberry Pi,  Android Arduino based devices
there are many Internet of things sample applications types

1. An electric motor can be controlled using Android devices to start/stop. 
2 Home appicatios like door lock and fan sensors to switch on/off via mobile
3. Weather sensors
From all these devices, a lot of data produced and stored in the cloud which we can use it for the analytical purpose to process and take better decisions for future purposes. 

 What is a node-red tool?

Node-red is the flow-based editor and it is an opensource application.  

It provides browser based visual flow editor and it is very easy  for developers/designers to wire the different nodes and services. here nodes are hardware devices This is a lightweight tool and built on the node.js framework. performance is good.

Node-red Installation and setup

To configure node-red tool, node.js installation is required. Please see steps documented at Npm and node installation.

Once NPM and node command is set up and running successfully. next step is to install node-red software

Setup on windows

-g option is to install the node-red globally

npm install -g --unsafe-perm node-red

Setup on Linux based operating systems


sudo npm install -g --unsafe-perm node-red

this installs node red successfully, run node-red command. this starts the node-red server on your machine and logs the below message to console.

17 Jul 22:28:41 - [info] Starting flows
17 Jul 22:28:41 - [info] Started flows
17 Jul 22:28:41 - [info] Server now running at http://127.0.0.1:1880/

then access http://127.0.0.1:1880/ in your browser, admin UI opened. By default user login details not required.

node red dashboard

dashboard admin tutorials

This screen has three columns. the left column has node section center column flow editor where nodes dragged, dropped and wiring using API endpoints.
Right side column has info and debugging tabs to test the flow
Top right side has an option for production deployment.

Different types of nodes available


Inject node is to manual triggering of a node during the scheduled interval. Time will be configured.
Debug nodes are used to log the payload and object message to console.
template node supports mustache based template
dashboard section has different nodes for HTML form components
social section has nodes supports social media providers.
storage section nodes has support for file storage

there are nodes for TCP/WebSocket protocols support for communication with services

Features of the Node-red application


  • It is built on Nodejs architecture
  • Simple to use graphical flow editor
  • Very easy to set up the local environment
  • There are lot cloud providers like Microsoft Bluemix, Bluemix and amazon web services 
  • support for accessing third party services
  • Easily save and get the data from the database.
  • Rapid application development for Internet of things
  • Created Flows are stored as a JSON object. So you can do import/export flows
  • Docker support

nodered alternative tools

there is another alternative to nodered is total.js platform available for the Internet of things (IOT). This is also based on nodejs framework

I will write next articles on Learn nodered examples on raspberry PI devices.


Read More

Typescript typeof, instanceOf operator examples

typeof operator introduction:


typeof and  instanceof tutorials
This operator used to get a string specifying the type of variable.  This will very helpful for developers to know the type of the object at runtime.

typeof operator syntax

typeof {operand}

parameter operand is supplied either as the primitive data type of custom object. possible operand values are variables, object variables, and strings.

var msg = "Cloudhadoop";
console.log(typeof msg); // returns string as string format

var variableNumber =16;
console.log(typeof variableNumber); // returns number as string format

var variableBoolean =true;
console.log(typeof variableBoolean); // returns boolean as string format

var variableNull =null;
console.log(typeof variableNull); // returns object as string format


var variableArray:string[]= ["1","2","3","4"] 
console.log(typeof variableArray); // returns object as string format

let variableTyple: [string, number]= ["type operator", 20]; 
console.log(typeof variableTyple); // returns object as string format
let variableAny: any = 4;
console.log(typeof variableAny); // returns number as string format

let variableUndefined: undefined = undefined;
console.log(typeof variableUndefined); // returns undefined as string format

enum Monday { Sunday, Monday, Tuesday }
let enumVariable: Monday = Monday.Sunday;
console.log(typeof enumVariable); // returns number as string format

var myFunction = new Function('123 + 6');
console.log(typeof myFunction) // outputs 'Function'

var currentDate = new Date();
console.log(typeof currentDate) // outputs 'object'

Learn instanceof operator 


instaneof operator checks if the object is another object type and returns true. This checking is done at runtime. This will be checked in conditional e

Syntax 

name instanceof type

the name is of object name and type is of object type

instanceOf Examples

In the below code, date object name checks against Class and returns.
any variable returns false against date object
var todaysDate = new Date();
var variableNumber: any = 12;
console.log (todaysDate instanceof Date) // returns true
console.log (variableNumber instanceof Date) // returns false

instanceof class example

if undefined type checked with class, return false

class ClassExample {}
let classes: Array = [];

console.log(classes[0]); // returns undefined
console.log(classes[0] instanceof ClassExample);
//returns false  undefiend is not of ClassExample 

classes.push(new ClassExample());
console.log(classes[0]); // outpus empty class
console.log(classes[0] instanceof ClassExample); // true

instanceof interface example

This is an example usage of the interface with instanceof operator

interface InterfaceExample { }
var interfaceVariable: InterfaceExample = { };
console.log(interfaceVariable instanceof InterfaceExample); // compilation error


class ClassExample { }
var variableClass: ClassExample = { };
console.log(variableClass instanceof ClassExample); // reutnrs false and no compilation error

as you see interface gives 'InterfaceExample' only refers to a type, but is being used as a value here error
Read More

Typescript Number to String or String to Number conversion examples

Typescript primitive data type conversion

Number and String are basic primitive data types in typescript programming language. The number to String and String to Number conversion are one of the simple tasks of typescript developers. For conversion to number, String must contain only numbers.

The number to String conversion examples.

We can do many ways to achieve this.

1 Append with an empty string.
2. toString() method
3. using string() method

Append with an empty string.

number values are 11,13.  add an empty string to Number value returns a string object typeof method checks value type and returns primitive type as a string

var a: number = 12
var str: string = a + '';
console.log(typeof (a)) // outpus number  as output
console.log(typeof (str)) // outpus string as output
console.log(str) //outputs 12 as string value

var a: number = null
var str: string = a + '';
console.log(typeof (str)) // outpus string as output
console.log(str) //outputs null in the form of string

var a: number = undefined
var str: string = a + '';
console.log(typeof (str)) // outpus string as output
console.log(str) //outputs undefined in the form of string

toString() method

calling toString() method  on any data types returns string object

var a: number = 17
var str: string = a .toString()
console.log(typeof (str)) // outpus string as output
console.log(a) //outputs 12 as string value

if we use null and undefined with toString() method, compiler through 'Cannot read property 'toString' of null' error

string() method

this is the easiest way of a convert to string method

var a: number = 12
var stringValue = String(a);
console.log(typeof(stringValue) // outpus string  as output

String to Number example 

This conversion can be achieved using various approaches

1. parseInt
2. parseFloat
3. Number()

parseInt() and parseFloat() method

parseInt and parseFloat methods take input number in the form of a string and return number

console.log(parseInt('32', 10)) // outpus 32 as output
console.log(parseInt('45')) // outpus 45 as outpu
console.log(parseInt('45abc')) // outpus 45 as output
console.log(parseInt('null')) // outpus NaN as output

console.log(parseFloat('32.32')) // outpus 32.32 as output
console.log(parseInt('45')) // outpus 45 as output

Number() method

Number() constructor takes a string as a parameter and returns number as output

console.log(Number('12')) // outpus 12 as output
console.log(Number('null')) // outpus NaN as output
console.log(Number('undefined')) // outpus NaN as output

This is understanding on primitive types conversion in typescript language.




Read More

Monday, 16 July 2018

Learn Javascript Blob Object with examples

Learn Blob Object tutorial

Javascript blob object filereader
In this blog post, I will walk through the javascript blob object examples. In javascript, File data can be represented in the Blob object. file types are images, video, audio, doc and excel files.

when dealing with local or remote files, blob object used. Blob content stored either temporary or in file storage.

Blob Object Syntax

Blob object can be created in many ways

The object can be created using Blob empty constructor or  Blob parameter constructor

Blob(blob parts[, options])
Blob() 

Parameters Syntax

Blob has two properties

The Size property - return size of bytes of a blob object
The type property - return the MIME type of blob object.return empty, If a type is not in proper format

Blob Methods


Blob.slice([start[, end[, contentType]]])

we can also create a new blob object using bytes range data and content type. This is another way of creating a blob object

var blob = new Blob();
console.log(blob.size);  // returns zero 
console.log(blob.type);  // returns empty as blob format type is not known

var blob = new Blob(['name', "kiran"], { type: 'plain/text' });
console.log(blob.size);  // returns 9 bytes 
console.log(blob.type);  // returns plain/text as output

Examples with tutorials

We will see basic examples of a blob object usage

How to convert Image to Blob object in Javascript?

This example also displays image in HTML using blob object

The image is presented in local or URL Make an asynchronous call to load remote URL ie image URL using XMLHttpRequest object of an Image URL, store the image data in the blob. Create a blob object using raw data.

<img id="logo"/>
var xhp = new XMLHttpRequest();
xhp.open( "GET", "https://www.freelogodesign.org/Content/img/laptop_free-logo.png", true );
xhp.responseType = "arraybuffer";
xhp.onload = function( e ) {
    var array = new Uint8Array( this.response );
    var blob = new Blob( [ array ], { type: "image/jpeg" } );
    var urlCreator = window.URL || window.webkitURL;
    var imageUrl = urlCreator.createObjectURL( blob );
    var img = document.querySelector( "#logo" );
    img.src = imageUrl;
};
xhp.send();

Check whether two blobs sizes are equal are not

Blob contents are not checked and using size property to check sizes are equal
var blob1 = new Blob(['a']);
var blob2 = new Blob(['b'])
var blob3 = new Blob(['ac'])
console.log(blob1.size===blob2.size) // returns true
console.log(blob1.size===blob3.size) // returns false

Get data from Blob using FileReader

Blob can be of XML, audio/video, word files which have raw bytes of data.
This is also FilreReader blog example for reading data and returns URL or in string format

var reader = new FileReader();
reader.onload = function(event) {
    var dataURL = event.target.result,

});

reader.readAsArrayBuffer(blob);

Read More

Javascript console object - Logging tutorials, example

Console Logging -  Important for Debugging


Console Object logging
During development environment, Developer needs to log the useful debug messages and print flow variable values. So application has to access the browser window console functionality.
For that, javascript provided Console object to log the messages to the console window. Console object behavior can be varied from browser to browser. All popular browsers support the major functionality of the console object. This object is a global object and can be accessed freely anywhere. all the information from console object output to console window


Browser console window. 

console window can be enabled in all popular browsers.

Chrome console window will be opened by right click  ( more tools + developer tools) on the browser and select Inspect option. the window opened and select the console tab as shown below
chrome console window inspect

Firefox browser console window will be opened using Firebug extension or inbuilt in developer tools console.

For Safari browser, Please see Safari Developer tools  article.

Console Object log syntax

console.log(  object1,{optional objects});
console.log(  message1,{optional objects});

console.log is in two variants one is having parameters which
contain a list of objects and one object is mandatory. other is having messages followed by zero or more objects.

Console Object methods

console object has a lot of methods for debugging logs/different logger formats/ display timer information.

The assert() method 

Print log messages to console if assert is false, if true, returns nothing. This will be useful in unit testing.

Syntax

console.assert(assertion, {List of objects, Atleast One is required);
console.assert(assertion, messages {list of options/objects}); 

Example

console.assert(false, "test passed");  // outputs "VM339:1 Assertion failed: test passed" message colored in red to console

console.assert(true, "test failed"); // outputs nothing to console

The clear() method 

this method clear the console messages. 

if preserve option is set, outputs 'console.clear() was prevented due to 'Preserve log'' to console window
if  preserve option is not set and output message "console was cleared" 

Syntax and Example:-

console.clear()

The count() method usage

this logs the count of the number of times count is being called.

Syntax :- 

console.count([label]); // syntax has label which is optional

Example:- 

console.count('calledcount'); // outputs  calledcount:1 

console.count('calledcount'); // outputs  calledcount:2
console.count('calledcount'); // outputs  calledcount:3

The dir() method usage

Output javascript object and their properties information in hierarchical tree(childs) form

Syntax and Example

console.dir(object);  // Syntax
console.dir(document.body); // outputs body object data in tree form as shown in below diagram

console dir method example


The dirxml() method usage

prints javascript object data in the form of XML/HTML to console window 

Syntax and Example:-

console.dirxml(object);
console.dirxml(document.body); // outputs body object data in html form as shown in below diagram.
console object dirxml method



Group(), groupEnd() and GroupCollapsed() method usage

these methods are used to group the console log messages and shown data in hierarchical format as shown below.
You can create a nested group inside group using group()method. groupCollapse creates new group + opens group by default

Syntax and Example:-

console.log("My Computer");
console.group();
console.log("C-Drive");
console.log("c-drive-file");
console.groupEnd();
console.group();
console.log("D-drive");
console.warn("D-drive-File");
console.groupEnd();
console.log("File-2");
console.log("File-3");

console object group methods


The table() method usage

Displays the data in table format.

Syntax 
console.table(data [, columns]);

data:- data array or object data to print
columns:  column array to include in the output

Example

function Employee(name, salary) {
  this.name = name;
  this.salary= salary;
}
var emp1 = new Employee("Sai", "9000");
var emp2 = new Employee("Ganesh", "8000");
var emp3 = new Employee("Venkat", "5000");

console.table([emp1,emp2,emp3]);

please see the screenshot form output how it looks

console Object table method example

time(), TimeEnd() methods

This method is used to log the time taken for operations/calculation processing.
time() method initializes and starts the timer when timeEnd encountered, timer stops and log the time taken.

console info() method usage

log information message to the Web console

The trace() method

trace method logs a stack trace of current request thread.
syntax 

console.trace()

The warn() method

prints warning messages to the web console

Syntax

console.warn(Obj1,[list of objects])
console.warn(msg,[list of objects])

 The Error() method usage

Prints Error messages to the web console. The syntax is similar to log method

Following 5 log messages, types are in order from left to right where  Left is termed as a critical message and Right one is termed as Low.

FATAL --> ERROR --> WARN -->INFO -->DEBUG


Read More