Abzetdin Adamov's IT Blog

IT is about doing more with less!

ChatGPT as a Coder or Will AI replace Programmers?

Posted by Abzetdin Adamov on January 3, 2023

“If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong.”

A couple days ago during tea time with my students of the joint program ADA University and George Washington University (Computer Science and Data Analytics) one of my students answering the question “what is your top concern about the future?” said “I worry that AI will take my job as a programmer”. I responded saying that probably it’ll not take place earlier than several decades.

Today I decided to check capabilities of the new arrival to AI landscape ChatGPT. Along with ability to be engaged in realistic conversations, chatbot can also generate computer programming codes. Today I’ve submitted the following query to the ChatGPT chatbot launched by OpenAI in November 2022 and built on top of the new version of well known large language model GPT 3.5.
My query was “generate code for web crawler with multithreading support that implements polymorphism in java”

As a response the model generated 50 lines of code in just a couple of seconds. I can’t say the code is ideal and ready to run, but it’s ideal as a template if you consider the time it took to generate, also it can be improved with further interactions with the model. The output is really amazing: it implemented the method for crawling, it implemented concurrency and it wrote the code in a truly object-oriented manner. After changing the last word of the query to “Python”, it generated the same task in Python language.

As a response to my another query “generate data model for HR system using ORM in java”, ChatGPT came with data model consisting of three classes:  “Employee”, “Department” and “Holiday” supporting it with quite detailed description of relationship types between table (one-to-many, many-to-one) and primary keys.
I continued my interaction with “Having just three classes for full HR system is too simple. Can you offer data model for real HR system for the large business entity?”. ChatGPT responded with a very kind reply “Certainly! Here is a more complete data model for an HR system for a large business entity, using Object-Relational Mapping (ORM) in Java:” and according to the model’s point of view, it’s enough for HR system of large business entity to extend data model with several fields in each class and add one new class “PerformanceReview”.

The query “Please, generate data model for HR system using ORM in Go” results with the same data model, but in Go programming language. That means that ChatGPT is familiar with the syntax of all popular programming languages. This also proves another important truth: one can doubt in the ability of model to generate code, assuming that it just brings codes from code repositories (like Github, SourceForge, Codebase, Bitbucket, StackOverflow, etc.), but the last query disproves that assumption. ChatGPT really generates code, learning from billions of code snippets and projects it has access to.
Of course, it’s far away from being mature enough to generate a real software product, or data model for the real system, but certainly, it’s in the right direction and will improve over time.

How it will affect education?

What does it mean for educators and students? Some students may think that they will be able to utilize ChatGPT and other AI systems to delegate them their programming assignments. But, in reality it is not as easy as it can sound. To make the model to produce exactly what is required, one will need to write very accurate and precise requirements that include enough details for the model to solve the task. As you know, good requirements and specifications are up to 70-80% of the success. If students are able to write those requirements, why not spend the remaining 20% of the effort to write the code by himself, instead of risking being caught for plagiarism. At the same time, students can continue using the AI system for the satisfaction of ego and how well they perform as a human in comparison with AI.
It also will bring some challenges to professors as well. Probably, those who want to prevent students from feeding their assignments directly to AI-powered systems, will need to spend more time making assignment descriptions be more abstract and at the same time clear enough for students to generate requirements and write resulting code. Certainly, in light of AI advancements and derived opportunities, the education community will be forced to revise existing policies.


As I can’t refer to myself as distinguished or neither elderly, so even after my experience with ChatGPT, I’m still  of the same opinion: programmers are not at the risk of losing their jobs, at least for another several decades. It’ll take time before the AI will be able to compete with human programmers (not coders). AI can produce the solution for the small coding task and quality will improve over the time.
But when it comes to programming of large and complex enterprise systems with sophisticated business models/logic, AI-powered code generators would not be helpful. This kind software development project is about many components including: project management, business intelligence, business analysis, quality assurance, platform/cloud engineering (DevOps), database management (ORM), data warehousing, backend/frontend development, UI/UX, security, scalability, testing, deployment. All these components should be in harmony with each other and strongly integrated.
Does it mean that AI will never be able to deal with that complexity? Of course, it doesn’t. But it’ll take a long before AI acquires enough knowledge about business and about the overall World. AI will also be prone to mistakes, as humans do, but those mistakes can’t be easily found and explained by humans. Possibly, AI will find a way to do it differently bypassing some steps and making the lifecycle less sophisticated. We will see what way AI chooses, probably, in a couple of decades, or sooner.

Posted in Uncategorized | Tagged: , | Leave a Comment »

Happy Birthday TCP/IP

Posted by Abzetdin Adamov on January 1, 2022

Tcp/ip networking. Transmission Control Protocol. Internet Technology concept.

Today is not just beginning of 2022, 1 January is the birthdays of one of most significant inventions in the history of humanity. January 1, 1983 TCP/IP protocol was born and put in use that for decades underpins the entire Internet. In early 1983 Vint Cerf (inventor of protocol) had no idea which platforms will communicate over TCP/IP and what kind of data to be transmitted over connected devices. Almost 4 decades passed since there, thousands new platforms and applications appear transferring hundreds data type/format using technology invented 4 decades ago… Happy birthday TCP/IP! Happy New Year All!

Posted in Uncategorized | Leave a Comment »

Will Quantum Computing replace Traditional Computing?

Posted by Abzetdin Adamov on April 22, 2020

Sophisticated architecture of IBM Quantum Computing platform

Same as with many other pioneering technologies (like Big Data, Deep Learning, Blockchain, IoT), Quantum Computing become buzzword for IT-empowered community thanks to tech giants including Google, IBM, Intel, Microsoft and promising startups. They invest billions to R&D related with field to secure own piece of pie in case if Quantum Computing become available in widespread use. A lot of people are talking about the technology, but limited have real application experience and proof of concept.
Most probably, in coming decades, Quantum computing will not be same efficient as traditional computers for general computational tasks, but they will be more productive for specific mathematical problems that matches well with their philosophy. What is more realistic, Quantum Computing will never replace traditional computing, instead will work in concert with them empowering each other. The future of Quantum Computing is same unclear as the stability of its superpositions. It is still in its infancy, the support infrastructure is too expensive and unstable.
The World has drastically changed in 20th century thanks to computing revolution recursively changing demand for computation. But if we look to history of computation, the only things that changed since first computer invention, are power, capacity, scale and application, but not the way of computation itself. It is right time for scientific and business communities worldwide to look for new computation paradigm expanding horizons of invention that may bring up an alternative for the traditional one that was invented centuries before the first computer appeared. At the beginning, it is not required for the new concept or paradigm to be better, it is enough to be different…

Posted in Uncategorized | Leave a Comment »

Tech Companies: winners and losers of the lockdown

Posted by Abzetdin Adamov on April 12, 2020

Airbnb and Netflix – opposite business model in context of current lockdown and call “Stay at Home”

The Internet is the most open and powerful platform that brings together those who want to buy something and sell something… The most successful Internet giants mostly do not produce any product or offer sophisticated services. They just provide user-friendly interface designed to collect Data in expense of providing narrow and simple, however largely demanded service. The main idea of the platform is to Analyze in-depth the voluntarily provided massive Data, learn more trying to see end-to-end picture of the client, catch his behavior and try to predict his next action (or even to manage his actions…). And at this point Deep Science (Big Data, Data Science and Analytics, Machine Learning, etc.) can help fine-tune the service to make it much better and appealing for wider audience.

  • Airbnb – the world’s largest accommodation provider owns no single room;
  • Netflix – the world’s largest movie house owns no cinemas;

Business Model is really important in tech industry. But, you never predict the market demand and how exceptional circumstances can affect this demand…
Two Internet giant companies with opposite concept in terms of “Stay at Home”: One of the most successful unicorn company AirBnb plunges down, while another one Netflix experience unseeing demand (badly affecting the speed of connectivity Worldwide)…

Posted in Uncategorized | Leave a Comment »

AICT2020 Coronavirus Statement

Posted by Abzetdin Adamov on April 12, 2020

On March 11, 2020, the World Health Organization declared COVID-19 a pandemic. The safety and well-being of all conference participants is our priority, therefore we are carefully considering our plans going forward.

We are following updates about the situation from the government of the Republic of Uzbekistan, which is working closely with the World Health Organisation (WHO) and will update our plans accordingly. We will follow to safety recommendation from IEEE.

We will provide additional information in timely manner. The AICT2020 continues to accept submissions and peer-revision will be in place according to timeline.

For the most current information about COVID-19, see the following resources:

Posted in Conferences - Events | Leave a Comment »

The IEEE AICT2020 – 2nd Scientific Pilgrimage to the birthplace of al-Khwarizmi

Posted by Abzetdin Adamov on February 21, 2020

Algorithms shape our lives. They decide how much to pay for goods or services, what to watch next, which information/news to see online, who will be our next friend, what route to take to reach a destination, etc. But have you ever thought what was the origin of the word Algorithm? AICT2020 is devoted to the scientific heritage of the man who formalized algorithms – Abdullah Muhammad bin Musa al-Khwarizmi. In 1979, Profs. Andrey P. Ershov and Donald E. Knuth with colleagues organized a Scientific Pilgrimage to the birthplace of al-Khwarizmi, the famous mathematician from Khiva (Khwarezm region), to celebrate the 1,200 years of his birth. He gave his name to the words “Algorithm” and “Algebra” which have become pillars of modern Computer Science and now Data Science.

After 40 years, we plan to organize in the framework of the AICT2020 the 2nd Scientific Pilgrimage to the birthplace of al-Khwarizmi to celebrate the 1,200 years of al-Khwarizmii becoming the head of the House of Wisdom in Baghdad (scientific research and teaching center). Every computer scientist to whom the word “Algorithm” makes great sense is welcome to join us during this Scientific Pilgrimage that will be complemented with a Summer School and Workshop for local university students and conference participants in Khiva/Urgench on 05-06 October, 2020.

Building on the success of previous conferences, AICT2020 is expected to be one of the most significant international academic conferences on ICT applications, which brings together researchers, academics, practitioners around the world to exchange, reveal and promote the latest advances on application of ICT. You are welcome to join us.

Abzetdin Adamov, AICT2020 Genera Chair

Posted in Uncategorized | Leave a Comment »

Setting Up CDH Cloudera Distribution Hadoop Multi-Node Cluster. Step-by-Step Tutorial for Semi-Automated Installation.

Posted by Abzetdin Adamov on January 6, 2020

Hadoop Architecture


This tutorial is about building small Hadoop Cluster (Cloudera CDH 6 – Cloudera Distribution Hadoop) on pre-built Virtualbox Virtual Machines (VM). 5 VMs will be used, one for NameNode (Master) and 4 others for DataNodes (Workers). The 6th VM will be used as a Repository where installation packages of different Components. Using local repository will help to make installation process mush faster. It is also only way in case if Workers don’t have Internet connection. Creation of 6 VMs, installation of OS and Network settings will require quite a lot of time and efforts. To make the process more effective, just one VM can be built and set and after that it can be Cloned 5 times to have necessary number of VMs.
While building clones make sure that all clones are Full Clones (not Linked Clone) and also chose setting Generate New MAC Addresses…

Virtual Machines:

  1. cluster-Node1 – 6GB/30GB VDI (dinamically allocated)
  2. cluster-Node2 – 5GB/30GB VDI (dinamically allocated)
  3. cluster-Node3 – 5GB/30GB VDI (dinamically allocated)
  4. cluster-Node4 – 5GB/30GB VDI (dinamically allocated)
  5. cluster-Node5 – 5GB/30GB VDI (dinamically allocated)
  6. cluster-Repo – 2GB/35GB VDI (dinamically allocated)

Each VM has two adapters: NAT for Internet Conenction and Host-Only for connectivity between all VMs and VMs with Host computer. For the convenience, static IP address was set for each Node.
Host computer is Windows 10 and all Guests are Centos 7.7.
All steps will be implemented using root user. Root user’s password is same for all Nodes for convenience. Install OS Centos 7.7.1908 x86 64 bit Minimal on each of VMs. Image for distribution is available at: Mirror server


Hadoop Cluster may consist of from several to hundreds computers (nodes) those must be set in certain way and installed with special components and tools. This process may require a lot of time in order to implement all necessary steps sequentially on each node. In order to resolve this issue, in current tutorial multiple commands will run sequentially or parallely on remote Nodes performing non-interactive password-based authentication.
SSHPASS utility – enables passing the password to SSH for authentication and avoid being asked to enter password interactively. Bash loop can be used to implement SSHPASS sequentially for each Node. In case if password-less authentication is set, we can use simply SSH.
PSSH utility – enable execution of commands in parallel on a number of remote Nodes through SSH connection. Since in our case after certain point of installation, we use password-less authentication, we will not pass password (otherwise key “-A” is required).

1. Update OS and install necessary utilities.

Since Cloudera Manager will be installed on Node Node1 we start from this VM.

yum -y update && yum -y upgrade

following command to enable EPEL repo on CentOS/RHEL server:

yum install epel-release

yum install python-pip
yum install pssh
yum install sshpass
yum install wget

root password is written into file located in root’s home directory to allow SSHPASS to read password in non-interactive way from file:

echo "password" >> ./passw.txt
chmod 600 passw.txt

2. Configure Hostnames for all computers (Nodes) of Cluster

Edit /etc/hosts and add IP addresses and FQDNs of each Node node1.cluster.vm node1 node2.cluster.vm node2 node3.cluster.vm node3 node4.cluster.vm node4 node5.cluster.vm node5 repo.cluster.vm repo

Additionally, file “/root/hostName” with all FQDN (Fully Qualified Domain Name) of Nodes is created

echo -e "node1.cluster.vm\nnode2.cluster.vm\nnode3.cluster.vm\nnode4.cluster.vm\nnode5.cluster.vm" >> hostNames

3. Set hostnames of all remote Nodes using shell script and SSHPASS utility

cat /root/hostNames | while read HOSTNAME
   sshpass -f /root/passw.txt ssh -T root@${HOSTNAME} << EOF
      echo "==========================${HOSTNAME}"
      hostnamectl set-hostname ${HOSTNAME}
      echo "HOSTNAME=${HOSTNAME}" >> /etc/sysconfig/network

Make sure that commands uname -a and hostname return same domain name

4. To introduce all Nodes to each other

This is optional step purposed to avoid being asked to approve fingerprint when connect to remote node the first time:

mkdir ~/.ssh
chmod 700 ~/.ssh

cat /root/hostNames | while read HOSTNAME
   echo "==========================${HOSTNAME}"
   ssh-keyscan ${HOSTNAME} >> ~/.ssh/known_hosts

To make known_hosts work for all users

cp ~/.ssh/known_hosts /etc/ssh/ssh_known_hosts

Copy appropriate files to all Nodes

cat /root/hostNames | while read HOSTNAME
   echo "==========================${HOSTNAME}"
   sshpass -f /root/passw.txt ssh -T root@${HOSTNAME} << EOF
      echo "==========================${HOSTNAME}"
      scp /etc/hosts/ root@${HOSTNAME}:/etc/hosts
      scp /etc/ssh/ssh_known_hosts root@${HOSTNAME}:/etc/ssh/ssh_known_hosts

5. Set Password-less SSH authentication

In order to install Cloudera CDH using Cloudera Manager, we need to set password-less SSH access from master-node (node1) where Cloudera Manager server will be installed to all slave-nodes. To do so, we should generate RSA keys (private and public) using ssh-keygen utility with default settings for location pressing ENTER, leave empty password pressing ENTER two times.


or just use following command to do things in silent mode

ssh-keygen -f id_rsa -t rsa -N ""

Following bash code will copy public key generated for master-node to all remote nodes adding the key to the authorized_keys file on each node.

cat /root/hostNames | while read HOSTNAME
   echo "==========================${HOSTNAME}"
   sshpass -f passw.txt ssh-copy-id root@${HOSTNAME}

6. Disable the Firewall

To desable firewall on all Nodes

cat /root/hostNames | while read HOSTNAME
   sshpass -f /root/passw.txt ssh -T root@${HOSTNAME} << EOF
      echo "==========================${HOSTNAME}"
      systemctl disable firewalld
      systemctl stop firewalld

7. Setting SELinux mode

Security-Enhanced Linux (SELinux) should be changed to permissive To check current state of SELinux the command getenforce can be used.
NOTE: Because we set PasswordLess SSH authentication, now we can use simply SSH without need to use SSHPASS.

cat /root/hostNames | while read HOSTNAME
   ssh -T root@${HOSTNAME} << EOF
      echo "==========================${HOSTNAME}"
      sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config
      setenforce 0  #To enforce changes without rebooting 

8. Enable an NTP Service

To Install NTP server on each Node. This time we use PSSF utility to install packages in parallel.

pssh --hosts hostNames -t 1000 --user root -i "yum install -y ntp;"

To add/replace NTP servers in /etc/ntp.conf config file on each Node

cat /root/hostNames | while read HOSTNAME
   ssh -T root@${HOSTNAME} << EOF
      echo "==========================${HOSTNAME}"
      sed -i '/^server/d' /etc/ntp.conf	# To remove existing servers in config file
      echo -e "server 0.pool.ntp.org\nserver 1.pool.ntp.org\nserver 2.pool.ntp.org" >> /etc/ntp.conf

To enable NTP service on each Node

pssh --hosts hostNames -t 1000 --user root -i "systemctl start ntpd; systemctl enable ntpd; hwclock --systohc;"

9. Change max number of open files

use following commands to check appropriate value in your system

ulimit -Sn
ulimit -Hn

To change values

pssh --hosts hostNames -t 1000 --user root -i "echo -e '* soft nofile 10000\n* hard nofile 10000\nroot soft nofile 10000\nroot hard nofile 10000\n' >> /etc/security/limits.conf"

10. Since some components in CDH 6 requires Python 2.7, make sure that all Nodes have it.

Use “python –version” to verify it or following command to check all nodes

pssh --hosts hostNames -t 1000 --user root -i "python --version;"

11. Install JDK

Cloudera recommends to install Oracle’s JDK or OpenJava version 8. Since, Oracle requires authorization for download, what can complicate the download process for many Nodes, we used another publicly available repository (repo.huaweicloud.com).

pssh --hosts hostNames -t 10000 --user root -i "curl -LO https://repo.huaweicloud.com/java/jdk/8u202-b08/jdk-8u202-linux-x64.rpm; rpm -Uvh jdk-8u202-linux-x64.rpm"

Java Home Directory in our case is /usr/java/jdk1.8.0_202-amd64/


1. Cloudera Manager Repository

Download the cloudera-manager.repo file for Centos 7 to the /etc/yum.repos.d/

wget https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/
rpm --import https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/RPM-GPG-KEY-cloudera

2. Configure Local Repository

It is recommended to build Local Repository especially in case of having number of Nodes more than 10 and if Nodes don’t have access to Internet. Local Repository will be set on dedicated Node (repo.cluster.vm) Installing Web server

yum install httpd

systemctl enable httpd

systemctl start httpd

To download the files for the latest Cloudera Manager 6.3 release

mkdir -p /var/www/html/cloudera-repos

wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.3.1/redhat7/ -P /var/www/html/cloudera-repos
wget https://archive.cloudera.com/cm6/6.3.1/allkeys.asc -P /var/www/html/cloudera-repos/cm6/6.3.1/

chmod -R ugo+rX /var/www/html/cloudera-repos/cm6

To download the files for the latest CDH 6.3 (6.3.2) release

wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.2/redhat7/ -P /var/www/html/cloudera-repos
wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.2/redhat7/ -P /var/www/html/cloudera-repos

chmod -R ugo+rX /var/www/html/cloudera-repos/cdh6
chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras6

Apache Accumulo for CDH

wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/redhat/ -P /var/www/html/cloudera-repos

chmod -R ugo+rX /var/www/html/cloudera-repos/accumulo-c5

Configuring Hosts to Use the Internal Repository

cat /root/hostNames | while read HOSTNAME
   ssh -T root@${HOSTNAME} << EOF
      echo "==========================${HOSTNAME}"
      rm -f /etc/yum.repos.d/cloudera-repo.repo
      echo -e "[cloudera-repo]\nname=cloudera-repo\nbaseurl=http://Repository.cluster.vm/cloudera-repos/cm6/6.3.1/redhat7/yum\nenabled=1\ngpgcheck=0\n" >> /etc/yum.repos.d/cloudera-repo.repo

Now we can check the content of Local Repository through Web-browser on following address http://repo.cluster.vm

3. Configuring a Local Parcel Repository

Add or edit the line in section of HTTP server settings file (/etc/httpd/conf/httpd.conf) as following:

AddType application/x-gzip .gz .tgz .parcel
Restart HTTP server
systemctl restart httpd
wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.3.2/parcels/ -P /var/www/html/cloudera-repos
wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.3.2/parcels/ -P /var/www/html/cloudera-repos

chmod -R ugo+rX /var/www/html/cloudera-repos/cdh6
chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras6

Local Repository Address for parcels: http://repo.cluster.vm/cloudera-repos/cdh6/6.3.2/parcels/ Apache Accumulo parcel for CDH

wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/accumulo-c5/parcels/1.7.2/ -P /var/www/html/cloudera-repos

chmod -R ugo+rX /var/www/html/cloudera-repos/accumulo-c5

4. Install Cloudera Manager Server

yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

5. Install and Configure Database for Cloudera Components

Cloudera Manager, CDH and other Components intensively using Database to store and read data. There are several alternative Database Management Systems supported by Cloudera: MariaDB, MySQL, PostgreSQL and Oracle. In this tutorial MariaDB will be used. Install MariaDB

yum install mariadb-server

Configure MariaDB

systemctl stop mariadb

Update/replace /etc/my.cnf config file according to following

transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd

key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.

#In later versions of MariaDB, if you enable the binary log and do not set
#a server_id, MariaDB will not start. The server_id must be unique within
#the replicating group.

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M


# include all files from the config directory
!includedir /etc/my.cnf.d

Enable MariaDB as a service and start it

systemctl enable mariadb
systemctl start mariadb

Set the MariaDB root password and other settings as followings


Enter current password for root (enter for none):
OK, successfully used password, moving on...
Set root password? [Y/n] Y
New password:
Re-enter new password:
Remove anonymous users? [Y/n] Y
Disallow root login remotely? [Y/n] N
Remove test database and access to it [Y/n] Y
Reload privilege tables now? [Y/n] Y
All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.
Thanks for using MariaDB!

Installing the JDBC Driver for MariaDB

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.46.tar.gz

mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46
cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

6. Create Databases for Cloudera Components

Login to MariaDB on console

mysql -u root -p

Create following Databases for different Cloudera software conmonents. For simplicity all database users are assigned with same password

GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '*****';

GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '*****';

GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '*****';

GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '*****';

GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '*****';

GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '*****';

GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '*****';

GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '*****';

GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '*****';

7. Set up the Cloudera Manager Database

To set Database, we use scm_prepare_database.sh script with following parameters: databaseType, databaseName, databaseUser. Default DatabaseName and DatabaseUser for Cloudera Manager are scm and scm.

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

8. Install CDH and Other Software

Start Cloudera Manager Server

systemctl start cloudera-scm-server

To observe the startup process

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

The string INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server is indicates that Cloudera Manager is started. Now we can start Cloudera Manager Admin Console on browser using following address and port http://node1.cluster.vm:7180
Log into Cloudera Manager Admin Console using default credentials:

Username: admin
Password: admin

9. Now we have DONE

Now you can follow to step-by-step instructions at UI of Cloudera Manager Admin Console in order to complete installation.

Posted in Big Data and Analytics | Leave a Comment »

Will Programmers Become Obsolete in the Future?

Posted by Abzetdin Adamov on July 4, 2019

Some “experts” claim that programming as a job will obsolete in coming years. This can be said by people who is far away from the domain or such claims can convince just those who would like this happen. Reality is different. During next, at least, 20 years demand for the traditional programmers will just increase. After that the concept of programming may slowly change, same as it changed in the past when the new paradigms appear like Object-Oriented or Functional programming those dealing with objects and functions as a building blocks accordingly. Its quite hard to predict, but the new concept can come as a transition to paradigm that will deal more with intelligent actors (IA) and entities (IE) as a building blocks. These IAs may be enriched with advanced concepts like Voice-User Interface (VUI), Cloud/Cluster Data Management Interface (CDMI), Artificial Intelligence as a Service (AIaaS), SaaS, etc. There maybe be millions of IAs and IEs available in repositories (open-source and commercial) developed by community and companies, same as millions APPs available today on app markets. Even today, there are some earlier symptoms of this concept in paradigms like agent-oriented programming and SOA. Programmers’ communities will prefer scripting languages supported by open-source community over commercial platforms. Open-source communities and largest vendors will try to set some fundamental standards to make development process more transparent, cross-platform, both front-end/back-end supportive and less dependent on specific coding language…

But what is certain even today, the role of programming in operation of the critical infrastructure will just increase, consequently increasing the cost of potential mistake. Just remind Boeing 737 Max crisis. As a response, new IDEs with with embedded Data manipulation and processing engines, BPM-interfaces and translators, reverse-engineering capabilities and testing modules based on artificial scenario-generators will appear.

Even so, human programmers will never be made obsolete…

Posted in Programming and Development, Web Programming and Design | Tagged: , , , , , | Leave a Comment »

Yahoo – Internet titan in Past, Sinking ship Today

Posted by Abzetdin Adamov on June 11, 2016

Yahoo - Internet titan in Past, Sinking ship Today

Yahoo – Internet titan in Past, Sinking ship Today

Yahoo, which was once an Internet titan, the only ruler of the entire Web, after years of mismanagement and underperformance looks like the Sinking Ship with irreparable hole in its side.

In February 2008 Microsoft made $44.6 billion bid for Yahoo. Later Yahoo announced its board of directors’ letter rejecting Microsoft’s hostile bid for the company as “massively undervalued. Now Yahoo’s board wants to sell its core business for $10 billion, and experts say that this price is slightly higher than its current market price.

Yahoo is still one of the biggest technological patent holders having nearly 3000 patents and pending applications. It is obvious that patents are heritage of the former very successful technological giant. If Yahoo made 399 patent applications in 2007, it filed just 66 new applications in 2013. Yahoo will receive fewer and fewer new patents in the coming years.
Yahoo plans to sell patents most probably initiating new Patent War (similar to Oracle vs. Google).

I think any company which acquire this huge sinking ship will follow to the same destiny. This would be a classic case of two wrongs trying to make a right.

Some References:

Yahoo is unloading 3,000 patents, and it will be a fire sale

Facebook Fights Back, Countersues Yahoo For Patent Infringement

Microsoft’s $44.6 billion bid for Yahoo

Posted in Uncategorized | 1 Comment »

2nd INTERNATIONAL FORUM “BIG DATA DAY BAKU 2016” (BDDB2016) has Successfully Completed its Work

Posted by Abzetdin Adamov on May 20, 2016

Assoc.Prof. Abzetdin Adamov, Chair of BDDB2016 - Opening Speech

2nd International Forum Big Data Day Baku 2016 – Opening Ceremony

The 2nd International Forum Big Data Day Baku 2016 (BDDB2016) was held under the slogan “Transforming Big Data into Big Value” on 17 May 2016 at the Qafqaz University, Baku Azerbaijan. This is highly meaningful that the BDDB2016 event coincides with World Telecommunication and Information Society Day. The Big Data Day Baku 2016 was jointly organized by CeDAWI Research Center (Applied Research Center for Data Analytics and Web Insights) and Barama Innovation and Entrepreneurship Center under sponsorship of the Azercell Telecom, support of the QUTechnopark, technical support of the Institute of Electrical and Electronics Engineers (IEEE) in co-operation with global Big Data vendors IBM, Oracle, Microsoft and local data-driven companies Azercell Telecom and ATL Group. During full day event distinguished experts from global and local Big Data companies and research institutions delivered Keynote Speeches dedicated to the Big Data and Analytics.

The event’s main goal was to increase public awareness of new opportunities and challenges brought by Big Data, share experience with industry and government on the development of state-of-the-art Data Analysis technologies, attract youth to make career and do outstanding research in Data Science.


EVENT’s WEBSITE: www.cedawi.org/BDDB2016

Posted in Uncategorized | Leave a Comment »