Blog

Confluent Kafka Hdfs Sink With Hive Integration

December 13, 2017

Exception : [2017-11-10 08:32:32,183] ERROR Task hdfs-sink-prqt-stndln-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerSinkTask:455) java.lang.RuntimeException: java.util.concurrent.ExecutionException: io.confluent.connect.hdfs.errors.HiveMetaStoreException: Invalid partition for default.srp-oracle-jdbc-stdln-raw-KFK_SRP_HDFS_SINK_TEST: partition=0 at io.confluent.connect.hdfs.DataWriter.write(DataWriter.java:226) at io.confluent.connect.hdfs.HdfsSinkTask.put(HdfsSinkTask.java:103) at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:435) at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:251) at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:180) at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.ExecutionException: io.confluent.connect.hdfs.errors.HiveMetaStoreException: Invalid partition for…
Read more

Solved: Running Hive as ec2-user access denied

December 5, 2017

Exception : [ec2-user@ip-123-45-67-890 ~]$ hive Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: falseException in thread “main” java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: Permission denied: user=ec2-user, access=WRITE, inode=”/user/ec2-user”:hdfs:hadoop:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:320) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1728) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1712) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1695) at org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3896) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:984) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)…
Read more

Install GRADLE in Amazon Linux

December 5, 2017

#!/bin/sh gradle_version=”4.4-rc-5″ #Your Custom Installation Path install_target_path=”/opt/gradle” wget -N https://services.gradle.org/distributions/gradle-${gradle_version}-all.zip mkdir -p ${install_target_path} unzip gradle-${gradle_version}-all.zip mv gradle-${gradle_version} ${install_target_path}/ ln -sfn gradle-${gradle_version} ${install_target_path}/latest chown -R ec2-user:ec2-user ${install_target_path} printf “export GRADLE_HOME=${install_target_path}/latest\nexport PATH=\$PATH:\$GRADLE_HOME/bin” > /etc/profile.d/gradle.sh . /etc/profile.d/gradle.sh # check installation gradle -v

Workflow and Dialog Process Stages and Life Cycle

November 28, 2017

The life cycle of a process describes the state transitions from creation through execution and Termination. A process can have following states: Draft, Ready, Suspended, Locked, and Completed. The events that occur throughout the lifetime of the process cause a transition from one state to another. Workflows The workflow life cycle and State Transition is…
Read more

101 Most Used Dynamics 365 CRM codes

November 23, 2017

Here is a list of most used code snippet used in Dynamics 365 CRM.Remember to add required namespace whenever required while inserting code. Keep the CRM SDK folder ready to take the reference of the assemblies for below namespace. Most frequently used namespaces are given below, [code lang=”php”] using System; using System.Configuration; using System.ServiceModel; //…
Read more

9 Top Tools used by Dynamics 365 CRM Developers

November 13, 2017

Microsoft Dynamics 365 CRM developers use various software tools to make the day-to-day productivity more and more. Also these are the ways by which developers can customize the complex CM platform. Below is a list of tools which are being used by dynamics CRM developers in day-to-day project works. These are listed below. Bonus Tool…
Read more

13 Essential Technologies for CRM Developers

November 10, 2017

Dynamics 365 CRM is a highly adopted platform for managing customer data efficiently used by starting from small-scale, medium scale industry to large-scale business organisations. The demand of consultants are also increasing highly for managing the technical and functional aspects of Dynamics 365 CRM. Although the demand of Technical, Functional and Techno-functional consultants increasing at…
Read more

Installing Maven using Yum on EC2 instance (Amazon Linux)

November 7, 2017

Install Maven : Following are the set of commands need to be executed sequentially to install maven. sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo sudo yum install -y apache-maven mvn –version And you all set to run any “mvn” (maven) command in ec2 instance. Output :

Install/upgrage Java 8 using Yum on EC2 instance (Amazon Linux AMI)

November 7, 2017

Step-1: Install Java Runtime – java 1.8 sudo yum install java-1.8.0 if you need a java compiler and other developer tools: sudo yum install java-1.8.0-openjdk-devel Step-2: If you have multiple versions and one of those default , use the alternatives command as follows and enter the selection number as guided in the terminal. sudo /usr/sbin/alternatives –config…
Read more

11 Top Websites for great Developers

October 16, 2017

Everybody dreams about being a great developer in Dynamics 365 CRM. Here I have compiled a list of websites with excellent resources by which you can be a master in developing the Dynamics CRM applications. Microsoft Dynamics 365 Developer Center This website is the official website of Microsoft to get the information you need to…
Read more

Step By Step : Installing Kafka in Mac

September 12, 2017

Open “Terminal” app from Applications or Command + Space and then type “Terminal” press Enter/Return Key Install Homebrew (Copy / Paste the following command in the Terminal window and press enter) ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)” < /dev/null 2> /dev/null Now install Kafka(Copy / Paste the following command in the Terminal window and press enter)…
Read more

Most Viewed Video Tutorials of Softchief

September 12, 2017

This post presents a list of most viewed video tutorials of soft chief published on YouTube channel official. Lets take a look of th tutorials and learn. This is Free Viewing tutorials. For Complete video tutorial please visit the channel HERE. Please subscribe the YouTube Channel.

Step By Step: Getting Kafka installed in Mac OS X Sierra

September 11, 2017

In this activity we are going to use the beautiful packaging manager tool Homebrew throughout the installation process. This tool make life easier to install and manage the latest version of the software and keep updated. Step 1 : Install Homebrew (as an administrator) $ /usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)” The above command will install the following…
Read more

Problem installing ZooKeeper & Kafka in macOS Sierra

September 11, 2017

Until I could successfully installed Kafka in my macOS Sierra environment, the problem I was facing is bringing the zookeeper up & running. We can start Kafka server only if zookeeper server is up. Downloaded the zookeeper trunk build from git repo and extracted in local When started the zookeeper first time from the extracted build,…
Read more

Step By Step : Build and Run Kafka in Eclipse IDE + [ Scala || Java ] + Gradle

September 10, 2017

As a coder, we are more comfortable with the editor tools (specially Eclipse IDE) for rapid development , build & continuous integration. When first time I was trying to develop some Kafka producer and consumer using Scala, I was wondering if I could setup the same through eclipse to make life easier, however after a…
Read more

REST API Maven Dependency : Jersey + Jetty

September 10, 2017

Hooooh …. finally !!! After devoting ample number of hours with lots of R&Ds, finally able to resolve the long running dependencies between Jersey and Jetty when developing a REST API. I am posting the article because working and running the REST API with TOMCAT or similar server would be easy but might not with…
Read more

Pass parameters to HTML Web resource

September 8, 2017

HTML web resource page can only accept a single custom parameter called data. To pass more than one value in the data parameter, you need to encode the parameters and decode the parameters in our page. The example can be found in the SDK folder that can be downloaded freely. The path is sdk\samplecode\js\webresources\showdataparams.htm First we have…
Read more

How to Open Entity or View using URL

September 6, 2017

We can use URLs to open specific entities and views whenever we requires in our custom applications. The below are the scenarios to open entity forms and views using URLs. NOTE : Use Xrm.Utility.openEntityForm when you open entity forms programmatically within the application by using web resources. Do not use window.open.Outside the application, where pages do not have access…
Read more

How to Open Report by using URL

September 6, 2017

Reports can be viewed by using Dynamics 365 report viewer. But sometimes we need to open a specific report from other applications referring to the report server reports or Dynamics 365 reports. SO for this scenario we can open report using the below methods. We can open a report by passing appropriate parameter values to…
Read more

Open Dialog using URL in Dynamics 365

September 6, 2017

Dialogs are the interactive processes which takes input from user and process the data and gives output. For many scenarios users run dialogs to achieve required task. Dynamics 365 provides a out-of-the-box command button option called as “Run Dialog” which gives a list of activated dialogs created against the specific entity. There is also a…
Read more

Dynamics 365 Customer Engagement Sales App/Module Explained

August 25, 2017

Dynamics 365 Customer Engagement (CE) Sales process is a streamlined business process module to generate potential sales opportunity for the business. The sales module covers starting from Lead generation to invoice and order closure and also involves Goal management in some cases. Watch the video below to learn in practical or scroll down to rad…
Read more

Send Email using C# Code Sample in Dynamics 365

August 24, 2017

Emails are the mostly used communication media in all types of industries. Microsoft Dynamics has a very flexible way to deal with Email activities. Emails are stored as an Activity record in Dynamics 365 CRM. Any business process, which requires to send emails first creates an email activity record in the system then by email router…
Read more

Dynamics 365 CRM Entity Relationships Understandings

August 23, 2017

Entity relationship defines how a specific entity record is related to another entity record. In this post we will discuss what is entity relations and details about this topic. We will learn below points in this article, Fundamentals of entity relationship Relationship V/s Connections Types of Entity Relationships Relationship Behaviors How to create Relationships Fundamentals…
Read more

Create, Retrieve, Update, Delete SharePoint Records using C#

August 11, 2017

Using Microsoft Dynamics 365, you can use the document management capabilities of Microsoft SharePoint from within Dynamics 365. You can store and manage documents in the context of a Dynamics 365 record on a SharePoint Server, and leverage the SharePoint infrastructure to share, manage, and collaborate efficiently. Because the documents are stored on a SharePoint…
Read more

Create Outlook Filters using C# in Dynamics 365 CRM

August 10, 2017

Microsoft Dynamics 365 uses online synchronization filters to determine which records to synchronize between Dynamics 365 and Microsoft Outlook (using Microsoft Dynamics 365 for Outlook), or Exchange (using server-side synchronization). You can modify the existing online synchronization filters or create new filters to synchronize certain types of records. You can also delete, deactivate, or activate…
Read more