Trending March 2024 # Introduction To Hyperledger Fabric In Blockchain Network # Suggested April 2024 # Top 7 Popular

You are reading the article Introduction To Hyperledger Fabric In Blockchain Network updated in March 2024 on the website We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested April 2024 Introduction To Hyperledger Fabric In Blockchain Network


In this article, we will provide an introductory overview of Hyperledger Fabric, a permissioned blockchain framework. We will explore its architecture, components, and functionalities, highlighting how it enables secure, scalable, and customizable blockchain networks for enterprise applications.

This article was published as a part of the Data Science Blogathon.

What is Hyperledger Fabric?

Hyperledger Fabric, initially developed by IBM and Digital Asset, is a permissioned blockchain infrastructure. It offers a modular architecture that defines distinct roles for nodes within the infrastructure. It executes Smart Contracts (chaincode) and provides configurable consensus and membership services. Its modular architecture ensures high levels of confidentiality, resiliency, flexibility, and scalability. The platform supports pluggable implementations of various components, accommodating the complexities of different economic ecosystems. While Fabric’s permissioned nature may seem restrictive, its modularity allows for plug-and-play components, including the consensus mechanism, enabling organizations to choose the most suitable components for their applications. This aspect has driven its adoption across finance, healthcare, and supply chain management industries.

Blockchain for business refers to applying blockchain technology in commercial and organizational contexts. It involves leveraging blockchain’s decentralized, transparent, and immutable nature to streamline business processes, enhance trust, and enable secure and efficient transactions. Blockchain for business offers several potential benefits, including increased transparency, reduced intermediaries, improved security, enhanced traceability, and cost savings. It can be used in various industries such as supply chain management, finance, healthcare, and logistics, revolutionizing traditional business models and unlocking new opportunities for innovation and collaboration. By utilizing blockchain technology, businesses can create more efficient and trustworthy systems that foster trust, accountability, and seamless stakeholder interactions.

Advantages of Hyperledger Fabric

Finally, Fabric is very secure. The Fabric uses cryptographic techniques to ensure that data is not tampered with. It makes Fabric an ideal platform for applications that require a high degree of security.

Many different frameworks and tools make up the Hyperledger project. Here is a brief overview of some of the most important ones:

Hyperledger Fabric: This is the core framework that other tools and applications are built on top of. It is a permissioned blockchain platform that supports pluggable components for consensus, identity, and smart contract execution.

Hyperledger Iroha: It is a simple yet powerful, permissioned blockchain platform designed for mobile and IoT applications.

Hyperledger Sawtooth: This modular blockchain platform supports dynamic consensus algorithms and allows for fine-grained permissions of users and applications.

Hyperledger Composer: This is a tool for quickly developing blockchain applications on top of Hyperledger Fabric. It provides a high-level programming model and an extensive library of reusable components.

Hyperledger Caliper: Hyperledger Caliper is an open-source performance benchmarking tool for blockchain technologies developed by the Hyperledger project. It allows users to measure the performance of a specific blockchain implementation with a set of predefined use cases.

The tool is designed to be extensible and to support multiple plug-ins so that it can be easily adapted to different blockchain implementations.

The Hyperledger Fabric platform has five major components:

Certificate Authority

Peer nodes that make up the network

Ordering service that maintains the global ordering of transactions

Private channel

Membership service that manages the identities of the participants (Chaincode)

Fabric CA (Certificate Authority)

A Certificate Authority (CA) is a trusted third party that issues digital certificates. A digital certificate is an electronic document used to verify the identity of a person or entity. It contains the public key of the person or entity, information about the issuer, the CA’s digital signature, and the certificate’s expiration date.

The CA is responsible for verifying the identity of the person or entity before issuing a certificate. They also manage the revocation process, which is used to invalidate a certificate if it is compromised or no longer needed.

The CA is a critical component of a Hyperledger Fabric network. They are responsible for issuing digital certificates to the various components of the network, such as the orderer nodes, peer nodes, and user identities. These certificates are used to authenticate the components and establish trust between them.

With a CA, it is easier to establish trust between the components of a Hyperledger Fabric network.


It must cryptographically sign every operation executed inside HyperLedger Fabric with this certificate.

You can add attributes, roles

Certificates are X.509 standards.

You can remove the necessity of certificates if you don’t need them.

Chaincodes read this data and make business decisions.

Source – Google

Role of Peer Nodes in Hyperledger Fabric

Peer nodes are the heart of any distributed ledger or blockchain network. They are responsible for validating and relaying transactions and maintaining the ledger state. In the Hyperledger Fabric blockchain platform, peer nodes also execute smart contracts, called chaincode, on behalf of clients.

The role of peer nodes is thus critical to properly functioning a blockchain network. Without peer nodes, there would be no distributed ledger and no blockchain.

There are two types of peer nodes in Hyperledger Fabric: endorsing peers and committing peers. Endorsing peers are responsible for validating transactions and executing chaincode. Committing peers are responsible for writing transactions to the ledger and maintaining the ledger state.

Both peer nodes are essential for properly functioning the Hyperledger Fabric platform. Endorsing peers ensures that transactions are valid before committing to the ledger. Committing peers to ensure that the ledger state is accurate and up-to-date.

Role of Ordering Service in Hyperledger Fabric

In a Hyperledger Fabric network, the ordering service is responsible for creating and maintaining a sequence of blocks, which are then delivered to the appropriate peers in the network. A single entity can run the ordering service or be distributed among multiple entities.

The ordering service is a critical component of a Hyperledger Fabric network, ensuring that all transactions are processed consistently. Ensuring the ledger’s accuracy and maintaining the transactions’ confidentiality is essential.

The ordering service is also responsible for creating new blocks on the ledger. It is done by selecting the transactions that should be included in the next block and packaging them into a block. The ordering service then signs the block and broadcasts it to the other peers in the network.

The ordering service is a crucial part of the Hyperledger Fabric network and plays a vital role in ensuring the accuracy and confidentiality of transactions.

Role of Channel in Hyperledger Fabric

The channel in Hyperledger Fabric is a private “subnet” of communication between two or more specific network members to conduct private and confidential transactions.

The channel allows members to exchange transactions without interacting with the rest of the network. It allows for a high degree of privacy and confidentiality and improved performance.

The channel is also the mechanism that allows for the creation of “smart contracts”, or chaincode, which can be used to facilitate transactions between members.

To create a channel, members must first join the network. Once they have entered, they can then create a channel and invite other members to join it.

Once a channel has been created, members can exchange transactions among themselves. These transactions will not be visible to the rest of the network.

To ensure privacy and confidentiality, members of a channel must use digital signatures when exchanging transactions.

Chaincode in Hyperledger Fabric

In Hyperledger Fabric, chaincode is a program written in Go, Java, or chúng tôi and deployed on a network of Hyperledger Fabric peers. Chaincode is used to manage the ledger state, which includes the assets and transactions on the ledger.

Invocations from applications trigger Chaincode. When an application invokes a chaincode function, the chaincode will read and write ledger state variables. The chaincode can also gather other chaincodes. For example, chaincode A might invoke chaincode B to read a value set by chaincode B.

Chaincode has access to a set of built-in functions that allow it to query and update the ledger state. For example, chaincode can use the GetState() function to read a value from the ledger state and the PutState() function to write a deal to the ledger state.

What is Hyperledger Composer?

Hyperledger Composer is a tool for rapidly building blockchain business networks.

It is an open-source development toolset that makes it easy to deploy and manage blockchain applications on the Hyperledger Fabric platform.

Composer provides a modeling language for describing the structure of a business network and a set of comprehensive tools for developing, deploying, and administering the business network.

The toolset includes a modeling language, a development environment, a user interface, and a set of command-line tools.

Composer is designed to make it easy to develop blockchain applications that can be deployed on the Hyperledger Fabric platform.

The toolset includes a modeling language that makes it easy to describe the structure of a business network.

The modeling language is based on the Unified Modeling Language (UML) and uses a graphical notation to represent the business network.


In conclusion, we have provided an introductory overview of Hyperledger Fabric, exploring its architecture, components, and functionalities. We have also discussed key differences between Hyperledger Fabric and other well-known blockchain protocols such as Bitcoin and Ethereum. While Bitcoin operates as a public blockchain, Hyperledger Fabric is a private blockchain with restricted access. Additionally, the consensus algorithms and transaction privacy mechanisms differ between the two. Hyperledger Fabric’s permissioned nature ensures participant identification and verification, contributing to a more controlled and secure network. Understanding these distinctions is crucial for selecting the most suitable blockchain solution for specific enterprise requirements.

Key Takeaways

Then, we have seen various frameworks that use HyperLedger Technology.

After that, we discussed all the components of a HyperLedger Fabric, which includes Certificate Authority, Peer Nodes, Ordering Service, Channel, etc.

Finally, we have concluded the article by discussing the key differences between the Bitcoin and HyperLedger Fabric.

Frequently Asked Questions


You're reading Introduction To Hyperledger Fabric In Blockchain Network

An Introduction To Mobile Seo

Mobile is to SEO what glaze is to Krispy Kreme. You can’t have one without the other. It is the backbone of Google’s index.

Sure, the mobile-first index just rolled out in 2023, but Google has been dropping not-so-little hints for the past few years.

In 2024, Google announced the mobile searches surpassed desktop. Then in 2024, mobilegeddon 2.0 rocked the SEO world. And, in 2023, Google introduced us to the mobile-first index.

But the question still remains:

What should my mobile strategy be?

It isn’t enough to have a mobile-friendly site.

This post will tell you all you need to know to get started with mobile SEO.

Step into your future with the basics of mobile SEO.

How Google Deals with Mobile Search

If it isn’t obvious yet, Google clearly favors mobile search.

But it can be quite confusing understanding how Google deals with mobile search.

So, here’s the lowdown on some common FAQs about mobile search and Google.

What URL Does Google Index If You Have Separate Mobile and Desktop URLs?

Google will display the desktop URL for the desktop searches.

And, the mobile searches will get the mobile URLs.

But, the indexed content (the big chalupa that determines how your rank) will be from mobile.

URLs in search: With Mobile-first indexing, we index the mobile version. When we recognize separate mobile URLs, we’ll show the mobile URL to mobile users, and the desktop URL to desktop users – the indexed content will be the mobile version in both cases.

— Google Webmasters (@googlewmc) June 14, 2023

Will I Lose My Ranking Positions with the Mobile-First Index?

But, mobile-friendliness is a ranking so your UX is still important.

Mobile-friendliness is reviewed page-by-page, which means you’ll want to update your money pages first.

On ranking: The mobile-first index doesn’t change anything for ranking other than that the mobile content is used. While mobile-friendliness is a ranking factor on mobile, being in the mobile-first index is not.

— Google Webmasters (@googlewmc) June 14, 2023

Allow me to let you in on a little secret:

Google wants both your desktop and mobile site to have the same content.

If you have the same content (like a responsive design), you will see no impact from the mobile-first index.

Plus, on the bright side, Google sends notifications to let webmasters know the mobile-first indexing is going down on your site.

Is Your Site Mobile-Friendly?

To help you find out if your site is mobile-friendly, here are some of my favorite tools.

Best Practices for Mobile SEO

Let’s break down how to optimize your site for mobile search.

We’ll start by exploring a few mobile SEO best practices and techniques that apply to all mobile sites.

Mobile Content

To sum up mobile SEO, you want the same exact content from your desktop on your mobile site.

All content formats (text, videos, images, etc.) should be crawlable and indexable in mobile.

Google updated their app and mobile results to display badges for image search.

This means those image alt attributes that you’ve been ignoring are becoming even more relevant in mobile search.

I mean, if Google can already recognize dog and cat breeds through photos, can you imagine what’s next?

Also, with the rise of voice search, you may want to consider aligning your content.

For example, I would recommend optimizing your meta titles for mobile search because they are shorter.

Remember, voice search is performed from a mobile device, so it makes sense to optimize your mobile site.

Voice search = mobile device.

This means redefining the way marketers perform keyword research.

Long-form queries and questions are dominating the SERPs, which is why things like featured snippets are having a major impact.

It’s about user intent now.

Mobile Site Performance

To quote Top Gun, “I feel the need for speed.”

Yes, Google is feeling the need for speed as the official mobile “Speed Update” hit the scene.

This is why Google introduced the Accelerated Mobile Pages Project to improve site speed and page load times for mobile content.

AMP allows content to be cached and served directly within a SERP (rather than sending the user to the original website).

This is also why the industry will start to see AMP pages integrate with PWAs.

I would recommend using responsive design as well as AMP pages.

For example, using AMP pages to serve your blog posts and services pages if you’re an SEO agency may be something to consider.

And, if you want to get really deep into page speed, listen to this podcast with Bastian Grimm and Brent Csutoras as they discuss paint timings.

Making Your Website Mobile-Friendly

There are three main approaches to making your website mobile-friendly:

Responsive Design

Adaptive Website

Separate Mobile Site

Here’s how to optimize each.

1. Optimizing Responsive Design for Mobile Search

There’s a mistaken belief that if your site is responsive then it’s automatically mobile-friendly.


Let me explain.

Responsive design maintains the same website content from desktop to mobile. It takes the same URLs, HTML, images, etc.

However, responsive design does not mean that the needs of your mobile visitors are met.

Responsive design still needs to be optimized for user experience.

With that said, Google has stated that responsive design is their preferred option for mobile SEO, but has confirmed there is no ranking boost to gain from having a responsive site.

And, based on a study by Appticles, published in Smashing Magazine, that responsive websites are the most common mobile-friendly site holding it down at 51.11 percent.

Here is what you need to know about optimizing your responsive design for mobile SEO:

Scale Images

Images already scale with responsive design. But, they may not be necessary for your mobile users. I’ll show you an example.

Here’s Navy Federal Credit Union’s homepage on desktop and mobile.

Now, here’s Bank of America’s homepage on desktop and mobile.

Bank of America, right?

Navy Federal’s desktop top image takes over the mobile website with no call-to-action. On the other hand, Bank of America’s CTA’s are front and center.

Key takeaway: Scale images for mobile users if you’re using responsive design. Ask your developer to create alternate images for different viewports. Don’t forget to update the meta name = “viewport.”

Clean Navigation

I would recommend monitoring your mobile user behavior to understand what they are searching for on your site.

Then, tailor your navigation to their needs.

For example, Best Buy keeps their navigation at the top with their main pages along with a hamburger menu in the center.

Side bar: Google has confirmed that hamburger menus are “fine” to use in mobile design.

Key takeaway: Keep your mobile navigation simple and clean. Limit the main pages to 4-8. If you  need more, utilize a hamburger menu.

Delete Mobile Pop-Ups

Google wants your mobile users to find what they need and fast.

To help their mobile users, Google introduced a new signal in 2023 that stated:

“Pages where content is not easily accessible to a user on the transition from the mobile search results may not rank as high.”

That’s not to say all pop-ups are bad.

Here’s an example of a bad pop-up:

The newsletter takes up the entire screen without letting users read the content behind it.

Here’s an example of a good pop-up:

The image does not take up the full screen and the visitors can still see the content.

Key takeaway: Proceed with caution when it comes to pop-ups. There is proof that pop-ups do work. In fact, Sumo saw a 9.28 percent conversion rate for their pop-ups. Just tread lightly.

Shorten Copy

Desktop copy does not often translate well to mobile copy.

Lots of text on a mobile site can crowd and overwhelm mobile users.

I like to keep things simple by reducing text. Let me show you.

Now, here’s their mobile site:

The copy is reduced above the fold to keep the CTA clear and concise.

They pushed the longer form copy down for users to learn more if they scroll.

Key takeaway: Less is more. Keep conversions high by reducing the amount of copy above the fold. Entice users to scroll with the intitial text, then give them the full monty after scrolling.

Design CTAs

iAcquire and SurveyMonkey discovered that 70 percent of mobile searches lead to action on websites within one hour.

But, mobile conversions are lower than desktop.


The call-to-action is not clear.

Here’s what I mean:

Look at’s mobile site:

They require the user to scroll to see the CTA button.

It’s likely they are losing out on mobile conversions.

Now, here is an example from Flywheel.

You can see that they want users to “Sign up for free” to use their product.

2. Optimizing an Adaptive Website for Mobile Search

An adaptive (or RESS/dynamically served) site uses the same URL, but the server sends a different version of the HTML (and CSS) based on what type of device is requesting the page.

You essentially have three different versions of your website:




Amazon is a great example of adaptive web design:

So, why did Amazon choose to use an adaptive web design over responsive?

Mobile Marketer reported that Amazon chose adaptive design to increase page speeds on mobile 40 percent.

If you’re a small business, I’d recommend going with the popular vote of a responsive design.

Adaptive websites require more resources.

Here is what you need to know about optimizing your adaptive website for mobile SEO:


Google will devalue your site if you’re showing one thing to the search engine and something different to the user.

This is cloaking.

To fix this issue, ask your host to use the Vary-HTTP Header.

This will guide the mobile crawler to the separate mobile content and signal to the server what type of device the user is coming from.

Customize Design

With adaptive design, developers have full control over the layout, display, and graphics.

If you’re website is tailored to multiple countries, then you may want to swap out the design elements based on region.

The downside to this is that you’ll have to manually update each version of the site.

For example, you can serve custom meta titles and meta descriptions that target mobile users.

Combine Adaptive with Responsive

There is an alternative before going knee deep in adaptive.

You can utilize responsive design with adaptive logic.

Developers can customize for mobile users using client-side JavaScript.

3. Optimizing a Separate Mobile Website for Mobile Search

A separate mobile site (or m-dot) is a completely different site.

The same basic SEO principles remain the same for your desktop, tablet, and mobile, but there are a few differences.

Here is what you need to know about optimizing your separate mobile website for mobile SEO:

Separate URLs

Each desktop URL should serve a mobile URL.

You will need to add the canonical URL rel=”canonical” tag to the mobile site pointing to the desktop URL.  Like this:

<link rel=”alternate” media=”only screen and (max-width: 640px)”

This can also be done in the sitemaps.

Implement Mobile Switchboard Tags

Switchboard tags are similar to canonical tags, they tell Google a mobile URL exists with a rel=alternate tag.

Without switchboard tags, Google may not crawl the mobile versions of your URL.

You will need to add the rel=”alternate” tag to the desktop URL that points to the mobile URL. Like this:

This can also be done in the sitemaps.

Detect User-agent Strings

Double check your redirects to make sure that your desktop URLs  coordinate to the correct mobile URL when redirecting.

Otherwise, you could create a faulty redirect (not good).

Luckily, Google Search Console will detect these faulty redirects for you.

Search Console Verification

Make sure you verify the mobile version of your website in Google Search Console.

Structured Data

Always include the same structured data on your mobile and desktop sites.

Your URLs in the structured data on mobile pages should be the mobile URL.


If you’re a global company using rel=hreflang, make sure your mobile URLs with the hreflang point to the mobile version of your country.

XML Sitemaps & Robots.txt

All links in the sitemaps should be available from the mobile version of the site.

This includes your chúng tôi file, too.


For all late nights cursing my laptop and stress eating that mobile SEO has caused me over the past few years, I’ll be the first to admit that the mobile-first index felt pretty blah.

The majority of the sites I work on are already responsive.

But, if you live for your separate mobile site, I won’t stop gushing about how important a uniform URL structure can be.

The end goal remains the same:

Allow the crawlers to access, read, and digest your content.

Optimize the UX for all devices.

Continue testing for better results.

Mobile search is no longer the future of SEO.

It’s here.

Do you have what it takes to make it out alive?

Image Credits

Featured Image: Paulo Bobita

Screenshots taken by author, September 2023

Http Request In Power Automate – An Introduction

Despite having a variety of connectors that we can choose from, Power Automate also has its limitations. It can’t cover everything we might need in order to build our workflows. There might be an instance when we want to integrate or trigger our flow using an application that’s not available in Power Automate.

The HTTP and Request connectors allow our flow to interact with third party APIs.

We’ve used Request to trigger a workflow in one of our previous tutorials. In that example, we had a third party application that triggered a Power Automate flow with approvals and conditional logic.

And that’s how we can basically connect a third party application to our flows. 

On the other hand, the HTTP connector allows us to ping any third party API that we have. So it’s not necessarily used as a trigger. It’s mostly used as an action.

Let’s say we have a flow where we have to get some information from an API. We send the customer’s ID via that API and we get back the customer’s name or passport number. Then we need that information in our Power Automate logic.

But we don’t want to keep that sensitive information within Power Automate. Therefore, we need to create an API in our third party system that can take in HTTP requests. And that’s what we’re going to do as an example.

I currently don’t have a third party API. So for this particular example, we’ll be using a website called chúng tôi It has a third party API endpoint that we can use in order to test if our flow is working or not. For example, they have a function called LIST USERS that contains a corresponding request. And if we request it, the items within the Response column is everything that we can get back.

Then, copy the URL. This serves as the API endpoint.

Then paste the URL that we have previously copied.

We can also enter headers, queries, and cookies if we want to. However, we’ll be skipping that for now.

Search and select the Slack connector.

Let’s post the contents to the random channel.

For the Message text, let’s choose the Body and see what we find.

Let’s rename our flow to HTTP Example.

Add a new step and choose the Data Operation connector.

Then choose Parse JSON.

Move the Parse JSON step in between the HTTP request and the Post message action.

Copy the codes from the request link.

As we noticed, it automatically created the schema that we can easily understand.

For the Content field, we need to put in whatever we want to parse. In this example, we want to parse the Body that we get from the HTTP request.

And why is this important? Well, we don’t have to post the whole body of the message anymore. Now, we have access to more dynamic contents such as email, first name, last name, avatar, company, URL, and many more.

Parse JSON allows us to take any JSON output we get, parse it into different dynamic content that we can then use later on in our subsequent flow steps. We used this action so that our flow won’t Slack the whole JSON content, and only displays the relevant information that we actually need.

Now, instead of Body, let’s change this to first_name.

Then, we’ll see that it changed our action into Apply to each.

This is because our flow receives multiple first names from the request.

As we can see, it only displays the first names now.

All things considered, we’re able to create an HTTP request that integrates our flow to a third party application. From there, we parsed the JSON content using Data Operation connector in Power Automate. It automatically generated a schema using a sample JSON payload. By parsing the JSON, we transformed a typical response from an HTTP request into a more relevant and understandable piece of information.

We can definitely do tons of different things with the HTTP request and Parse JSON actions. Hopefully, you were able to understand their importance and how they work. 

All the best,


An Introduction To Using Zenmap On Linux

Zenmap is a cross-platform application which is available for Linux, Windows and OS X. Other than any Linux specific information, like the installation process, this tutorial applies equally to all of the supported platforms. Talking of the installation process, you can install it on Ubuntu using the Ubuntu Software Center (just search for “zenmap”) or from the command line using:


apt-get install


The above command also works on the Raspberry Pi and probably most other Debian or Ubuntu derived distributions. For other distros that use yum, like Fedora, then use:



"yum install nmap-frontend"

Although Zenmap can be launched via the desktop, it is however best to start it via the command line with root privileges, otherwise Zenmap can’t use some of nmap's functionality.

To start it on Ubuntu run:



There are two main ways to start nmap scan using Zenmap, either by entering a target address and selecting a scan type from the “Profile” drop-down list or by entering the command directly in the “Command” field. If you are familiar with nmap or you want to try out some of the commands from the previous articles, you can use the “Command” field directly.

The power of Zenmap is that it stores and sorts all the information from any scans performed and allows you to build up a picture of your network. The easiest thing to do is a Ping scan to see what devices are alive on your network. In the “Target” field enter and select “Ping scan” from the Profile list. If you are using a different network range from 192.168.1.x then I will assume from here on that you know how to enter the correct range. For more details, please see the previous parts of this series.

Down the left side of the window, you will see a list of the devices (hosts) found on your network and on the right, the output from the nmap command. Above the output pane is a set of tabs: “Nmap Output”, “Ports/Hosts”, “Topology”, “Host Details” and “Scans”. Each one of these tabs shows more information about your network and the information presented is accumulative. This means the more scans you do, the more information is available.

Run an Intense scan against to discover all the open ports and operating system on each host. After the scan, the OS icons will change in the hosts list on the left and the Ports/Hosts tab plus the “Host Details” tab will offer more information about each host.

Each circle on the diagram represents a host found on the network. If a host has less than three open ports, it will be green; more than three but less than six open ports, yellow; and more than six open ports, red. Hosts with filtered ports will have a yellow padlock symbol next to them.


As a further exercise try using some of the scans listed in the first two parts of this series by entering them directly into the “Command” field. Also if you want to permanently add these to the “Profile” drop-down list then use the built-in profile editor (under the Profile menu). The profile editor is also a good way to experiment with other scan parameters since the editor itself presents many of the nmap options as part of its user interface.

Gary Sims

Gary has been a technical writer, author and blogger since 2003. He is an expert in open source systems (including Linux), system administration, system security and networking protocols. He also knows several programming languages, as he was previously a software engineer for 10 years. He has a Bachelor of Science in business information systems from a UK University.

Subscribe to our newsletter!

Our latest tutorials delivered straight to your inbox

Sign up for all newsletters.

By signing up, you agree to our Privacy Policy and European users agree to the data transfer policy. We will not share your data and you can unsubscribe at any time.

Introduction To Azure Data Lake Storage Gen2

This article was published as a part of the Data Science Blogathon.

Introduction ADLS Gen2

The ADLS Gen2 service is built structured, semi-structured, and unstructured data in their original file formats. For example, it can store Text files, CSV files, JSON files, XML files, images, videos, etc. When the uploading of files gets completed after that we can use any technologies like Databricks, or Hadoop, to process and analyze the data as per our business needs.

Data Lake Storage Gen2 makes Azure Storage the inspiration for building enterprise knowledge lakes on Azure Cloud. it’s designed to service multiple petabytes of data while sustaining many gigabits of turnout, Data Lake Storage Gen2 helps you to simply manage huge amounts of data.

A fundamental part of Data Lake Storage Gen2 is the addition of a hierarchical namespace to Blob storage. The hierarchical namespace organizes objects/files into a hierarchy of directories for efficient data access.

In this article, we will explore Azure Data Lake Storage Gen2 (ADLS Gen2) service. We are going to c orage containers and upload the data inside the containers from our local system.

Steps to Create ADLS Gen2 Storage Account


Need to have at least an Azure free tier subscription. I will use a free tier subscription to perform the below steps.

Step 2 – Now we will create the storage service.

Step 2.2 – Type “storage account” and select the only option “storage account” to create this service.

Step 2.4 – Provide the following information in the “Basics” tab of the “Create a storage account” page –

“Project Details” Section –

Subscription – Select the proper “Subscription” to use from the Dropdown.

Resource Group – Select “RG-Storage” from the Dropdown of “Resource Group”.

          “Instance Details” Section –

Storage account name – Type the name “blogdemostg ” in the provided input box.

Region – Select the default option “East US” from the Dropdown of “Region”. If you want you can change it as per your choice.

Performance – Select the option “Standard: Recommended for most scenarios (general-purpose v2 account)”.

Redundancy – Select the “Locally-redundant storage(LRS)” Option in the Dropdown.

Step 4 – Finally our service gets launched and we can see all the credentials that we have defined during the creation of the ADLS Gen2 storage account.

Uploading Data in ADLS Gen2

We have successfully created our first ADLS Gen2 storage account. Now, we will upload the data inside it using Microsoft Azure Storage Explorer. To upload the data we are going to create a folder with the name “raw”. Inside this folder, we are going to upload our data. Let’s go…

Steps to upload the Data in ADLS Gen2 Storage Accounts

Step 2 – Type the name of your folder inside the provided box as “raw”.

Step 5 – Now your file gets uploaded into the Azure storage. You can check the “Activities” section which will display the status of your task, and whether your task is successful or gets failed.

This article has covered the following topics:

Details about ADLS Gen2 Storge Service.

Steps to create your fi

Connecting your Azure account with Microsoft Azure Storage Explorer.

Creating containers in your ADLS Gen2 storage account with the help of Microsoft Azure Storage Explorer.

Uploading the data files and folders inside your containers from your local system.

Happy Learning!

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion. 


A Friendly Introduction To Knime Analytics Platform


In recent years, data science has become omnipresent in our daily lives, causing many data analysis tools to sprout and evolve for the data scientist Joe to use. Python, R, or KNIME Analytics Platform, are some of the most common tools. The innovative character of the KNIME Analytics Platform consists of its visual programming environment with an intuitive interface that embraces a wide variety of technologies.

In this blog post, we would like to give you a friendly introduction to the KNIME Analytics Platform, by showing you the user interface, explaining the most crucial functionalities, and demonstrating how to create a codeless data science process. In order to do that, we will use a specific example of implementing a workflow for customer segmentation based on the k-Means clustering procedure.

General Concepts: Nodes, Workflows, & Components

KNIME Analytics Platform is a free, open-source software for the entire data science life cycle. KNIME’s visual programming environment provides the tools to not only access, transform, and clean data but also train algorithms, perform deep learning, create interactive visualizations, and more.

The KNIME Analytics Platform user interface also referred to as workbench, is typically organized as shown in Fig. 1.

Figure 1. Overview of the KNIME workbench.

Nodes perform tasks in your data science process.

When you are assembling a visual workflow, you will be using “nodes”. A node is displayed as a colored box and performs an individual task. A collection of interconnected nodes is your assembled workflow and represents some part – or all – of your data analysis project.

Each node can perform all kinds of tasks, e.g., reading and writing files, transforming data, training models, or creating visualizations. All the different types of nodes are found in the Node Repository (in the lower-left corner). The data are routed through the node via input and output ports. A node can have data as input or output as well as other objects, such as connections or machine learning models, SQL queries, or data properties. Each object inputs or outputs the node via a dedicated port. Only ports of the same type can be connected. Nodes are color-coded, according to their category, e.g., all yellow nodes are for data wrangling. Depending on their task, nodes also have specific settings, which can be adjusted in their configuration dialog.

A simple traffic light system underneath each node shows you whether the node is already configured, executed, or whether an error has occurred.

Figure 2. The different states of a node

Workflows are assembled with nodes, metanodes, and components.

A workflow in KNIME Analytics Platform consists of several, combined nodes. The data flows through the workflow from left to right through the node connections.

You can use a so-called annotation – a colored frame that can be placed freely inside a workflow – to document the steps in your workflow.

Figure 3. A simple workflow performing customer segmentation through a k-Means clustering procedure. The workflow’s task as well as each step- read the data – preprocessing – apply k-Means – visualization – is documented inside annotation boxes.

You can also identify isolated blocks of logical operations in your workflows and include these nodes into so-called metanodes or components. Components are like metanodes but instead of just grouping some nodes for the sake of transparency, they encapsulate and abstract the functionalities of the logical block. Components serve a similar purpose as nodes, whereas a metanode is more of a visual appearance improvement.

Enrich functionality with extensions and integrations and utilize resources

Aside from all the above-mentioned functions, the KNIME Analytics Platform has two further important elements – extensions and integrations. The variety of extensions and integrations provide additional functionalities to the KNIME core functions. For example, the KNIME Deep Learning – Keras Integration or the Text Processing extension are only two of many exciting possibilities.

Finally, just a few words on where to get help and find resources.

The KNIME Hub is a public repository, where you can find vast amounts of nodes, components, workflows, and extensions and provides a space to collaborate with other KNIME users. On KNIME Hub, you can find example workflows and pre-packaged components to get started.

The KNIME Community Forum serves as an environment to exchange experiences with other KNIME users, to seek help, or to offer your skills to help others.

If you need help getting started, on our Learning webpages you will find additional courses and material.

The Example: Customer Segmentation

Let’s now put our knowledge into practice and assemble a visual workflow in which a k-Means clustering is applied to segment customer data.

The dataset we use for this example can be downloaded from Kaggle and contains some basic customer data: “Customer ID”, “Gender”, “Age”, “Annual Income”, and “Spending Score”.

There are many ways to perform customer segmentation. Most of them include some previous knowledge on at least one of the input attributes. When nothing is known, in terms of knowledge or goals, customer segmentation is often performed via a clustering technique.

In general, clustering is used to detect underlying patterns in the data. Similar traits – or data points – are grouped together based on similarity and assigned into clusters. Amongst all clustering techniques, k-Means is a very simple one, yet effective enough.

a) Read the Dataset with CSV Reader

First, you need to read your data into the workflow. KNIME Analytics Platform offers a variety of data reading options for different file types, for example, Excel files with the Excel Reader node, text files with the File Reader node, or CSV files with the CSV Reader node. It is also possible to connect to and read from a database using the dedicated DB nodes. To read your data into KNIME Analytics Platform there are two different options.

You can drag and drop your file into the workflow. In this case, if it is a known file extension, the KNIME Analytics Platform automatically creates the correct reader node and automatically feeds the file location into the node configuration settings.

Note that those are standard operations.

Figure 4. Searching the CSV Reader node via Node Repository

Figure 5. The configuration window of the CSV Reader node.

b) Apply Normalization to Attributes with Normalizer Node

The k-Means algorithm requires normalized numerical attributes. In general, normalization is necessary, when the attributes are in two different non-comparable units (e.g., cm and kg) or the variance between attributes is quite different. By normalizing the values, we make sure that one input feature does not dominate the other features in the distance or variance calculation. Additionally, nominal attributes should be discretized before feeding them into the algorithm, otherwise, it cannot handle them properly. Discretize nominal variables using the Category to Number node.

Our dataset consists of five attributes, of which one is a categorical variable (“Gender”), and the other four are numerical variables. Usually, the best practice is to use all input attributes, but in this case, we decided to limit ourselves to just two: Annual Income and Spending Score. We did this to obtain a clearer visualization of the results in a 2D scatter plot and thus an easier comparison of different k-Means runs.

In the configuration window of the Normalizer node (Fig. 6), you need to select the attributes for which normalization is required. We decided to use a min-max normalization to simply transform the values into a [0,1] interval, where the smallest value is transferred to 0 and the largest value to 1. But of course, there are other options, like a z-score normalization (suitable when having many outliers) or normalization by decimal scaling.

Figure 6. The configuration window of the Normalizer node. 

We then configure the Normalizer node appropriately (Fig. 6).

Nodes can either be executed step by step to examine intermediate results or the entire workflow at once. The two green arrowed buttons in the toolbar are responsible for these two types of execution.

c) Apply k-Means Algorithm and Configure Cluster Number

Now, connect the normalized dataset to the k-Means node and configure the number of clusters k. The correct choice of a number of clusters cannot be known beforehand. Usually, a few numbers clusters are tried and then the final cluster sets are visually compared and evaluated. Alternatively, the quality of the different cluster sets can be measured and compared, for example via the Silhouette Coefficient. The optimal k could also be obtained by running an optimization method, like the elbow method, the Silhouette optimization, or the Gap Statistics.

Let’s start with k=3. In the node configuration window of the k-Means node (Fig. 7), we can decide whether to initialize the algorithm with the first k rows or with k random data points of the dataset. Also, we can include or exclude attributes for the distance calculation.

You might now wonder why there is no option for the distance measure. In this case, let me point out that the k-Means node uses the Euclidean distance by default. Notice that the Euclidean distance only applies to numerical attributes and therefore only numerical columns are available to move from the Include to the Exclude panel and vice versa. We use the two normalized input attributes Annual Income and Spending Score. Would we obtain the same results when adding attribute Age? You can try…

The configuration window also includes an emergency stop criterion, to avoid infinite running without convergence to a stable solution. This is the setting named Max. Number of Iterations.

Figure 7. The configuration window of the k-Means node.

After executing this node successfully, it outputs the k cluster centers for the k=3 clusters (Fig. 8). You can try to run the algorithm again with a different number of clusters and see if and how the cluster centers change.

Figure 8. The three cluster centers.

d) Determine Clustering Quality with Silhouette Coefficient

Now we successfully identified three clusters. But how good is our clustering? More precisely, how good is the choice of our k? The quality of the cluster set can be measured via the Silhouette Coefficient as calculated via the Silhouette Coefficient node.

As in the k-Means node, an Include/Exclude panel allows to include attributes for the calculation of the Silhouette Coefficient. In addition, here, the cluster column must be selected, which in our case comes from the previous k-Means node, is called “Cluster”, and contains a string value indicating the cluster affiliation.

e) Assign Color to Clusters with Visualization Nodes, Color Manager & Scatter Plot

The last step refers to visualizing the obtained clusters. To do so, we use the Color Manager node to assign a specific color to each cluster (Fig. 9) and after that the Scatter Plot node to visualize the resulting cluster set (Fig. 10). Indeed, a visual inspection can help us evaluate the quality of the cluster set.

Figure 9. The configuration window of the Color Manager node.

In Fig. 10, you see the configuration window for the Scatter Plot node and its most important setting: selecting attributes for the x and y axis. In the tab “General Plot Options”, chart title and subtitle can be specified.

Figure 10. The configuration window of the Scatter Plot node.

Now, let’s look at the visualization (Fig. 11). As you can see, the clusters are quite widespread, especially cluster 0 and cluster 1. This and the Silhouette Coefficient of 0.45 indicates that we might need to rethink our parameter choice.

f) Document Workflow with Comments and Annotations

The final workflow is shown in Fig. 3 and can be downloaded from the KNIME Hub.

Try Out Different k’s

In case the clustering is not as satisfying as expected, simply rerun k-Means with different parameters and see if a better clustering can be achieved.

Below, we report the mean Silhouette Coefficient for k=3, k=5, and k=8 and the corresponding scatter plots.

Table 1. The mean overall Silhouette Coefficient for different k

k=3 k=5 k=8

Overall Silhouette Coefficient 0.45 0.56 0.44

Figure 11. The visualization of the resulting cluster set for k=3.

Cluster set for k=5

Figure 13. Cluster set for k=8

By comparing the scatter plots (visually) and the silhouette coefficient values, k=5 seems to be the preferable choice, so far. However, using the optimization procedure based on the Silhouette Coefficient and implemented in the component named “Optimized k-Means (Silhouette Coefficient)”, k=5 turns out to be the best choice for setting k.

Now, we should explain how to create and use a component… but this is material for the next article.

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.


Update the detailed information about Introduction To Hyperledger Fabric In Blockchain Network on the website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!