UiPath Web Automation - Web Data Extraction Using RPA

UiPath Web Automation - Web Data Extraction Using RPA UiPath Web Automation One Stop Solution To Web Extraction Back Home Categories Online Courses Mock Interviews Webinars NEW Community Write for Us Categories Artificial Intelligence AI vs Machine Learning vs Deep LearningMachine Learning AlgorithmsArtificial Intelligence TutorialWhat is Deep LearningDeep Learning TutorialInstall TensorFlowDeep Learning with PythonBackpropagationTensorFlow TutorialConvolutional Neural Network TutorialVIEW ALL BI and Visualization What is TableauTableau TutorialTableau Interview QuestionsWhat is InformaticaInformatica Interview QuestionsPower BI TutorialPower BI Interview QuestionsOLTP vs OLAPQlikView TutorialAdvanced Excel Formulas TutorialVIEW ALL Big Data What is HadoopHadoop ArchitectureHadoop TutorialHadoop Interview QuestionsHadoop EcosystemData Science vs Big Data vs Data AnalyticsWhat is Big DataMapReduce TutorialPig TutorialSpark TutorialSpark Interview QuestionsBig Data TutorialHive TutorialVIEW ALL Blockchain Blockchain TutorialWhat is BlockchainHyperledger FabricWhat Is EthereumEthereum TutorialB lockchain ApplicationsSolidity TutorialBlockchain ProgrammingHow Blockchain WorksVIEW ALL Cloud Computing What is AWSAWS TutorialAWS CertificationAzure Interview QuestionsAzure TutorialWhat Is Cloud ComputingWhat Is SalesforceIoT TutorialSalesforce TutorialSalesforce Interview QuestionsVIEW ALL Cyber Security Cloud SecurityWhat is CryptographyNmap TutorialSQL Injection AttacksHow To Install Kali LinuxHow to become an Ethical Hacker?Footprinting in Ethical HackingNetwork Scanning for Ethical HackingARP SpoofingApplication SecurityVIEW ALL Data Science Python Pandas TutorialWhat is Machine LearningMachine Learning TutorialMachine Learning ProjectsMachine Learning Interview QuestionsWhat Is Data ScienceSAS TutorialR TutorialData Science ProjectsHow to become a data scientistData Science Interview QuestionsData Scientist SalaryVIEW ALL Data Warehousing and ETL What is Data WarehouseDimension Table in Data WarehousingData Warehousing Interview QuestionsData warehouse architectureTalend T utorialTalend ETL ToolTalend Interview QuestionsFact Table and its TypesInformatica TransformationsInformatica TutorialVIEW ALL Databases What is MySQLMySQL Data TypesSQL JoinsSQL Data TypesWhat is MongoDBMongoDB Interview QuestionsMySQL TutorialSQL Interview QuestionsSQL CommandsMySQL Interview QuestionsVIEW ALL DevOps What is DevOpsDevOps vs AgileDevOps ToolsDevOps TutorialHow To Become A DevOps EngineerDevOps Interview QuestionsWhat Is DockerDocker TutorialDocker Interview QuestionsWhat Is ChefWhat Is KubernetesKubernetes TutorialVIEW ALL Front End Web Development What is JavaScript â€" All You Need To Know About JavaScriptJavaScript TutorialJavaScript Interview QuestionsJavaScript FrameworksAngular TutorialAngular Interview QuestionsWhat is REST API?React TutorialReact vs AngularjQuery TutorialNode TutorialReact Interview QuestionsVIEW ALL Mobile Development Android TutorialAndroid Interview QuestionsAndroid ArchitectureAndroid SQLite DatabaseProgramming ... Robotics Process Au tomation using UiPath (34 Blogs) Become a Certified Professional AWS Global Infrastructure Introduction to RPA What is Robotic Process Automation? â€" An Introduction to RPARobotic Process Automation (RPA) Tutorial â€" Learn To Automate Tasks in RPARobotic Process Automation â€" All You Need To Know About RPAKnow The Top 10 Challenges of RPA Implementation RPA Tools RPA Tools List and Comparison â€" Leaders in RPA Software RPA in Various Domains: How RPA Solves Industry Level Problems? UiPath How To Install UiPath on Windows? â€" Your One Stop Solution To Install UiPathWhat is UiPath? â€" A Quick Guide To The Top RPA ToolUiPath Tutorial : Introduction to UiPath and its ComponentsUiPath RPA Architecture â€" A Deconstruction of the Components of UiPathEverything You Need To Know About Variables, Data Types One Stop Solution To Web Extraction Last updated on May 14,2020 7.5K Views Sahiti Kappagantula Bookmark 9 / 15 Blog from UiPath Become a Certified Professional En terprises and startups, both of them set their applications on widely structured web-based systems. Information present in these web-based systems is quite complex and needs extensive processing to be read, understood and analyzed. Well, to this, we need robotic process automation. With the help of UiPath Web Automation, we can automate various kinds of tasks such as form filling, screen scraping, data extraction websitetesting and so on.The following topics will be covered in this article :What is Robotic Process Automation?What is UiPath?What is Web Automation?Usage of Web AutomationHands-On:Web Scraping of Google ContactsHands-On:Extracting Data From a E-Commerce Website and storing in EmailSo, let us get started!!What is Robotic Process Automation?The Process of automating the workflows with the help of robots/software to reduce the involvement of humans is said to beRobotic Process Automation.Over here there are mainly three terms that you need to understand:Robotic, Processand Automation.Let me explain you each of these terms one by one.Robotic:Entities which mimic human actions are called Robots.Process:Sequence of steps which lead to meaningful activity. For example, the process of making tea or your favorite dish etc.Automation:Any process which is done by a robot without human intervention.If we summarize all the terms together then, mimicking human actions to perform a sequence of steps that lead to meaningful activity, without any human intervention is known as Robotic Process Automation.Now, to implement Robotic Process Automation, we need tools, and one such tool is, UiPath. So, next in this article on UiPath Web Automation, let us get a brief on UiPath.What is UiPath?UiPath is a Robotic Process Automation tool used mainly for Windows Desktop Automation. This tool offers a Community Edition, which is free for lifetime and works on the feature of Drag and Drop functionalities. With UiPath, you can perform various types of automation such as Excel A utomation, Citrix Automation, Web Automation, PDF Automation and so on. Well, in this article we will focus on Web Automation.What is Web Automation?Web Automation is a process of automation through which you can identify the web elements by their attributes and manipulate them accordingly. This kind of automation works with any website, and can also be remotely deployed on various machines in the network. UiPath provides a built-in recorder that can read and enact web-based activities with the help of an intuitive graphical workflow editor.You may also go through the recording of UiPath Web Automation where ourcertifiedexperts of RPA Training have explained the concepts in depth.UiPath Web Automation | Automate Web Data Extraction UiPath | Edureka This video will help you know how to automate web using UiPath. Now, let us look into the top implementations of Web Automation in UiPath.Usage of Web AutomationThe top usages of web automation in todays industry are:Data Extraction: Hum ongous amount of data can be extracted and stored into files and databases from various websites.Recording: Actions performed on a website can be recorded to implement them further in the future.Website Testing: Your website functionalities and features can be tested according to the requirement of the user.Report Generation: A report can be generated by extracting all data and then analyzing it on various parameters.Application Transfer: Applications can be transferred from one domain to another, on the basis of use-requirement.Web scraping can be used for various types of web page formats such as JSON, HTML. Now, that you what is web automation, let us see a practical approach on how to use UiPath to perform Web Automation.We will see two hands-on in this article. They are:Scraping data from Google ContactsScraping data from an E-Commerce Website and storing in e-mailLet us start with the first hands-on in this article on UiPath Web Automation.Hands-On:Web Scraping of Google Conta ctsTaskExtract data from Google Contacts and store it in a file.Steps To AutomateFollow the below steps to achieve the task:Use the Data Scraping tool to extract data.Extract the correlated values accordingly.Store the data in a CSV file by using the Write CSV activity.SolutionStep 1:Open the Google Contacts page to extract data.Step 2: Now, open UiPath Studio and create a Black Project.Step 3: Choose theData Scrapingoption from the Ribbon and select the element you want to choose. PressNextin the below Dialog Box.Step 3.1:Hover the mouse over a data source field and then click on the data source field.Here I want to choose the name.Step 3.2:After that, you will see another Dialog box which will ask you to choose the second element to create a pattern.Step 4: Once you choose the second element to create a pattern, you will get an option toConfigure Columns. In the Dialog Box, that opens up, you can rename the Column name and extract URL also. After that click on Next. Refer Below.St ep 5:Now, to extract other data sources from the website, click on theExtract Correlated Dataoption and repeat the above steps. Here, I am going to extract the email-id of the contacts. You would see the below output.Step 6:Once you are done extracting all the required data, click onFinish. This action would open a dialog box which will ask you if you wish to span the data across multiple pages. Refer below.Step 7:To span across multiple pages, chooseYesand hover the mouse over the region which redirects you to the next page. Then you will be redirected to your UiPath Dashboard.Step 8:Now to store all the extracted data into a .csv file, drag aWrite CSV activityinto theDosection of theData Scraping.Step 9:In theFile Path sectionof this activity, mention thepath of the .csv filewhere you want to store the extracted data and then mention theExtractDataTable variablein theDataTable section. Refer below.NOTE:The ExtractDataTable variable is the output variable that is automatically gene rated from the Data Scraping Wizard. You can find this variable in the Extract Structured Data activity.Step 10:Execute this sequence by clicking on theRunbutton. You would see that the data is getting stored in the .csv file.So, folks. thats how you can extract data from a website. Now, that you know how to scrape data from a website, let us move on to our second hands-on in this article on UiPath Web Automation.Hands-On:Extracting Data From an E-Commerce WebsiteTaskIn this hands-on, my task is to design automation which reads the subject line of an email and performs a custom search. So, for example, if my subject line says LG TV then, the designed automation must search for LG TVs in Flipkart and extract the name, URL, and the price.Steps To AutomateFollow the below steps to achieve the task:Drag the Get IMAP Mail messages activity and configure the activity.Drag the For Each activity to perform a set of actions.Drag the Open Browser activity and mention the URLUse the Data Scrap ing tool and extract the data.Extract the correlated values accordingly.Drag the Write CSV activity and mention the path of file.Use the Send SMTP Mail Messages activity to send an email.SolutionStep 1: Create a Blank projectStep 2: Our first task is to get an email with a subject line. Here I want the subject line to be apple phones. So, to do that, drag the Get IMAP Mail Messages activity.Step 2.1: Go to the Properties Pane of this activity, and mention the following:Port Number: 993Server Name: imap.gmail.comEmail: Mention your email IDPassword: Mention your passwordTop: 1 (This would read only the 1stemail in your Inbox.)Check the box of Read only Unread Messages. (This would read only the Unread Messages.)Refer below.Step 2.2: Now to store the data from the email, we need to create a variable. So, to create a variable, go to Variable Pane and mention the name and Type of variable. Here I will mention the name as MailMessages and type as ListT.Step 2.3: Now, in the T, you have t o mention the Mail Message. To do that, Browse for Types in the variable section and search for System.Net.Mail.Under this section, you have to choose the MailMessage. Refer below.Step 2.4: Mention the variable MailMessages in the Output section ofGet IMAP Mail Messages activity.Step 3: Now, to enable the information to be read for n number of emails, you have to drag the For Each activity, below the Get IMAP Mail Messages activity.Step 3.1:Next, in the item section mention mail, and in the expression section, mention the MailMessages variable. This will enable you to perform a set of actions mentioned in the Body section for each and every mail. Also, in the Properties section of this activity, mention the Type Argument to be System.Net.MailMessage. Refer below.Step 4: Now, you have to open the Flipkart website. To do that, you have to drag the Open Browser activity in the Body section of the For Each activity and mention the URL in double quotes. Here I will search for Fossil watc hes and mention the URL.Note: It doesnt matter what you search for, since we will change the URL in the next step.Step 5: Now, since our task is to read the subject line and then search for the items mention in the subject line, we have to modify the URL mentioned in the Open Browser activity.So, if your URL was as follows:https://www.flipkart.com/searchq=fossil%20watchesotracker=searchotracker1=searchmarketplace=FLIPKARTas-show=onas=off, then you have change the item under the search parameter and include +mail.Subject+ in quotes. So, your new URL should be something like below.https://www.flipkart.com/search?q=+mail.Subject+otracker=searchotracker1=searchmarketplace=FLIPKARTas-show=onas=offRefer below.Till Now, you have designed your automation to read the subject line of an email, and then perform custom search in the Flipkart website. So, if the subject line had apple phones, then the designed automation will read the subject line and search for apple phones in Flipkart.Now, our next task is to perform data scraping.Step 6: To perform Data Scraping, you can refer to our previous hands-on and perform the Steps from 1-7.Step 7: Now, once you have scraped the data and have been redirected to the UiPath Dashboard, you have to remove the Attach Browser activity and just keep the ExtractDataTable section in the Body section of For Each activity.Step 8:After that, store all the extracted data into a .csv file, drag aWrite CSV activityinto theDosection of the For Each Activity.Step 9:Next, in theFile Path sectionof this activity, mention thepath of the .csv filewhere you want to store the extracted data and then mention theExtractDataTable variablein theDataTable section. Refer below.Step 10: Now, to get an email with all the data scraped, you have to drag the Send SMTP Mail Messages activity in the Sequence.Step 10.1: In this activity you have to mention the following:To email address: abc@gmail.comSubject: attachmentBody: PFAThis step would design the task to se nd an email containing the scraped data, to the mentioned To email address with the subject line attachment and the Body be PFA. Refer below.Step 10.2: Since, we havent yet attached the file, to attach the file click on the Attach Files option in this activity and mention the path of the file in the dialog box that opens up. Then click OK. Refer below.Step 10.3: Now, you have to go to the Properties Pane of the SMTP activity and mention the following:Port Number - 465Server Name: smtp.gmail.comEmail: Mention the FROM email addressPassword: Mention the Password of FROM email addressFrom: Mention the FROM email addressStep 11: Now, compose a mail to yourself with the subject line apple%20phones (%20 is used to identify space in UiPath). Once you have received the mail, go back to the UiPath Dashboard and click on Run to execute the automation. You would see a mail with an attachment of the scraped data. Refer below.So, folks, thats an end to this article on UiPath Automation Examples. I hope you enjoyed reading this article on UiPath Web Automation and learned how to automate tasks. If you wish to further learn about Robotic Process Automation and build your career as anRPA Developer, then you can check out our course onRobotic Process Automation Using UiPath.This course will let you enhance your knowledge on RPA and will give you an extensive hands-on experience in UiPath. Got a question for us? Please mention it in the comments section of UiPath Web Automation and I will get back to you.Recommended blogs for you How to Build an Impressive RPA Developer Resume? Read Article 5 UiPath Automation Examples That You Can Practice Read Article Everything You Need To Know About Variables, Data Types Activities In UiPath Read Article RPA Tools List and Comparison Leaders in RPA Software Read Article UiPath Tutorial : Introduction to UiPath and its Components Read Article UiPath Careers Learn How To Build A Career In RPA Read Article UiPath Recording Tutorial Compreh ensive Guide On Recording In UiPath Read Article UiPath RPA Architecture A Deconstruction of the Components of UiPath Read Article UiPath ReFramework Tutorial A Comprehensive Guide on the Robotic Enterprise Framework Read Article What Is RPA Blue Prism? A beginners Guide to Blue Prism Read Article UiPath Web Automation One Stop Solution To Web Extraction Read Article How to become an RPA Developer? Learning Path For RPA Read Article Blue Prism Installation Learn How to Install, Configure and Activate Blue Prism Read Article Robotic Process Automation (RPA) Tutorial Learn To Automate Tasks in RPA Read Article Uipath Studio Your One Stop Solution To The Dashboard Of UiPath Read Article RPA Developer Roles and Responsibilities Read Article RPA in Various Domains: How RPA Solves Industry Level Problems? Read Article What is Blue Prism and How to use it? Read Article What is UiPath? A Quick Guide To The Top RPA Tool Read Article Top 10 Reasons To Learn RPA Read Article Comments 0 Comments Trending Courses in Robotic Process Automation Robotic Process Automation Training using Ui ...22k Enrolled LearnersWeekendLive Class Reviews 5 (8750)