{"id":7260,"date":"2021-07-23T13:12:35","date_gmt":"2021-07-23T13:12:35","guid":{"rendered":"https:\/\/rax-stg-wp-aws.com\/blogs\/python-for-effective-data-management-and-reporting\/"},"modified":"2026-11-03T19:28:21","modified_gmt":"2026-11-03T19:28:21","slug":"python-for-effective-data-management-and-reporting","status":"publish","type":"blogs","link":"https:\/\/www.rackspace.com\/en-gb\/blogs\/python-for-effective-data-management-and-reporting\/","title":{"rendered":"Python for Effective Data Management and Reporting"},"content":{"rendered":"<p><strong>Introduction<\/strong><br \/>\nThe blog introduces using Python\u00ae; for data management and reporting. Python uses Jupyter\u00ae; Notebook to access data sources to explore reading and writing data for generating reports from the source data.<\/p>\n<p><strong>Jupyter Notebook<\/strong><\/p>\n<p>In this post, I use the classic Jupyter Notebook running inside a browser from a server that can be local or remote. Jupyter Notebook supports over 40 programming languages, including Python, Java\u00ae, and JavaScript\u00ae.<\/p>\n<p>Jupyter Notebook consists of the following components:<\/p>\n<ul>\n<li>&nbsp;Jupyter Server<\/li>\n<li>&nbsp;Notebook<\/li>\n<li>&nbsp;Kernel<\/li>\n<li>&nbsp;Reports and portfolios<\/li>\n<\/ul>\n<p><strong>Install Jupyter Notebook, open a Notebook, and set the Kernel<\/strong><\/p>\n<p><strong>NOTE:<\/strong> I used Ubuntu\u00ae 18 on my server for this installation.<\/p>\n<p>Perform the following steps:<\/p>\n<p><strong>1.&nbsp;<\/strong>Install all security patches on the server using the following command:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; sudo apt upgrade -y<\/em><\/p>\n<p><strong>2<\/strong>. Run the following command to install one or more packages for converting notebooks to PDF:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp;sudo apt install -y texlive-xetex<\/em><\/p>\n<p><strong>3<\/strong>. To use Jupyter notebook, install Anaconda, a data science and machine learning<br \/>\n&nbsp; &nbsp;platform based on Python. Download the <strong>Anaconda3-2019.03-Linux-x86_64.sh<\/strong>&nbsp;package.<\/p>\n<p><strong>4.<\/strong> Run the following command to ensure this download package is not corrupt:<\/p>\n<p>&nbsp; &nbsp; &nbsp; &nbsp;<em>&nbsp;bash Anaconda3-2019.03-Linux-x86_64.sh<\/em><\/p>\n<p>&nbsp;<\/p>\n<p><strong>5.<\/strong> Run the following command to install Anaconda:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; bash Anaconda3-2019.03-Linux-x86_64.sh<\/em><\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture1_8.png\" alt=\"anaconda pic 1\" \/><\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture2_4.png\" alt=\"anaconda picture 2 \" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture2_3.png\" alt=\"anaconda pic 2\" \/><\/p>\n<p><strong>6<\/strong>. After the installation completes, restart the bash shell to include the changes<br \/>\n&nbsp; &nbsp;by running the following command:<\/p>\n<p>&nbsp; &nbsp; &nbsp; &nbsp; source ~\/.bashrc<\/p>\n<p>\n<strong>7. <\/strong>Update the conda\u00ae; base code by running the following commands:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; conda config &#8211;set auto_activate_base false<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; conda update -n base -c defaults conda<\/em><\/p>\n<p><strong>8.<\/strong> Create the conda virtual environment (such as **python\\_data**) using the latest Python version by running the following command:<\/p>\n<p>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<em>conda create -n python_data python=3<\/em><\/p>\n<p><strong>9<\/strong>. Activate this virtual environment y running the following command:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; conda activate python_data<\/em><\/p>\n<p><strong>10.<\/strong>&nbsp;Run the following commands to make a directory to hold this GitHub repo:<\/p>\n<p>&nbsp; &nbsp; &nbsp; &nbsp;<em>mkdir python_data<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; cd python_data<\/em><\/p>\n<p><strong>11<\/strong>. Run the following commands to install the packages:<\/p>\n<p>&nbsp; &nbsp; &nbsp; &nbsp; <em>conda activate python_data<br \/>\n&nbsp; &nbsp; &nbsp; &nbsp; conda install jupyter psutil<\/em><\/p>\n<p><strong>Connect to the Jupyter Notebook server<\/strong><\/p>\n<p>\nPerform the following steps to connect to the Juniper Notebook server:<\/p>\n<p><strong>1.<\/strong> Activate the virtual environment using the following command:<\/p>\n<p>&nbsp; &nbsp;<em> &nbsp; &nbsp; conda activate python_data<\/em><\/p>\n<p><strong>2. Run the following script to start the server:<\/strong><\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;python get_notebook_token.py&nbsp;<\/em><\/p>\n<p>&nbsp; &nbsp;This script uses the following command to start the Jupyter Notebook server&nbsp; &nbsp;and continues to run it outside of the terminal window:<\/p>\n<p>&nbsp; &nbsp; &nbsp; &nbsp;<em> nohup jupyter notebook &#8211;no-browser &#8211;port=8086<\/em><\/p>\n<p>&nbsp; &nbsp;This provides the token and searches the nohup output if you run the preceding<br \/>\n&nbsp; &nbsp;command in the background by using **get\\_notebook\\_token.py** as highlighted<br \/>\n&nbsp; &nbsp;in the following snapshot:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; nohup jupyter notebook &#8211;no-browser &#8211;port=8086<\/em><\/p>\n<p>This provides the token and searches the nohup output if you run the preceding command in the background by using <strong>get\\_notebook\\_token.py<\/strong> as highlighted in the following snapshot:<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture3_3.png\" alt=\"python pic 3\" \/><\/p>\n<p><strong>3.<\/strong> Go to the terminal window of your local machine and enter the following command:<\/p>\n<p><em>&nbsp; &nbsp; &nbsp; &nbsp; ssh -N -L localhost:8087:localhost:8086 username@&lt;the public IP address of the server&gt;<\/em><\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture4_4.png\" alt=\"Python Pic 4\" \/><\/p>\n<p><strong>4<\/strong>. Leave this terminal open while using the Jupyter Notebook server in this session.<\/p>\n<p><strong>5<\/strong>. In the browser, enter the following URL **http:\/\/localhost:8087** and provide<br \/>\n&nbsp; &nbsp;the token as input.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture5_2.png\" alt=\"python pic 5\" \/><\/p>\n<p>After logging, &nbsp;you can create a Jupyter Notebook by using Python code and Markdown cells.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/26342-body-Picture6_1.png\" alt=\"python pic 6\" \/><\/p>\n<p><strong>&nbsp;Examining the data in a DataFrame<\/strong><\/p>\n<p>Use Pandas DataFrames inside of Jupyter Notebook for examining the data in a<br \/>\nDataFrame. Run the following command to import the <strong>pandas<\/strong>&nbsp;package.<\/p>\n<p><em>&nbsp; &nbsp; import pandas as pd<\/em><\/p>\n<p>There are different ways to examine data using Pandas DataFrame:<\/p>\n<p>&#8211; <strong>head<\/strong>(): Displays the first five rows in the DataFrame.<br \/>\n&#8211; .<strong>tail<\/strong>(): Displays the last five rows in the DataFrame.<br \/>\n&#8211; <strong>vet\\_records<\/strong>: Displays all the records of the DataFrame.<br \/>\n&#8211; .<strong>dtypes<\/strong>: Shows statistical operations on columns where you can perform these operations.<br \/>\n&#8211; .<strong>at<\/strong>: &nbsp;Allows the user to change the value of a specific cell.<br \/>\n&#8211; <strong>f: .assign<\/strong>: Used to add another column of data.<br \/>\n&#8211; <strong>vet_records.type.count<\/strong>(): Used for counting and grouping data.<br \/>\n&#8211; <strong>loc<\/strong>: Allows you to use column names to slice data.<br \/>\n&#8211; &#8211; .<strong>head()<\/strong>: Displays the first five rows in the DataFrame.<br \/>\n&#8211; <strong>.tail()<\/strong>: Displays the last five rows in the DataFrame.<br \/>\n&#8211; <strong>vet\\_records<\/strong>: Displays all the records of the DataFrame.<br \/>\n&#8211; .<strong>dtypes<\/strong>: Shows statistical operations on columns where you can perform these operations.<br \/>\n&#8211; <strong>.at<\/strong>: &nbsp;Allows the user to change the value of a specific cell.<br \/>\n&#8211; <strong>f: .assign<\/strong>: Used to add another column of data.<br \/>\n&#8211; <strong>vet_records.type.count()<\/strong>: Used for counting and grouping data.<br \/>\n&#8211; <strong>loc<\/strong>: Allows you to use column names to slice data.<br \/>\n&#8211; <strong>iloc<\/strong>: Requires the use of index numbers. &nbsp;For example:<br \/>\n&nbsp; .iloc[row, column]. remember: python` indexes starting at 0.<br \/>\n&#8211; <strong>.isin<\/strong>: Used to gather data about a list of items.**: Requires the use of index numbers. &nbsp;For example:<br \/>\n&nbsp; `.iloc[row, column]. remember: python` indexes starting at 0.<br \/>\n&#8211; .<strong>isin<\/strong>: Used to gather data about a list of items.<\/p>\n<p><strong>Writing a report<\/strong><br \/>\nFor writing a report, you need to use LaTex\u00ae. LaTex, built into JupyterNotebook, helps you create technical and scientific documents and is good at displaying mathematical formulas. You can write LaTex formulas in a Markdown cell.&nbsp;For the Jupyter report, use a data set that allows you to consider some aspects&nbsp;of socioeconomic factors that may impact it. To create the report and make sure to run the following commands to install supporting packages:<\/p>\n<ul>\n<li>&nbsp; &nbsp; &nbsp;import matplotlib.pyplot&nbsp;<\/li>\n<li>&nbsp; &nbsp; &nbsp;import numpy&nbsp;<\/li>\n<li>&nbsp; &nbsp; &nbsp;import pandas<\/li>\n<li>&nbsp; &nbsp; &nbsp;import random<\/li>\n<\/ul>\n<p><strong>Conclusion<\/strong><\/p>\n<p>You can create and share documents that contain live codes, equations, and visualizations that help in data cleaning, transformation, and statistical modeling in data science.&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Python for Effective Data Management and Reporting<\/p>\n","protected":false},"author":2,"featured_media":7262,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","format":"standard","meta":{"content-type":"","nid":"26342","content_type":"blog","created_by":"10130","authored_on":"2023-12-05T14:12:35+00:00","field_article_industries":"","tag_category":"Technical","field_blog_author_text":"by Rackspace Technology Staff","seo_description":"Harness Python for data management and report generation. Simplify data exploration and reporting using Jupyter Notebook.\r\n","field_featured_image":"","insights_image":"\/wp-content\/uploads\/2026\/blog_technical_images\/26342-insights_image-productivity-technical-blog-1-insights.jpg","field_partners":"","rxt_platforms":"Big Data","product":"","publisheddate":"2021-07-23T13:12:35","rxt_solutions":"Cloud, Cloud Optimization","opengraph_tag":"","x_tag":"","teaser_flag":"Technical Blog","seo_titile":"Python for Effective Data Management and Reporting| Rackspace Technology","field_topics":"","field_slug":"using-python-for-data-management-and-reporting","footnotes":""},"categories":[178],"tags":[204],"class_list":["post-7260","blogs","type-blogs","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","tag-technical"],"_links":{"self":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs\/7260","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs"}],"about":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/types\/blogs"}],"author":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/comments?post=7260"}],"version-history":[{"count":2,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs\/7260\/revisions"}],"predecessor-version":[{"id":9283,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs\/7260\/revisions\/9283"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/media\/7262"}],"wp:attachment":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/media?parent=7260"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/categories?post=7260"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/tags?post=7260"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}