Categories
c

Adventures in C

If you’re more familiar with Python you’ll need to tackle pointers…

Let’s look at an example :

The first thing you may notice is that there is no need for a “return” from the function….

Example 1 – Pointers

#include<stdio.h>

void tripler(int *p);


int main(){
    
    int num = 5;
    int *ptr;
    ptr = &num;

    tripler(ptr);
    printf("%d",*ptr);
}


void tripler(int *p){
    *p = *p * 3;
}

pointer and address of operator

Example 2 – Pointer that updates conditionally, based on values read from an array

Example 3 – How to convert seconds into h:m:s

Convert seconds into h:m:s with C

Example 4 – Hundreds, tens, units from a 3 digit integer.

Use the modulus operator and division by 10

2d Arrays in C

2d-array-in-c
Numpy is easier? Same concept though…
Categories
ebay api requests

Python API – POST Requests

Many API endpoints require use of the POST method.

Here we look at how you can use Python and Requests to work with an API. Example code and a real world example connecting to the EBAY API.

Endpoint – The URL that delineates what data you are interacting with.

Method – Specifies how you’re interacting with the resource located at the provided endpoint. REST APIs can provide methods to enable full Create, Read, Update, and Delete (CRUD) functionality. Here are common methods most REST APIs provide:

GET – Retrieve data

PUT – Replace data

POST – Create data

DELETE – Delete data

Data – If you’re using a method that involves changing data in a REST API, you’ll need to include a data payload with the request that includes all data that will be created or modified.

Headers contain any metadata that needs to be included with the request, such as authentication tokens, the content type that should be returned, and any caching policies.

Authenticate to a REST API

The most common framework for API authentication is OAuth. acts as an equivalent to a username/password combination;

Once you have an access token, you can provide it as a bearer token in the request header: this is the most secure way to authenticate to a REST API with an access token:

my_headers = {‘Authorization’ : ‘Bearer {access_token}’} response = requests.get(‘http://httpbin.org/headers’, headers=my_headers)

API Response

response.content() # Return the raw bytes of the data payload response.text() # Return a string representation of the data payload response.json() # This method is convenient when the API returns JSON

– examples –

https://www.w3schools.com/python/ref_requests_post.asp https://www.w3schools.com/python/ref_requests_response.asp

Categories
JavaScript

JavaScript tips

A collection of useful JavaScript functions and handy tips / study notes

forEach

                    
// forEach
const x = [1,3,4,66,77];
x.forEach(v => {log (v*2)})

Map

                    
// Map
const x = [1,3,4,5,6,7,87]
let f = x.map(x=>x*2);
log(f);

Filter

                    
// Filter
const ages = [1,3,4,5,6,7,87];
let filteredAges = function(ages){
return (ages > 5);
}
let f = ages.filter(filteredAges);
log(f);

Invoking a class

                    
// Create Animal.js
export default class Animal{
constructor() {
console.log(“i am an animal”);
}
getClassType() {
return “Animal”;
}
}
// In Index.js
// import Animal from ‘./Animal’;
var animal = new Animal();
log(animal.getClassType()

Clickable div – linked to a js function

Use a JavaScript function on a div – (flex)
                    
function klik1() {
document.getElementById(“moo1″).innerHTML=”M”;
}

Use JavaScript to fetch data

Use Stringify to convert JSON to text

API – JS – HTML Demo

We use JavaScript to fetch from an API and create a new element in the html for each character in the API response

                    
const fetchapi = fetch(“https://rickandmortyapi.com/api/character/”)
.then(response => response.json())
.then(characters => showCharacters(characters.results));
//
const showCharacters = characters => {
const charactersDiv = document.querySelector(“#rmc”);
characters.forEach(character => {
const characterElement = document.createElement(‘p’);
characterElement.innerText = `Character Name: ${character.name}`;
charactersDiv.append(characterElement); })};
                    
<!DOCTYPE html>
<html lang=”en”>
<head>
<meta charset=”UTF-8″>
<meta http-equiv=”X-UA-Compatible” content=”IE=edge”>
<title>Flex</title>
<meta name=”viewport” content=”width=device-width, initial-scale=1″>
<script src=”https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js”></script>
<script src=”https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/js/bootstrap.min.js”></script>
<link rel=”stylesheet” href=”style.css”>
</head>
<body>
<div class=”container-fluid”>
<h1>My API Demo Page</h1>
<script src=”app.js” defer=”true”></script>
<div id=”rmc”></div>
</div>
</body>
</html>
js-api-html-demo
Categories
JavaScript

Callback functions

Javascript callback functions explained.

As well as Python we’ll be adding semi regular tips and case studies of JavaScript code and features. In this article we’ll look at “callback” functions in JavaScript.

Let’s simplify them down to a very basic example which we show here:

                    
    function cbdemo(a, b, callback){
        console.log(callback(a,b));
    };
    var callback = function(a,b){
        return a * b;
    };
    var bx = cbdemo(2,3, callback);

                

The above will print to console.

We have a function calling another function inside itself.

Let’s use the response from the cbdemo function and display on the page

Now we get a nice bold “h2” number 6 viewable in our page in the browser!

Categories
Python Code

Creating a Python Package

Make your own python package and make it installable with pip install.

If you are wanting to create your own Python module for others to use and provide the best way to install it then https://pypi.org/ is the way to go.

Watch how I learned to make a pip package and successfully created it
# __init__.py

# Version of the gimdl package
__version__ = '0.0.2'
__author__ = 'DrPi'
__email__ = 'redandgreencode@email.com'
# setup.py

from setuptools import setup

with open("README.md", "r") as fh:
    long_description = fh.read()

setup(
    name='gimdl',
    version='0.0.12',
    description='Google Image Search Downloader - using API ',
    py_modules=["gimdl"],
    install_requires=['python-dotenv'],
    package_dir={'': 'src'},
    
    classifiers=[
        'Development Status :: 3 - Alpha',
        'License :: OSI Approved :: MIT License',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: 3.8',
        'Programming Language :: Python :: 3.9',
        'Operating System :: OS Independent',
    ],
    project_urls={  # Optional
        'GitHub': 'https://github.com/RGGH/gimdl',
    },

    keywords='googleimages, images, search', 
    long_description=long_description,
    long_description_content_type="text/markdown"
)
                    

TL;DR

$ python3 bdist_wheel sdist
$ sudo pip install python3-twine
$ twine register dist/gimdl-0.0.1.tar.gz
$ twine upload dist/gimdl-0.0.1.tar.gz
$ pip install gimdl

To upgrade package after change to source code:
$ python3 bdist_wheel sdist
$ twine –skip-existing dist/*

see: https://twine.readthedocs.io/en/latest/


                
See also https://pythonhosted.org/an_example_pypi_project/setuptools.html
Categories
Python Code

Google Image Search

And How To Use Python to automate it!

image-search-google
Step 1 – install Google-Images-Search
instructions-google
Step 2 – follow the installation instructions
env variables
Step 3 – set up the .env file with the API key and CX code you got when you generated your API credentials with Google.
Note : Make sure you enabled image search by setting it to “on”

code:

from google_images_search import GoogleImagesSearch
from dotenv import load_dotenv
import os
# load the API key and CX code from .env file
load_dotenv()

# create an 'ims' sub directory if it doesn't already exists
if not os.path.exists('ims/'):
  os.mkdir('ims/')
pathz = 'ims/'
# Get env variables
DK = os.environ.get('DEVELOPER_KEY')
CX = os.environ.get('CX')

# custom progressbar function
def my_progressbar(url, progress):
    print(url + " " + progress + "%")

# create google images search - object
gis = GoogleImagesSearch(DK, CX, progressbar_fn=my_progressbar)
# using contextual mode (Curses)
with GoogleImagesSearch(DK, CX) as gis:
    # define search params:
    _search_params = {"q": "tree", 
        "num": 3, 
        "safe": "off", 
        "fileType": "png"
        }
    gis.search(search_params=_search_params, path_to_dir=pathz)

print("\nDone\n")
3 images of trees – here is one of them!

Limit Exceeded!

If you see “Error 429 when requesting…” or…

RESOURCE_EXHAUSTED

then you may have exceeded your daily allowance for API calls. So be careful when testing the code not to make too many requests.

quota-reached-google-api
Quota reached!

To install our code you can use pip:

pip install gimdl

https://pypi.org/project/gimdl/

Categories
Python Code

GitHub Support for password authentication

Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.

This article shows you how to fix it!

remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.
remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information.

2 Steps to fix

(base) [rag@pipdem]$ git remote set-url origin git@github.com:RGGH/pipdem.git

replace RGGH with your user (account) name

replace pipdem with your repo name


(base) [rag@pipdem]$ git commit -am "Update"
On branch main
nothing to commit, working tree clean


(base) [rag@pipdem]$ git push --set-upstream origin main


Enumerating objects: 16, done.
Counting objects: 100% (16/16), done.
Delta compression using up to 8 threads
Compressing objects: 100% (12/12), done.
Writing objects: 100% (16/16), 2.40 KiB | 1.20 MiB/s, done.
Total 16 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), done.

                    
$ ls -al ~/.ssh
# Lists the files in your .ssh directory, 
# (if they exist)

# By default, the filenames of the public keys # are one of the following: id_rsa.pub id_ecdsa.pub id_ed25519.pub # If you don’t have an existing public # and private key pair then generate a # new SSH key

Check the GitHub instructions on how to generate a new SSH key:

https://docs.github.com/en/enterprise-server@2.22/github/authenticating-to-github/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent

Categories
Python Code

Remove non printable characters from text

import string

my_str = "htt😆, moon, boo, f🔥re "

def clean_words(my_str):
    for i in my_str.split():
        word = "" if (any([char not in string.printable for char in i])) else i
        print(word)

clean_words(my_str)
-
Categories
Pandas Python Code

map & lambda

Introduction

Using lambda can save you having to write a function.

If you’ve not used ‘map’ then we’ll show you how it can perform the same task as lambda in an example

import pandas as pd
pd.set_option('max_rows',10)
import numpy as np
reviews = pd.read_csv("winemag-data-130k-v2.csv",index_col=0)
reviews

next we’ll drop any rows full of NaNs

reviews.dropna()

now we have good data…

reviews.price.mean()

35.363389129985535

We can now use a lambda expression to run all the way down the price column and update it to show whether it is more or less than the mean:

reviews_price_mean = reviews.price.mean()
reviews.price.apply(lambda p : p - reviews_price_mean)

What does this do exactly?

lambda p is equivalent to the price value in each row

p - reviews_price_mean

We subtract the mean review price from the ‘p’ value to give us a positive or negative value compared to the mean price.

By applying it with apply we can go all the way through the dataframe.

We can create a new column reviews[‘price_dfif’] and set that equal to the result of our lambda function.

0               NaN
1        -20.363389
2        -21.363389
3        -22.363389
4         29.636611
            ...    
129966    -7.363389
129967    39.636611
129968    -5.363389
129969    -3.363389
129970   -14.363389
Name: price, Length: 129971, dtype: float64

The result now shows as a price as +/- the mean

Summary:

Using map gives the same results:

reviews.price_diff.map(lambda p : p - reviews_price_mean)
wine-reviews-price-vs-points
For those interested, here are the prices vs points from the reviewers

Both of these ways allow you to apply a function without the need for a traditional Python ‘for’ loop.

Categories
Python Code

Data Analysis With Pandas

If you want to learn about Data Analysis with Pandas and Python and you’re not familiar with Kaggle, check it out!

Time to read article : 5 mins

TLDR;

We show how to use idxmax and apply with Pandas

Introduction

Here we will look at some functions in Pandas which will help with ‘EDA’ – exploratory data analysis.

Kaggle-Screenshot-functions-and-maps-Data Analysis
Using Kaggle is much like Jupyter Notebook

Once you have signed in you can locate Pandas tutorials and begin learning and testing your understanding by running through the exercises and if you get stuck there are hints, and also the solution.

Pandas ‘idxmax’ example:

One such exercise is shown here, where you are asked:

Which wine is the “best bargain”? Create a variable bargain_wine with the title of the wine with the highest points-to-price ratio in the dataset.

The hint tells us to use idxmax()

Here is an example of what idxmax does:

idxmax in pandas-Data Analysis
We can now use this as a binary mask.

Solution:

bargain_idx = (reviews.points / reviews.price).idxmax()
bargain_wine = reviews.loc[bargain_idx, 'title']

The Kaggle solution locates the row where idxmax is True, and returns the ‘title’

In this next example we see how you can “apply” a function to each row in your dataframe:

kaggle-pandas-function-Data Analysis
Using a function on all rows

Pandas ‘apply’ example:

“Find number of dealers by area”

import pandas as pd
import numpy as np

pd.set_option('display.max_colwidth', 150) 
df = pd.read_csv("used_car.csv")
df.head(6)
NameAddressPhone
0PakWheels KarachiSuit No : 303 Third Floor Tariq Centre Main Tariq Road3105703505
1PakWheels Lahore37 Commercial Zone, Liberty Market Lahore614584545
2PakWheels Islamabad37 Commercial Zone, Liberty Market Lahore614584545
3Sam Automobiles8 Banglore town,7/8 Block near awami markaz shahrah e faisal karachi3422804414
4Marvel Motors27-E, Ali Plaza, Fazal e Haq Road, Blue Area, Islamabad518358006
5Merchants AutomobilesPlot 167 /C Shop#4 Parsa City luxuria PECHS Block 3 at Main khalid bin waleed road2134552897

In [152]:

# create a function to assign a city based on address details

def area(row):
    if 'lahore' in (str(row.Address)).lower():
        return "Lahore"
    elif "faisalbad" in (str(row.Address)).lower():
        return "Faisalbad"
    elif "karachi" in (str(row.Address)).lower():
        return "Karachi"
    elif "islamabad" in (str(row.Address)).lower():
        return "Islamabad"
    else:
        return "Other"
ans = df.apply(area, axis=1)
ans
0           Other
1          Lahore
2          Lahore
3         Karachi
4       Islamabad
          ...    
2332        Other
2333        Other
2334        Other
2335        Other
2336        Other
Length: 2337, dtype: object
# Check how many times each city occurs in the dataframe

ans.value_counts()
Other        1433
Karachi       471
Lahore        331
Islamabad     102
dtype: int64

Next, what if we want to find out if there are other dealerships that use the same phone number?

Pandas ‘isin’ with ‘loc’

df.loc[df.Phone.isin(['614584545'])]
NameAddressPhone
1PakWheels Lahore37 Commercial Zone, Liberty Market Lahore614584545
2PakWheels Islamabad37 Commercial Zone, Liberty Market Lahore614584545
437Rehman MotorsOld Bahawalpur Road,Multan614584545
We’ve found 3 business names, and 2 addresses that share the same phone number….

Summary

Kaggle is free and even if you are not pursuing a career in data science you can still gain valuable Python skills from it.

See the Red and Green – Kaggle example on Kaggle