Udacity

From Wiki RB4

Courses

Content

  • SQL
  • ORM
  • python
  • Postgres
  • Json
  • Flask
  • Ninja
  • HTTP
  • REST
  • CORS
  • API design
  • cURL
  • testing
  • documentation
  • IT security
  • Postman
  • Authentication patterns
  • Auth0
  • Docker
  • Kubernetes
  • AWS services

Full Stack Web Developer

  • C:\Uwes\python\UdacityFullWebDeveloper

Authorization

GET https://YOUR_DOMAIN/authorize?
 audience=API_IDENTIFIER&
 scope=SCOPE&
 response_type=code&
 client_id=YOUR_CLIENT_ID&
 redirect_uri=https://YOUR_APP/callback&
 state=STATE
  • after authentification the callback URL is called with a JWT


$env:FLASK_APP = "app.py"
flask run --reload


Coffee Shop App

Solutions
Backend Installation
sudo apt install python3.8-venv
  • create directory \Uwes\python\UdacityFullWebDeveloper with sudo
sudo git clone https://github.com/udacity/FSND.git
  • cd backend
sudo apt-get install python-dev
sudo apt-get install gcc
sudo python3 -m venv venv
sudo chown -R ubuntu:ubuntu venv/
source ./venv/bin/activate
pip install wheel
deactivate venv
Running the Backend
cd /Uwes/python/UdacityFullWebDeveloper/FSND/projects/03_coffee_shop_full_stack/starter_code/backend
export FLASK_APP=api.py;
source ./venv/bin/activate
cd /src
flask run --reload --port=5321 --host=0.0.0.0
// test
http://54.212.123.129:5321/drinks
// if CTRL + C does not terminate the process
netstat -tulpen
sudo kill -9 <PID>
Running the Frontend
cd C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\03_coffee_shop_full_stack\starter_code\frontend
ionic serve // test it with http://localhost:8100
Implementation
  • implement /drinks in api.py w/o any access check
  • Create new API 'Coffee Shop' with ID (Audience) 'coffee-shop' by the Auth0 account (see above) and enable RBAC and select 'Add Permission to token'
  • add permissions to the API 'Coffee Shop'
    • get:drinks
    • get:drinks-detail
    • post:drink
    • patch:drink
    • delete:drink
  • create roles 'Barista' (get:drinks, get:drinks-detail) and 'Manager' and assign permissions for API 'Coffee Shop'
  • assign role 'Manager' to user 'uwe.heuer@eon.com' and 'Barista' to user 'uwe.heuer@gmail.com'
  • implement missing methods in /auth/auth.py


  • implement /drinks-detail in api.py with @requires_auth

login with https://uweheuer.eu.auth0.com/authorize?audience=coffee-shop&response_type=token&client_id=Po3cpRtDEzR2LPUyrkeAStpoGSQSaeax&redirect_uri=https://127.0.0.1:8080/login-results

API Apps

$env:FLASK_APP = "flaskr" // folder to look for the init file
$env:FLASK_ENV = "development" // automatically restart of server in case of changes

pip install flask_cors

// extract origin (see below)
// from 1_Requests_Starter/readme.md
cd backend
python -m venv venv
// select virtual env interpreter in VCS by Ctrl+Shift+P
// open new terminal in VCS which executes the activate script
pip install -r requirements.txt
// decommented #psycopg2-binary==2.8.2, because compilation with VC++ fails
psql.exe -h uweheuer.spdns.de -U postgres postgres
  \i setup.sql
\psql.exe -h uweheuer.spdns.de -U student -d bookshelf -f books.psql
pip install psycopg2-binary
// replace C:\Uwes\python\UdacityFullWebDeveloper\cd0037-API-Development-and-Documentation-exercises-master\1_Requests_Starter\backend\venv\Lib\site-packages\sqlalchemy\util\compat.py
// row 331 time_func = time.clock -> time_func = time.time
python -m flask run
Origin

Trivia App

cd C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\02_trivia_api\starter\backend 

python -m venv venv
.\venv\Scripts\activate
// decommented #psycopg2-binary==2.8.2, because compilation with VC++ fails
pip install -r requirements.txt
C:\Uwes\Programme\PostgreSQL\12\bin\psql.exe -h uweheuer.spdns.de -U postgres postgres
  CREATE DATABASE trivia;
  CREATE USER trivia WITH ENCRYPTED PASSWORD 'trivia';
  GRANT ALL PRIVILEGES ON DATABASE trivia TO trivia;
  CREATE DATABASE trivia_test;
  CREATE USER trivia_test WITH ENCRYPTED PASSWORD 'trivia_test';
  GRANT ALL PRIVILEGES ON DATABASE trivia_test TO trivia_test;
C:\Uwes\Programme\PostgreSQL\12\bin\psql -U trivia -h uweheuer.spdns.de -f trivia.psql trivia
C:\Uwes\Programme\PostgreSQL\12\bin\psql -U trivia_test -h uweheuer.spdns.de -f trivia.psql trivia_test
// replace in C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\02_trivia_api\starter\backend\venv\Lib\site-packages\sqlalchemy\util\compat.py row 331
#time_func = time.clock
time_func = time.time
C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\02_trivia_api\starter\frontend> npm install

// to run the app
.\venv\Scripts\activate
$env:FLASK_APP = "flaskr" // folder to look for the init file
$env:FLASK_ENV = "development" // automatically restart of server in case of changes
$env:DB_HOST = "uweheuer.spdns.de:5432"
python -m flask run
C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\02_trivia_api\starter\frontend> npm start

// to run the tests
python .\test_flaskr.py
Upload for Check
pip freeze > requirements.txt
Solutions

ToDo App

Fyyur App

  • run as prepartion (rest was already installed for lesson)
pip3 install virtualenv
npm install bootstrap@3
// creating .env file for VSC debugging
  • execute Development Setup from GitHub
  • with psql
create database fyyur;
create user fyyur with encrypted password 'fyyur';
grant all privileges on database fyyur to fyyur;
  • in C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\01_fyyur\starter_code
python -m virtualenv env
  • to run the app
C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\01_fyyur\starter_code>.\env\Scripts\activate
pip install Flask-migrate // one-time
pip3 install psycopg2 // one-time
flask db init // one-time

flask db migrate // if needed

C:\Uwes\python\UdacityFullWebDeveloper\FSND\projects\01_fyyur\starter_code>.\env\Scripts\activate
#$env:FLASK_ENV = "development" // moved to .env file
python .\app.py
http://127.0.0.1:5000/
Origin
Solutions
Running System


AWS Project

Create Admin User for AWS CLI
  • IAM Dashboard -> Add User
  • no tags
  • create user
  • download CSV to C:\Uwes\python\UdacityFullWebDeveloper\AWS\
  • this adds sections in C:\Users\U1728\.aws\config and C:\Users\U1728\.aws\credentials.
  • check configuration by
  • S3 Dashboard -> Create Bucket 'uweheuerudacity1'
  • and by AWS CLI
  • PS C:\Users\U1728> aws s3api delete-bucket --bucket uweheuerudacity2 --profile aws_cli_profile
Prepare the Project via AWS Console
  • find out the AWS acount id
aws sts get-caller-identity --profile aws_cli_profile
  • create files trust.json and iam-role-policy.json in C:\Uwes\python\UdacityFullWebDeveloper\AWS\ and run
aws iam create-role --role-name UdacityFlaskDeployCBKubectlRole --assume-role-policy-document file://trust.json --output text --query 'Role.Arn' --profile aws_cli_profile
aws iam put-role-policy --role-name UdacityFlaskDeployCBKubectlRole --policy-name eks-describe --policy-document file://iam-role-policy.json --profile aws_cli_profile
  • create role in IAM dashboard -> Next
  • -> Next
  • -> Create Role
  • attach AmazonEKSServicePolicy
  • create role 'uweheuerEKSWorkerNode' with permissions 'AmazonEKSWorkerNodePolicy, AmazonEC2ContainerRegistryReadOnly, AmazonEKS_CNI_Policy'
  • create SSH key pair 'uweheuerVMSSHPair' for the VMs by EC2 service -> Network & Security -> Key Pairs with ppk
  • Create cluster by 'EKS -> Clusters -> Add Cluster -> Create'
  • when the cluster is active add a node group
Prepare the Project via Command Line
eksctl get cluster
  • kubectl is installed via Docker
  • create demo cluster
eksctl create cluster --name eksctl-demo  [--profile <profile-name>]
  • create a public Docker repository named 'simple-flask' via the Docker Hub web portal
  • create a test flask application
C:\Uwes\python\UdacityFullWebDeveloper\AWS\simple flask application> git clone https://github.com/udacity/FSND-Deploy-Flask-App-to-Kubernetes-Using-EKS
  • start Docker
  • build Docker image
C:\Uwes\python\UdacityFullWebDeveloper\AWS\simple flask application\FSND-Deploy-Flask-App-to-Kubernetes-Using-EKS\examples\Deploy_Flask_App> docker build -t uweheuer/simple-flask .
  • push the image to the repository
C:\Uwes\python\UdacityFullWebDeveloper\AWS\simple flask application\FSND-Deploy-Flask-App-to-Kubernetes-Using-EKS\examples\Deploy_Flask_App> docker push uweheuer/simple-flask:latest
  • create C:\Uwes\python\UdacityFullWebDeveloper\AWS\deployment.yml
  • deploy the application to the cluster
 C:\Uwes\python\UdacityFullWebDeveloper\AWS> kubectl apply -f .\deployment.yml
  • check it by
C:\Uwes\python\UdacityFullWebDeveloper\AWS> kubectl cluster-info
  • clean up for the time being
C:\Uwes\python\UdacityFullWebDeveloper\AWS> kubectl delete deployments/simple-flask-deployment
C:\Uwes\python\UdacityFullWebDeveloper\AWS> eksctl delete cluster eksctl-demo
Final Project
Resources
Activities
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject> git clone https://github.com/UweHeuer/cd0157-Server-Deployment-and-Containerization.git
  • create a virtual environment
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> python -m venv venv
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> .\venv\Scripts\activate
(venv) PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> pip install -r .\requirements.txt
$env:JWT_SECRET = "myjwtsecret"
$env:JWT_SECRET = "DEBUG"
  • start the backend
python main.py
  • test from Powershell (replace token from the return of the first call)
PS C:\Uwes\Programme\curl\curl-7.76.1-win64-mingw\bin>.\curl.exe --data "{\""email\"":\""abc@xyz.com\"",\""password\"":\""mypwd\""}" --header "Content-Type: application/json" -X POST localhost:8080/auth | jq -r ".token"
PS C:\Uwes\Programme\curl\curl-7.76.1-win64-mingw\bin>.\curl.exe --request GET "http://localhost:8080/contents" -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NDU4MDkxMDIsIm5iZiI6MTY0NDU5OTUwMiwiZW1haWwiOiJhYmNAeHl6LmNvbSJ9.yuNDXFRHgterpUuvAqgILL4yrq4mLqaKOgWIB2vQFzQ" | jq .
  • build Docker image
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> docker build -t myimage .
  • run container on local port 81
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> docker run --name myContainer --env-file=.env_file -p 81:8080 myimage

docker container ls
docker ps
docker container stop <CONTAINER_ID>
docker container rm <CONTAINER_ID>
  • find out account data
aws sts get-caller-identity
{
   "UserId": "AIDA44LO4KKN3BXHFK4TO",
   "Account": "885532676763",
   "Arn": "arn:aws:iam::885532676763:user/Admin"
}
  • create an EKS cluster by
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> eksctl create cluster --name simple-jwt-api
... uses region us-west-2

PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> kubectl get nodes
  • role UdacityFlaskDeployCBKubectlRole will be used from preparation
  • allowing the role to access the cluster
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> kubectl get -n kube-system configmap/aws-auth -o yaml > /temp/aws-auth-patch.yml
  • copy the file to C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization
  • edit the file and patch the clusters config map by the following command (the access to the patch file modified, because the command from the tutorial did not work)
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> kubectl patch configmap/aws-auth -n kube-system --patch-file .\aws-auth-patch.yml
  • generate a GitHub Token by the GitHub web portal and store it in C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\MyInfos.txt
  • edit C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization\ci-cd-codepipeline.cfn.yml
  • create a stack by the file via the CloudFormation service dashboard -> Create Stack
  • edit C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization\buildspec.yml with JWT_SECRET
  • put the secret in the AWS Parameter Store by
aws ssm put-parameter --name JWT_SECRET --overwrite --value "UwesSecret" --type SecureString
  • add changes to git
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> git add .\Dockerfile
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> git add .\buildspec.yml
PS C:\Uwes\python\UdacityFullWebDeveloper\AWS\FinalProject\cd0157-Server-Deployment-and-Containerization> git commit -m Update1
  • get the url to connect to by
kubectl get services simple-jwt-api -o wide
NAME             TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)        AGE   SELECTOR
simple-jwt-api   LoadBalancer   10.100.107.241   ab375b7c2eb4f47838c051bbe449c108-485277684.us-west-2.elb.amazonaws.com   80:31272/TCP   34m   app=simple-jwt-api
  • test the encrpytion by
PS C:\Uwes\Programme\curl\curl-7.76.1-win64-mingw\bin> .\curl.exe --data "{\""email\"":\""abc@xyz.com\"",\""password\"":\""mypwd\""}" --header "Content-Type: application/json" -X POST ab375b7c2eb4f47838c051bbe449c108-485277684.us-west-2.elb.amazonaws.com/auth | jq -r ".token"
  • test decryption by using the output of the former command in
.\curl.exe --request GET "ab375b7c2eb4f47838c051bbe449c108-485277684.us-west-2.elb.amazonaws.com/contents" -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE2NDYwNjg0MzQsIm5iZiI6MTY0NDg1ODgzNCwiZW1haWwiOiJhYmNAeHl6LmNvbSJ9.QYEEoTwZd4OS1FCSPf20e7b5xOjEUFcaEXHCVqLTGI4" | jq .
Issues
  • AWS Portal screen shots out-dated
  • description out-dated and not complete for Windows
  • roles may not be deleted
  • missing diagrams
  • chocolately admin rights
  • windows commands did not work (see patching the clusters config map)

CapStone

Resources
Implementation
  • download starter as ZIP and copy /heroku_sample/starter to C:\Uwes\python\UdacityFullWebDeveloper\UweHeuer_Final_Project_Capstone
  • create a repository
PS C:\Uwes\python\UdacityFullWebDeveloper\UweHeuer_Final_Project_Capstone> git init
PS C:\Uwes\python\UdacityFullWebDeveloper\UweHeuer_Final_Project_Capstone> git add .
PS C:\Uwes\python\UdacityFullWebDeveloper\UweHeuer_Final_Project_Capstone> git config --global user.email uwe.heuer@gmail.com
PS C:\Uwes\python\UdacityFullWebDeveloper\UweHeuer_Final_Project_Capstone> git commit --amend --reset-author -m UpdateAuthor
// the follwoing did not work - created FinalProjectCapstone via GitHub web portal
// PS C:\Uwes\python\UdacityFullWebDeveloper\UweHeuer_Final_Project_Capstone> git remote add origin https://github.com/UweHeuer/FinalProjectCapstone.git
git push -u origin master
Retrieved from "http://uweheuer.spdns.de/mediawiki_new/index.php?title=Udacity&oldid=9787"