Uweheuer: Difference between revisions

From Wiki RB4
 
(122 intermediate revisions by 3 users not shown)
Line 1: Line 1:
==Concept==
==Concept==
===Encryption===
===Encryption===
Notes are encrypted by own algorithm implemented in Present.js, bookmark comments are encrypted by AES. Client-side decryption is done by CryptoJS (in detail aes.js and mode-ecb-min.js are used from the downloaded zip-file).
====Notes====
Notes are encrypted (and decrypted) by own algorithm implemented in Present.js and Present.java. Encryption and decryption (<code>wrap(), unwrap()</code>) works by a logical OR of a single character with a token (indiscernible) and represent the result as a 3 letter block consisting of a the character code with base 25 and filling the rest up randomly with a character of a list of p-z.  


#fool = user
#fool = user
Line 10: Line 11:
#notepresent = package of present and camouflaged note
#notepresent = package of present and camouflaged note
#present = package of fool, haze, camouflage of haze; used to verify if the notepresent sent was camouflaged correctly and valid, by checking if the unwrapped camouflage is equal to haze
#present = package of fool, haze, camouflage of haze; used to verify if the notepresent sent was camouflaged correctly and valid, by checking if the unwrapped camouflage is equal to haze
To check if the service callers are authorized, the actual timestamp is send uncrypted (haze) and encrypted (camouflage). On the server it is checked that the decrypted camouflage is equal to haze.
====Bookmarks====
All bookmarks data (menu name and comment, as well as url name, link and comments) are en- and decrypted by AES (see <code>clientserver.Crypt.java</code>) and also stored as [[EnAndDecoding#Base64Encoding|Base64]] in DB. To read the data or to test encryption there are two [[Uweheuer#Encryption_REST_Services|REST services]].
For encrypting the communication with client via REST services the samee encryption as for [[Uweheuer#Notes|notes]] is used.
===App Cache===
* Application Cache API manifest selection is deprecated
* Service Workers
* Cache Storage API / global caches object
* sw-toolbox and sw-precache


===Bookmarks===
===Bookmarks===


* root node identified by name = 'My Bookmarks' or id = 1
* root node identified by name = 'My Bookmarks' or id = 1
====Backup====
C:\Uwes\Batches\RaspberryBookmarksToLocalhost.bat
====Data Model====
* see diagramm <code>Bookmarks Java Classes</code>


====Tree Versioning====
====Tree Versioning====


# bookmark tree version is set to latest update version
# bookmark tree version and random ID (client ID) is stored in the root node of an tree
# bookmark tree version is set to latest update or create action and stored at the root tree element
# bookmark random ID (client ID) is set randomly after loading a tree from the DB
# an update/save of a bookmark tree is only valid if the tree version of the tree to be saved is identical to last update or create action or the random ID (client ID) of the tree to be saved is identical to the last update or create action (the second is needed because multiple saves by one client outdates the tree version on the client)


====Tooltip====
====Tooltip====
* implemented by using [http://stevenbenner.github.io/jquery-powertip/ PowerTip]
* implemented by using [http://stevenbenner.github.io/jquery-powertip/ PowerTip]
====Online Bookmarks====
* see sequence diagram <code>BookmarksApp</code>
====Offline Bookmarks====
* see sequence diagram <code>CachedBookmarksApp</code>
=====Menu Implementation=====
* in <code>BookmarksController.renderCachedBookmarks()</code> items which have a menu are assigned a specific class (<code>cachedUrl, cachedMenu, cachedMenuHeader</code>)
* there is <code>tapholdXXX()</code>-method registered via JQuery selector and <code>on()</code> for taphold-event and this class


===Notes===
===Notes===
JavaScript application design is based on C:\Uwes\Documents\Software_Development\Programming\Languages\JavaScript\JQuery\How to Build a jQuery Mobile Application v1.1.pdf. See sequence diagram <code>NotesApp</code>.
====Note Categories====
* add column to note table:
ALTER TABLE `note` ADD `category_id` BIGINT(20) NOT NULL DEFAULT '0' ;
* add table 'notecategory' by importing
\src\notecategory.sql
* categories can be selected for a note in the note editor, the id of the category is saved as attribute of a note
* after creation a new category gets a random negative id which is updated with the server id in the local storage after the successful Ajax save call
* see also Object Diagramm <code>Notes JavaScript Objects</code>
====Backup Notes====
* notes are copied from the [[Server4You|Server]] to [[EONISLaptopHPNew|Laptop]] by <code>SynchronizeHostsTask->backupNotesFromVServer4You()</code> which calls
c:\Uwes\Batches\RaspberryNotesToLocalhost.bat


====Saving a note====
====Saving a note====
* see sequence diagram onSaveNoteButtonTapped()
* see sequence diagram onSaveNoteButtonTapped()
===Management===
====DisAndEnable Application Service====
* see /server/rest/ManagementService.java restartApp() method
* executes AppConfiguration.getRestartAppCommand() with is stored and encrypted in <JBOSS_DIR>/standalone/data/uweheuer/Configuration.properties
** on Laptop:
cmd /c c:\\Uwes\\Batches\\DisAndEnableUweHeuerLaptop.bat
** on HomePC:
/myprogs/DisAndEnableUweHeuer.bat
====Reboot Service====
* see /server/rest/ManagementService.java rebootServer() method
* see [[#Runtime_Configuration|Runtime Configuration]]
====Sync Services====
* see /server/rest/ManagementService.java getSyncStatus() method
===Runtime Configuration===
* When deployed, the name of the WAR becomes, by default, the root path of the web application
* managed by class <code>AppConfiguration</code>
* some parameters are read from the configuration file <code>Configuration.properties</code> in <JBOSS_DIR>\standalone\data\uweheuer directory, data stored AES encrypted Crypt and decrypted by <code>/clientserver/Crypt.java</code>, encryption of entries can be created by Eclipse project <code>CryptTest</code>
* some parameters are set by string replacement in <code>AppConfiguration.java</code> during ant build before compilation
** SAFE_MODE // which controls the <code>Present.check()</code>, in case it its true, the method does not perform any check
===Libraries===
====JavaScript====
* [https://github.com/jbrooksuk/jQuery-Timer-Plugin/ JQuery Timer Plugin]
* [[JStorage|JStorage]]
==ToDos==
* encrypt menu comments => encrypted everything
* client error messages (e.g. at saving, ...) presentation and OK button
* put a salt into encryption in order empty string not always be the same representation in DB
* Authentification
* JS framework
** responsiveness
** drag & drop
** tree


==Documentation==
==Documentation==
# s. <PROJECT_DIR>/doc/uweheuer.vsd
# s. <PROJECT_DIR>/doc/uweheuer.vsd
# s. in eclipse project 'uweheuer_new' sub folder 'vpproject/diagrams' (Visual Paradigm Plus)
## Build and Deployment Process -> see sheet 'Generation'
## Client Server Communication
# s. <PROJECT_DIR>/vpproject/uweheuer.vpp (from eclipse Right Click -> 'Open with' -> 'System Editor')
## Synchronization
 
===Page Flow===
see state diagram <code>Page Flow</code>:
 
[[File:UweHeuerPageFlow.JPG]]


===Interfaces===
===Interfaces===


====Web Services====
====Web Services====
Web Services are implemented with [[Axis|Axis]].
Web Services are implemented with [[Axis#Adding_a_WebService_to_a_WebApp|Axis]]. The source files for the classes are in package <code>server.webservices</code>. Callbacks and stubs have been generated long time ago (see automatic comment in generated files) and needed for client implementations. The application acts as a web service client while doing synchronization. WSDL files are generated by target '''gen_wsdl''' but are not used. The mechanism used (Axis Servlet) is described [http://www.developer.com/services/article.php/3777111/Embedding-Apache-Axis2-into-Existing-Applications.htm here].


==Initial Installation==
====[[Rest]] Services====
Create and fill database mysql505 per mysql tools. Copy war-file or directory tree to the webapp directory of the application server. Adjust Apache or Tomcat for proxying to JBoss ([[HomePCNew#Apache]])
[[Rest]] services are implemented in package <code>server.rest</code>. The call path to the services is set in class <code>JAXRSActivator</code>.


==Persistence==
=====Bookmarks REST Services=====
* DB configuration see <PROJECT_DIR>\src\WEB-INF\classes\META-INF\[[JavaJEEJPA#persistence.xml|persistence.xml]] and [[JBoss#Database_Configuration|JBoss configuration]], which defines the driver and the database connection string, which includes the DB name
Bookmarks Rest services are implemented in class <code>BookmarkTreeService</code>.
 
=====Encryption REST Services=====
The services are implemented in class <code>server.rest.EncryptionServices</code>.
* to encrypt all bookmarks data (handle carefully, should only be called once)
* to encrypt a string
* to decrypt a string (the + character has to be replaced by %2B)
 
==Installation and Configuration==
Create and fill database mysql505 per mysql tools. Copy war-file or directory tree to the webapp directory of the application server. Adjust Apache or Tomcat for proxying to JBoss (see e.g. [[HomePCNew#Apache|Apache on HomePC]]).
 
===Persistence===
* Data source configuration see <PROJECT_DIR>\src\WEB-INF\classes\META-INF\[[JavaJEEJPA#persistence.xml|persistence.xml]]
* DB configuration see [[JBoss#Database_Configuration|JBoss configuration]], which defines the driver and the database connection string
* EntityManagementUtil creates two EntityManagerFactories, one for '''bookmarks''' (definition see persistence.xml) and one for '''uweswiki''' (definition see persistence.xml)
* EntityManagementUtil creates two EntityManagerFactories, one for '''bookmarks''' (definition see persistence.xml) and one for '''uweswiki''' (definition see persistence.xml)
===Computers===
====[[HomePCNew#UweHeuer|HomePC New]]====
====[[EDTLaptop1#Wildfly|Eon ELITEBOOK]]====


==Synchronization==
==Synchronization==
* see sequence [[Uweheuer#Documentation|diagram]] 'Synchronization'
* see sequence [[Uweheuer#Documentation|diagram]] 'Synchronize Data'
* init configuration in web.xml
* init configuration in [[JEE#Deployment_Descriptor|deployment descriptor web.xml]]


==Releases==
==Releases==
Line 53: Line 160:
* Migration from Hibernate to JPA implemented by Hibernate
* Migration from Hibernate to JPA implemented by Hibernate
* Migration from Runtime JBoss 4.2 to JBoss 7.1
* Migration from Runtime JBoss 4.2 to JBoss 7.1
==Test Cases==
* add URL
* add menu
* search url and menu


==Operation==
==Operation==


==Generierung and Deployment==
==Generierung and Deployment==
Die Generierung erfolgt über Ant (s. build.xml and [[#Documentation|Documentation->Build Process]]).
Die Generierung erfolgt über Ant (s. build.xml and [[#Documentation|Documentation -> Build and Deployment Process]]).


===Shell===
===Shell===
  '''ant''' ['''deploy_laptop_jboss7'''|'''deploy_home'''|'''deploy_vserverjboss7''']
  '''ant''' ['''deploy_raspberry_wildfly'''|'''deploy_laptop'''|'''deploy_home'''|'''deploy_vserverjboss7''']


===Eclipse===
===Eclipse===
''Project -> Build Project'' ruft das target deploy_laptop auf.
''Project -> Build Project'' ruft das default target 'deploy_laptop_jboss7' auf.


===Update Applet===
===Update Applet===
Line 82: Line 194:
===Update Contacts on Local Laptop===
===Update Contacts on Local Laptop===
* open C:\Uwes\WinScripts\ExportOutlookContactsbyVBA4Excel.xlsm
* open C:\Uwes\WinScripts\ExportOutlookContactsbyVBA4Excel.xlsm
* open Entwicklertools->Visual Basic
* open Entwicklertools->Visual Basic and start Makro OutlookContactsToExcelWorksheet2 per Icon in top menu bar or
* start Makro OutlookContactsToExcelWorksheet2 per Icon in top menu bar => this will create the file C:\Uwes\Programme\Java\JBoss\jboss-eap-6.1.0\jboss-eap-6.1\standalone\data\uweheuer\OutlookContactExportbyVBA4Excel.csv  
* Ansicht -> Makros
* call web service http://localhost/uweheuer/services/ContactServices?UpdateContactsFromServerFile
=> this will create the file C:\Uwes\Programme\Java\JBoss\standalone\data\uweheuer\OutlookContactExportbyVBA4Excel.csv  
* call web service http://localhost:8090/uweheuer/services/ContactServices?UpdateContactsFromServerFile


==Logging and Debuging==
==Logging and Debuging==


===Logging===
===Server Logging===
Configuration is done via log4j. Configuration in JBoss 7 [[JBoss#Logging_configuration|here]] done by adding a uweheuer log category and a UWEHEUER_FILE log handler. The coding uses '''LogUtil.java''' an the static method. It creates a logger named '''uweheuer''' which is used through out the code.  
Server-side logging configuration is done via configuration in [[JBoss#Logging_Configuration|JBoss]]. The configuration is (details of Raspberry Wildfly 11 see [[:File:RaspberryUweHeuerLoggerConfiguration.pdf|here]]):
* Configuration->Core->Logging->Log category 'uweheuer', Log Level 'INFO', handler 'uweheuer_periodic_file_handler'
* Configuration->Core->Logging->Periodic handler 'uweheuer_periodic_file_handler', log level 'INFO', formatter '%d{HH:mm:ss,SSS} %-5p [%c] %C.%M() %s%E%n', ...
 
The coding uses '''LogUtil.java''' an the static method. It creates a logger named '''uweheuer''' which is used through out the code and which has the same name as the log category.
 
===Client Logging===
Logging on the '''client side''' is implemented in <code>MyLog.js</code>, which just has to be included to use it. There are for methods to be uses:
*myLog.info(<message>, <funcname>);
*myLog.warn(<message>, <funcname>);
*myLog.error(<message>, <funcname>);
*myLog.log(<message); // obsolete


Logging on the '''client side''' is done via the printDebug() method, which goes at least to the java console.
Logging always goes to console, in addition also to a special div (var view, setView();) within a window or a seperate window (showLogWindow();). Because the seperate window gets the focus when displaying a message it should be used with the split view of Chrome.


===Eclipse Debugging===
===Eclipse Debugging===

Latest revision as of 09:04, 7 July 2020

Concept[edit]

Encryption[edit]

Notes[edit]

Notes are encrypted (and decrypted) by own algorithm implemented in Present.js and Present.java. Encryption and decryption (wrap(), unwrap()) works by a logical OR of a single character with a token (indiscernible) and represent the result as a 3 letter block consisting of a the character code with base 25 and filling the rest up randomly with a character of a list of p-z.

  1. fool = user
  2. indiscernible = (unsichtbar) private token to hash a note
  3. camouflage = (Tarnung); hashed string; consists of chunks of 3 chars, the last chars are the XOR of string to hash and key converted to base 25 filled up to 3 with random letters not used in base.
  4. haze = (Nebel) actual timestamp
  5. bft = (AES) private token to hash bookmark comments (JS: amFrb2J0aGltbzEyMzQ1Ng==, Java on server and client (Crypt.java): jakobthimo123456)
  6. notepresent = package of present and camouflaged note
  7. present = package of fool, haze, camouflage of haze; used to verify if the notepresent sent was camouflaged correctly and valid, by checking if the unwrapped camouflage is equal to haze

To check if the service callers are authorized, the actual timestamp is send uncrypted (haze) and encrypted (camouflage). On the server it is checked that the decrypted camouflage is equal to haze.

Bookmarks[edit]

All bookmarks data (menu name and comment, as well as url name, link and comments) are en- and decrypted by AES (see clientserver.Crypt.java) and also stored as Base64 in DB. To read the data or to test encryption there are two REST services.

For encrypting the communication with client via REST services the samee encryption as for notes is used.

App Cache[edit]

  • Application Cache API manifest selection is deprecated
  • Service Workers
  • Cache Storage API / global caches object
  • sw-toolbox and sw-precache

Bookmarks[edit]

  • root node identified by name = 'My Bookmarks' or id = 1

Backup[edit]

C:\Uwes\Batches\RaspberryBookmarksToLocalhost.bat

Data Model[edit]

  • see diagramm Bookmarks Java Classes

Tree Versioning[edit]

  1. bookmark tree version and random ID (client ID) is stored in the root node of an tree
  2. bookmark tree version is set to latest update or create action and stored at the root tree element
  3. bookmark random ID (client ID) is set randomly after loading a tree from the DB
  4. an update/save of a bookmark tree is only valid if the tree version of the tree to be saved is identical to last update or create action or the random ID (client ID) of the tree to be saved is identical to the last update or create action (the second is needed because multiple saves by one client outdates the tree version on the client)

Tooltip[edit]

Online Bookmarks[edit]

  • see sequence diagram BookmarksApp

Offline Bookmarks[edit]

  • see sequence diagram CachedBookmarksApp
Menu Implementation[edit]
  • in BookmarksController.renderCachedBookmarks() items which have a menu are assigned a specific class (cachedUrl, cachedMenu, cachedMenuHeader)
  • there is tapholdXXX()-method registered via JQuery selector and on() for taphold-event and this class

Notes[edit]

JavaScript application design is based on C:\Uwes\Documents\Software_Development\Programming\Languages\JavaScript\JQuery\How to Build a jQuery Mobile Application v1.1.pdf. See sequence diagram NotesApp.

Note Categories[edit]

  • add column to note table:
ALTER TABLE `note` ADD `category_id` BIGINT(20) NOT NULL DEFAULT '0' ;
  • add table 'notecategory' by importing
\src\notecategory.sql
  • categories can be selected for a note in the note editor, the id of the category is saved as attribute of a note
  • after creation a new category gets a random negative id which is updated with the server id in the local storage after the successful Ajax save call
  • see also Object Diagramm Notes JavaScript Objects

Backup Notes[edit]

  • notes are copied from the Server to Laptop by SynchronizeHostsTask->backupNotesFromVServer4You() which calls
c:\Uwes\Batches\RaspberryNotesToLocalhost.bat

Saving a note[edit]

  • see sequence diagram onSaveNoteButtonTapped()

Management[edit]

DisAndEnable Application Service[edit]

  • see /server/rest/ManagementService.java restartApp() method
  • executes AppConfiguration.getRestartAppCommand() with is stored and encrypted in <JBOSS_DIR>/standalone/data/uweheuer/Configuration.properties
    • on Laptop:
cmd /c c:\\Uwes\\Batches\\DisAndEnableUweHeuerLaptop.bat
    • on HomePC:
/myprogs/DisAndEnableUweHeuer.bat

Reboot Service[edit]

Sync Services[edit]

  • see /server/rest/ManagementService.java getSyncStatus() method

Runtime Configuration[edit]

  • When deployed, the name of the WAR becomes, by default, the root path of the web application
  • managed by class AppConfiguration
  • some parameters are read from the configuration file Configuration.properties in <JBOSS_DIR>\standalone\data\uweheuer directory, data stored AES encrypted Crypt and decrypted by /clientserver/Crypt.java, encryption of entries can be created by Eclipse project CryptTest
  • some parameters are set by string replacement in AppConfiguration.java during ant build before compilation
    • SAFE_MODE // which controls the Present.check(), in case it its true, the method does not perform any check

Libraries[edit]

JavaScript[edit]

ToDos[edit]

  • encrypt menu comments => encrypted everything
  • client error messages (e.g. at saving, ...) presentation and OK button
  • put a salt into encryption in order empty string not always be the same representation in DB
  • Authentification
  • JS framework
    • responsiveness
    • drag & drop
    • tree

Documentation[edit]

  1. s. <PROJECT_DIR>/doc/uweheuer.vsd
    1. Build and Deployment Process -> see sheet 'Generation'
  2. s. <PROJECT_DIR>/vpproject/uweheuer.vpp (from eclipse Right Click -> 'Open with' -> 'System Editor')

Page Flow[edit]

see state diagram Page Flow:

Interfaces[edit]

Web Services[edit]

Web Services are implemented with Axis. The source files for the classes are in package server.webservices. Callbacks and stubs have been generated long time ago (see automatic comment in generated files) and needed for client implementations. The application acts as a web service client while doing synchronization. WSDL files are generated by target gen_wsdl but are not used. The mechanism used (Axis Servlet) is described here.

Rest Services[edit]

Rest services are implemented in package server.rest. The call path to the services is set in class JAXRSActivator.

Bookmarks REST Services[edit]

Bookmarks Rest services are implemented in class BookmarkTreeService.

Encryption REST Services[edit]

The services are implemented in class server.rest.EncryptionServices.

  • to encrypt all bookmarks data (handle carefully, should only be called once)
  • to encrypt a string
  • to decrypt a string (the + character has to be replaced by %2B)

Installation and Configuration[edit]

Create and fill database mysql505 per mysql tools. Copy war-file or directory tree to the webapp directory of the application server. Adjust Apache or Tomcat for proxying to JBoss (see e.g. Apache on HomePC).

Persistence[edit]

  • Data source configuration see <PROJECT_DIR>\src\WEB-INF\classes\META-INF\persistence.xml
  • DB configuration see JBoss configuration, which defines the driver and the database connection string
  • EntityManagementUtil creates two EntityManagerFactories, one for bookmarks (definition see persistence.xml) and one for uweswiki (definition see persistence.xml)

Computers[edit]

HomePC New[edit]

Eon ELITEBOOK[edit]

Synchronization[edit]

Releases[edit]

  • Migration from Axis 1 to Axis 2
  • Migration from Hibernate to JPA implemented by Hibernate
  • Migration from Runtime JBoss 4.2 to JBoss 7.1

Test Cases[edit]

  • add URL
  • add menu
  • search url and menu

Operation[edit]

Generierung and Deployment[edit]

Die Generierung erfolgt über Ant (s. build.xml and Documentation -> Build and Deployment Process).

Shell[edit]

ant [deploy_raspberry_wildfly|deploy_laptop|deploy_home|deploy_vserverjboss7]

Eclipse[edit]

Project -> Build Project ruft das default target 'deploy_laptop_jboss7' auf.

Update Applet[edit]

  • press x (clear class loader cache) in java console
  • press 'Bookmark Applets' in menu

Crypt Data Handling[edit]

Die Generierung eines leeren Data-Files namens 'data.crypt' erfolgt über das Kommando:

C:\Uwes\eclipse\workspace\uweheuer\build\class>java server.CreateCryptDataFile.

Die Datei muss in das Verzeichnis <TOMCAT_HOME>/webapps/uweheuer/data kopiert werden z.B. mit

C:\Uwes\eclipse\workspace\uweheuer\build\class>copy data.crypt \Uwes\xampp\tomcat\webapps\uweheuer\data

Contacts[edit]

Update Contacts on Local Laptop[edit]

  • open C:\Uwes\WinScripts\ExportOutlookContactsbyVBA4Excel.xlsm
  • open Entwicklertools->Visual Basic and start Makro OutlookContactsToExcelWorksheet2 per Icon in top menu bar or
  • Ansicht -> Makros

=> this will create the file C:\Uwes\Programme\Java\JBoss\standalone\data\uweheuer\OutlookContactExportbyVBA4Excel.csv

Logging and Debuging[edit]

Server Logging[edit]

Server-side logging configuration is done via configuration in JBoss. The configuration is (details of Raspberry Wildfly 11 see here):

  • Configuration->Core->Logging->Log category 'uweheuer', Log Level 'INFO', handler 'uweheuer_periodic_file_handler'
  • Configuration->Core->Logging->Periodic handler 'uweheuer_periodic_file_handler', log level 'INFO', formatter '%d{HH:mm:ss,SSS} %-5p [%c] %C.%M() %s%E%n', ...

The coding uses LogUtil.java an the static method. It creates a logger named uweheuer which is used through out the code and which has the same name as the log category.

Client Logging[edit]

Logging on the client side is implemented in MyLog.js, which just has to be included to use it. There are for methods to be uses:

  • myLog.info(<message>, <funcname>);
  • myLog.warn(<message>, <funcname>);
  • myLog.error(<message>, <funcname>);
  • myLog.log(<message); // obsolete

Logging always goes to console, in addition also to a special div (var view, setView();) within a window or a seperate window (showLogWindow();). Because the seperate window gets the focus when displaying a message it should be used with the split view of Chrome.

Eclipse Debugging[edit]

  • debug server by JBoss
  • debug applet by Run -> Debug configurations -> Java Applet -> Debug TunnelAppletx