Issue 17 (#24)

* Testing update

* Clean

* Adding interpreter

* Disable checking compiler

* Cmd

* terminal work

* Missing files

* Fixing

* Found bug - fixed

* Fixes

* Fixes

* Szlify interfesju

* Dump functionality

* Fixed last missed byte in read via fstream

* New layout

* Colors on termina

* Cleaning terminal

* Fix ut and cleaning Buffer before kick off

* xrdb fixes

* Float and Double - intro

* Set status in terminal:

* Payload correct form of template

* Architectural change

* Fixes

* Reverse iterator support

* Fixes

* Fixes on help

* Return values of get/set

* Missing template materialization

* Fast access enabled

* Revert "Fast access enabled"

This reverts commit 5d06fca.

* Cmake

* Permanent posix desciptor - object

* Byte array added

* Byte array

* Fix help in terminal

* Readme for rdb

* Update of readme

* Kicking out CBuffer

* Changes - prep kick off

* Revert to get proper dsp example

* Fix for conflicting name

* Fix

* CI update and ut name change

* CI update - fix

* Update README.md

conflicting xrdb name fix

* Add description of RDB in UML

* UML fix

* UML desc

* Markdown show

* Diagram in MD - Show

* Fail on merge support

* revert

* dir struct

* UML of xtrdb and clean of main.cpp of xtrdb

* Typos

* First working POC w/o CBuffer

* Remove of CBuffer

* Fixes in Readme, Cleaning legacy cbuff

* Remove of unused dump function

* Description of methods.
Cleaning.

* Encasuplation process

* Encapsulation

* Code quality fixes

* Embold suppres

* Code quality changes

* Parser atempt

* Staging of ANTLR4

* Antlr4 conan depenecy is on

* Change scope of Embold analysis.

Parser is code generated
Stage is Staging area not for production
Scripts are coming back - they are solid part of code.

* G4 - work on grammar

* Obsolete readme

* Work on parser

* Work on new parser

* Parser work

* Add new type

* Fix

* Extend test area

* fix on rdb

* Regressiont test on xtrdb

* x is ignored

* Work on Parser

* Work on parser

* Fix on call string

* Work on parser

* xrdb failure catch

* Adding error support in parser

* Listener aproach to parser

* Removing unused visitor - remain listener

* Enumeratation inside listener

* work on parser

* work on json

* Typo

* Fix generator

* Put changes

* Conflict marker remove

* Work on parser

* cat on conan profile

* working

* Syntax error drops out

* Work

* fix - no more conan source step

* Work

* Work

* Mandatory filename on declare

* Work with book

* Move of coreInstance into qstruct cpp

* Work on parser

* Work on compiler

* Astyle support

* Work on compiler

* Astyle

* case insnsitve keywords

* example 5 - similar

* work on type definitions

* RQL work on types

* Work on rdb unification

* Cleaning not used parser part

* Fix on readme

* Fix on funtion call

* examples compatible with old parser

* Add agse support

* Cover all examples

* Fixing bug in compiler affects UT pattern sequence

* astyle

* Polishing

* Move stage into compiler

* Change cpp into regression

* Fix on mod/div streams operators

* Astyle whole

* Remove unused class

* Parser add to embold

* Exclude generated code by antlr form embold

* Additional pattern covering Agse

* fix on config.yml

* Fix on config yaml

* Fix on config yaml

* Update conanfile.py

* Update config.yml

* Update config.yml

* Work in conan

* Set default options

* Prepare for Antlr4 - 4.10

* Fix for option parameter via conan install .. -o antlr4=4.10

* Antlr4.10.1

* Issue 17 - cover pattern

* Add storage parameter - no support yet

* Change time of testing branch

* Cut off legacy non used code

* Add rdb into astyle and path support

* Fix for wrong resource storage allocation

* Used cmake format

* CLang support

* Add clang to ci

* Work on clang force

* fix for clang ci

* Build missing parts on clang conan profile

* large resource

* Move clang edge build to nightly

* issue 17 - work on core problem

* Build compiler signature included in executable

* Add astyle for cmake

* Smoke test

* Make config line more convinient

* Fix for vim and exmaple that fails on #17

* Fix on log line

* Remove namespace contamination from header

* Remove std dependency from processor.cpp

* Removing std deps

* Ugly bug with parsing types mismatch - catched on plot-simple

* Appearently dsp algo works fine!

* Remove config.h from version control

* remove config.h from version control

* Trying to remove

* remove redunant git ignore files

* Modify gitignore

* Clean

* Remove std contamination

* Remove std contamination

* Missing .antl ignore

* Remove astyle from Parser generator result

* Fix for missing = in make astyle

* Polishing bash scripts

* Remove rest of std contaminated code

* full scan of cmakes

* Make more relative paths in main cmake

* Try include antlr into CI

* 2nd attepmpt

* Handle of std::system result from Dot

* Set default mode to JSON

* Remove unused test

* Storage for examples

* Remove one step in edge ci

* gitignore - work on temp

* Lambdas in loops

* 20 req

* Force c++20 in 9.4 compiler

* Fix for 2.0

* Fix for 2.0

* Revert of C++20 features

* Trying gcc 10 instead of 9.4 on CI

* Fix

* Trying to purge 4.9 from edge

* Trying to purge 4.9 from edge

* Removing redundant method of ipc communication

* change result of getblock from int to bool

* Bump boost version

* get into readData

* Added type desciption into xdumper

* Refactor - remove rendunant call

* Fix dumper

* Build timestamp

* Refactor of dbstream.* files

* testing

* Remove ugly enum generator with something more maintainable

* Fix astyle

* Identation of defines and incorporation of define in tokenDef.h

* eType same as tokens

* new line missing

* brackets on vectors

* cformat intro

* Fix clang

* if cascade to swtich case pattern

* cformat

* Definitions name fix

* Unifiy types stage 1

* String to STRING

* Unifiy types stage 2

* Unifiy types stage 3

* Unifiy types stage 4

* cformat

* fix on idnet

* Fix on GetFieldType

* tuple serialization

* Remove nameset

* remove of fieldName from field

* remove of dFieldType and use fieldDesc

* Purge fetFirstFieldName function

* work on schema

* Work

* Remove ununsed function

.gitignore

@@ -1,25 +1,31 @@
 CMakeCache.txt
 CMakeFiles/*
+CTestTestfile.cmake
+CMakeLists.txt.user
+cmake_install.cmake
+src/include/config.h
+src/compiler/.antlr
+examples/generator/CMakeFiles
+examples/generator/xgenerator
+examples/temp/
 .idea/*
 cmake-build-debug/*
-CMakeLists.txt.user
 Makefile
-cmake_install.cmake
 *.cbp
 out.*
-query.dmp
-query.lkn
-query.qry*
 testfile
-CTestTestfile.cmake
 Testing/
-thesis-data/
-query.dot
-query.jpg
 nohup.out
 build/
 scripts/CMakeFiles/*
 .vscode/
 astyle/
 compile_commands.json
 install_manifest.txt
+*.dmp
+*.lkn
+*.qry*
+*.dot
+*.jpg
+*.png
+*.swp

CMakeLists.txt

@@ -1,9 +1,13 @@
-cmake_minimum_required(VERSION 3.3 FATAL_ERROR )
+cmake_minimum_required(VERSION 3.3 FATAL_ERROR)
 
-project (retractordb)
+project(retractordb)
 
 enable_testing()
 
+set(CONAN_DISABLE_CHECK_COMPILER TRUE)
+
+set(CMAKE_CXX_STANDARD 20)
+
 include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
 conan_basic_setup()
 

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2008-2021 Michal Widera
+Copyright (c) 2008-2022 Michal Widera
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in

README.md

@@ -21,21 +21,22 @@ Commands
 How to install
 -----------------------------------------------------------------------
 
+NOTE: If you are using conan fist time plese create default conan profile
+```
+conan profile new default --detect
+```
+
 ```
 sudo apt-get -y install gcc cmake make build-essential python3 python3-pip
 pip install conan
 conan profile update settings.compiler.libcxx=libstdc++11 default
 git clone https://github.com/michalwidera/retractordb.git
 cd build
-conan source ..
 conan install ..
 conan build ..
 make test
 ```
-If you are using conan fist time plese create default conan profile
-```
-conan profile new default --detect
-```
+
 
 After installation _xretractor_, _xqry_ ... etc will be installed in ~/.local/bin on path.
 Please check proper installation by typing in command prompt - for instance: _xqry -h_
@@ -84,4 +85,4 @@ Project created in 2003-2022 by Michal Widera
 
 
 ## License
-[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fmichalwidera%2Fretractordb.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fmichalwidera%2Fretractordb?ref=badge_large)
+[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fmichalwidera%2Fretractordb.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fmichalwidera%2Fretractordb?ref=badge_large)

embold.yaml

@@ -0,0 +1,30 @@
+version: 1
+
+# Please visit https://docs.embold.io/v2-quick-start-guide/#default-configurations
+#  to know more about configurations available
+
+sources:
+  #Files or Directories to be Excluded are mentioned in exclusions
+  exclusions:
+    - 'test'
+    - 'examples'
+    - 'build'
+    - 'conanfile.py'
+    - '.git'
+    - 'src/compiler/Parser/*'
+    - 'src/stage'
+
+# Specify the languages you would like to Scan. Default is all Detected
+languages: 'CPP, PYTHON'
+
+
+# Define modules one would like to run or disable. This is enough for Basic Users
+# Get List of Modules (Which are Default Enabled/Disabled) at the documentation link above
+#modules:
+#  - name: pmd
+#    enabled: true
+#  - name: gamma_cxx
+#    enabled: false
+#  - name: jshint
+#    enabled: true
+

examples/generator/CMakeLists.txt

@@ -1,19 +1,19 @@
 cmake_minimum_required(VERSION 3.5)
-project (generator)
+project(generator)
 
-set(Boost_USE_STATIC_LIBS ON )
+set(Boost_USE_STATIC_LIBS ON)
 set(Boost_USE_MULTITHREADED ON)
 
-FIND_PACKAGE( Boost COMPONENTS    
-    system
-    program_options
-    REQUIRED )
+find_package(
+  Boost
+  COMPONENTS system program_options
+  REQUIRED)
 
 include_directories(${Boost_INCLUDE_DIR})
 
 link_libraries(${Boost_LIBRARIES})
 
-INCLUDE_DIRECTORIES(${Boost_INCLUDE_DIR})
-ADD_DEFINITIONS( "-DHAS_BOOST" )
+include_directories(${Boost_INCLUDE_DIR})
+add_definitions("-DHAS_BOOST")
 
 add_executable(xgenerator generator.cpp)

examples/generator/generator.cpp

@@ -1,13 +1,13 @@
 #include <iostream>
 #include <fstream>
-#include <string.h>
+#include <cstring>
 #include <cstdlib>
 #include <vector>
 #include <limits>       // std::numeric_limits
 #include <boost/crc.hpp>
 #include <boost/program_options.hpp>
 #include <boost/system/error_code.hpp>
-using namespace std;
+
 using namespace boost ;
 
 const int CRC16_CCITT_R=0x8408;  //https://en.wikipedia.org/wiki/Cyclic_redundancy_check
@@ -17,11 +17,11 @@ const int DEFAULT_PACK_COUNT=10;
 unsigned int packSize = DEFAULT_SIZE_OF_PACK;
 unsigned int packCount = DEFAULT_PACK_COUNT;
 
-vector<unsigned int> data;
-vector<unsigned int> remote;
+std::vector<unsigned int> data;
+std::vector<unsigned int> remote;
 
-string sOutputFile = "file.binary";
-string sInpuFile = "";
+std::string sOutputFile = "file.binary";
+std::string sInpuFile = "";
 
 unsigned int sawSize = std::numeric_limits<int>::max();
 unsigned int rndSize = 0;
@@ -38,8 +38,8 @@ int main(int argc, char* argv[])
     ("addcrc,c", po::value<unsigned int> (&packSize), "count crc after this")
     ("addsum,u", po::value<unsigned int> (&packSize), "count simple sum after this")
     ("datacount,d", po::value<unsigned int> (&packCount), "count of genrated packs")
-    ("outfile,f", po::value<string> (&sOutputFile), "outputfilename, default:file.binary")
-    ("inputfile,k", po::value<string> (&sInpuFile), "get data from inputfile instead of algo" )
+    ("outfile,f", po::value<std::string> (&sOutputFile), "outputfilename, default:file.binary")
+    ("inputfile,k", po::value<std::string> (&sInpuFile), "get data from inputfile instead of algo" )
     ("saw,s", po::value<unsigned int> (&sawSize), "algorithm: saw modulo (default:maxint)")
     ("rnd,r", po::value<unsigned int> (&rndSize), "algorithm: random modulo (default:0)")
     ("mul,m", po::value<unsigned int> (&mulSize), "algorithm: multiplication (default:1)")
@@ -53,21 +53,21 @@ int main(int argc, char* argv[])
 
     if (vm.count("help"))
     {
-        cerr << argv[0] << " - data generator" << std::endl;
-        cerr << "The idea of this program is to generate data with or without crc."<< std::endl;
-        cerr << "For testing purposes."<< std::endl;
-        cerr << "Algorithm: ((rnd + (j*CRC_packSize + i)) * mul) % saw" << std::endl;
-        cerr << "Examples:"<< std::endl;
-        cerr << "\t./xgenerator -d 8 -s 30 -p -m 10:"<< std::endl;
-        cerr << "\t\t>0 10 20 0 10 20 0 10"<< std::endl;
-        cerr << "\t./xgenerator -d 8 -p"<< std::endl;
-        cerr << "\t\t>0 1 2 3 4 5 6 7"<< std::endl;
-        cerr << "\t./xgenerator -d 8 -p -r 20"<< std::endl;
-        cerr << "\t\t>3 7 19 18 17 20 12 19"<< std::endl;
-        cerr << "\t./xgenerator -d 2 -p -c 4"<< std::endl;
-        cerr << "\t\t0 1 2 3 37368"<< std::endl;
-        cerr << "\t\t4 5 6 7 24536"<< std::endl;
-        cerr << desc << endl ;
+        std::cerr << argv[0] << " - data generator" << std::endl;
+        std::cerr << "The idea of this program is to generate data with or without crc."<< std::endl;
+        std::cerr << "For testing purposes."<< std::endl;
+        std::cerr << "Algorithm: ((rnd + (j*CRC_packSize + i)) * mul) % saw" << std::endl;
+        std::cerr << "Examples:"<< std::endl;
+        std::cerr << "\t./xgenerator -d 8 -s 30 -p -m 10:"<< std::endl;
+        std::cerr << "\t\t>0 10 20 0 10 20 0 10"<< std::endl;
+        std::cerr << "\t./xgenerator -d 8 -p"<< std::endl;
+        std::cerr << "\t\t>0 1 2 3 4 5 6 7"<< std::endl;
+        std::cerr << "\t./xgenerator -d 8 -p -r 20"<< std::endl;
+        std::cerr << "\t\t>3 7 19 18 17 20 12 19"<< std::endl;
+        std::cerr << "\t./xgenerator -d 2 -p -c 4"<< std::endl;
+        std::cerr << "\t\t0 1 2 3 37368"<< std::endl;
+        std::cerr << "\t\t4 5 6 7 24536"<< std::endl;
+        std::cerr << desc << std::endl ;
         return system::errc::success;
     }
 
@@ -93,7 +93,7 @@ int main(int argc, char* argv[])
     for(auto j = 0; j < packCount; j++)
     {
 
-        vector<unsigned int> crcq;
+        std::vector<unsigned int> crcq;
         unsigned int sumq = 0;
 
         for(auto i = 0; i < packSize; ++i)
@@ -120,7 +120,7 @@ int main(int argc, char* argv[])
             }
             if (vm.count("print"))
             {
-                cout << val << " ";
+                std::cout << val << " ";
             }
 
             if (vm.count("addsum"))
@@ -136,7 +136,7 @@ int main(int argc, char* argv[])
             crcfn.process_bytes(crcq.data(), sizeof(unsigned int) * crcq.size());
             if (vm.count("print"))
             {
-                cout << crcfn.checksum() << endl ;
+                std::cout << crcfn.checksum() << std::endl ;
             }
             data.push_back(crcfn.checksum());
             crcqCnt++;
@@ -146,7 +146,7 @@ int main(int argc, char* argv[])
         {
             if (vm.count("print"))
             {
-                cout << sumq << endl ;
+                std::cout << sumq << std::endl ;
             }
             data.push_back(sumq);
             sumqCnt++;
@@ -155,22 +155,22 @@ int main(int argc, char* argv[])
 
     if (vm.count("print"))
     {
-        cout << endl ;
+        std::cout << std::endl ;
     }
 
     myfile.write((char*)data.data(), data.size()*sizeof(unsigned int));
     myfile.close();
 
-    cout << "count:"<< packCount * packSize << endl;
+    std::cout << "count:"<< packCount * packSize << std::endl;
     if (vm.count("addcrc"))
     {
-        cout << "crc cnt:"<< crcqCnt << endl;
+        std::cout << "crc cnt:"<< crcqCnt << std::endl;
     }
     if (vm.count("addsum"))
     {
-        cout << "sum cnt:"<< sumqCnt << endl;
+        std::cout << "sum cnt:"<< sumqCnt << std::endl;
     }
-    cout << "output:" << sOutputFile << endl;
-    cout << "done." << endl;
+    std::cout << "output:" << sOutputFile << std::endl;
+    std::cout << "done." << std::endl;
     return system::errc::success;
 }

examples/plot-dsp.sh

@@ -4,27 +4,24 @@ trap control_c SIGINT
 
 control_c()
 {
-    stty sane
-    pkill plotblock
     xqry -k
+    pkill plotblock
+    stty sane
 }
 
-linuxver=$(uname -r)
-if [[ $linuxver == *"Microsoft"* ]] ; then
-    TTYPE="--term wxt"
-else
-    TTYPE=""
-fi
-
-echo "Type ctrl+c to stop."
+STREAM=outputAll
 
 if ! xcompiler -q query-dsp.rql ; then exit 1 ; fi
 
+rm nohup.out
 nohup xretractor &
 
-sleep 4
+sleep 2
+
+echo "Type ctrl+c to stop."
+
 if [ -z "$DISPLAY" ]
 then
 export DISPLAY=:0
 fi
-xqry -s outputAll | plotblock.py 50 256 "output:red;source:blue" --sleep 0.05 $TTYPE | gnuplot 2>/dev/null
+xqry -s $STREAM | plotblock.py 50 256 "output:red;source:blue" --sleep 0.05 | gnuplot 2>/dev/null