Code

Feature scraper#93

Code

Open

lesedimotoma wants to merge 155 commits intodevelopCOS301-SE-2021/Silicon-Scraper:developfrom feature-scraperCOS301-SE-2021/Silicon-Scraper:feature-scraperCopy head branch name to clipboard

+20,173-12,639

Commits

Commits on Aug 10, 2021

• tests: changed integration tests

  lesedimotoma

  committed
  eed5bcd

  Copy full SHA for eed5bcd

Commits on Aug 11, 2021

• tests: fixed axios not a function error

  lesedimotoma

  committed
  ca3d1cb

  Copy full SHA for ca3d1cb

Commits on Aug 17, 2021

• feat:added a module to ping th e scraper.

  MkSibeko

  committed
  881a229

  Copy full SHA for 881a229

Commits on Aug 26, 2021

• chore: merged changes

  lesedimotoma

  committed
  49c84f9

  Copy full SHA for 49c84f9

Commits on Aug 27, 2021

• refactor: added brand precondition to title parser

  lesedimotoma

  committed
  7f6c437

  Copy full SHA for 7f6c437

• refactor: changed dreamware title selector

  lesedimotoma

  committed
  a0bbee6

  Copy full SHA for a0bbee6

• feat: manufactures urls

  MkSibeko

  committed
  d3359f9

  Copy full SHA for d3359f9

• feat: manufactures urls

  MkSibeko

  committed
  43705a0

  Copy full SHA for 43705a0

• Merge branch 'feature-scraper-descriptions' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper-descriptions

  lesedimotoma

  committed
  93e0815

  Copy full SHA for 93e0815

• feat: added sapphire url creator function

  lesedimotoma

  committed
  6a7e048

  Copy full SHA for 6a7e048

• feat: added amd url function

  lesedimotoma

  committed
  f7176c1

  Copy full SHA for f7176c1

• feat: added url function for cpu

  lesedimotoma

  committed
  0af6250

  Copy full SHA for 0af6250

• fix: fixed config for different gpus

  lesedimotoma

  committed
  5f05e4d

  Copy full SHA for 5f05e4d

• refactor: added url for workstation graphics

  lesedimotoma

  committed
  724f8cf

  Copy full SHA for 724f8cf

• refactor: removed log

  lesedimotoma

  committed
  fa4a40b

  Copy full SHA for fa4a40b

• refactor: removed log

  lesedimotoma

  committed
  dd2e7da

  Copy full SHA for dd2e7da

• feat: created a description selector class

  MkSibeko

  committed
  8ccf0f8

  Copy full SHA for 8ccf0f8

• feat: add amd selector class

  MkSibeko

  committed
  f561d6d

  Copy full SHA for f561d6d

• feat: add nvidia selector class

  MkSibeko

  committed
  ce713a7

  Copy full SHA for ce713a7

• feat: add nvidia selector class

  MkSibeko

  committed
  e1eccd9

  Copy full SHA for e1eccd9

• feat: added sapphire selector class

  MkSibeko

  committed
  9cd2b02

  Copy full SHA for 9cd2b02

• feat: added sapphire selector class

  MkSibeko

  committed
  327e5d4

  Copy full SHA for 327e5d4

• feat: added sapphire selector class

  MkSibeko

  committed
  f28cae4

  Copy full SHA for f28cae4

• feat: making manufacture's selector objects

  MkSibeko

  committed
  0af4d8c

  Copy full SHA for 0af4d8c

• feat: making manufacture's selector objects

  MkSibeko

  committed
  b69f376

  Copy full SHA for b69f376

• feat: making manufacture's selector objects

  MkSibeko

  committed
  9673855

  Copy full SHA for 9673855

• comments on the descriptions

  MkSibeko

  committed
  231361d

  Copy full SHA for 231361d

• fix: changed the return type of the get url functions for the manufactures

  MkSibeko

  committed
  87b700e

  Copy full SHA for 87b700e

• feat: added a get description function

  MkSibeko

  committed
  84180d4

  Copy full SHA for 84180d4

• feat: scraping the manufactures for descriptions

  MkSibeko

  committed
  98ca184

  Copy full SHA for 98ca184

• feat: scraping the manufactures for descriptions

  MkSibeko

  committed
  2473aa2

  Copy full SHA for 2473aa2

• feat: scraping the manufactures for descriptions

  MkSibeko

  committed
  4a60541

  Copy full SHA for 4a60541

• feat: scraping the manufactures for descriptions

  MkSibeko

  committed
  4a8eda3

  Copy full SHA for 4a8eda3

• feat: scraping the manufactures for descriptions

  MkSibeko

  committed
  cc76d5a

  Copy full SHA for cc76d5a

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  27f28da

  Copy full SHA for 27f28da

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  d9a8a13

  Copy full SHA for d9a8a13

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  7830234

  Copy full SHA for 7830234

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  3b86d39

  Copy full SHA for 3b86d39

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  a6dbd8a

  Copy full SHA for a6dbd8a

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  112cd4d

  Copy full SHA for 112cd4d

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  6ae16db

  Copy full SHA for 6ae16db

• refactor: added condition for workstation cards url

  lesedimotoma

  committed
  7435d94

  Copy full SHA for 7435d94

• feat: parse the descriptions array into a key value pair object

  MkSibeko

  committed
  a362a17

  Copy full SHA for a362a17

• fix: fixed nvidia url

  lesedimotoma

  committed
  ef57b83

  Copy full SHA for ef57b83

• fix: fixed pro graphics url

  lesedimotoma

  committed
  c236fc7

  Copy full SHA for c236fc7

• feat: add intel base url

  lesedimotoma

  committed
  e822bbd

  Copy full SHA for e822bbd

• Merge branch 'feature-scraper-descriptions' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper-descriptions

  lesedimotoma

  committed
  7eaef0f

  Copy full SHA for 7eaef0f

• fit: added a type param to the descriptions parser

  MkSibeko

  committed
  ad59305

  Copy full SHA for ad59305

• feat: implementing conditional parsing according to the manufacture

  MkSibeko

  committed
  65f64fe

  Copy full SHA for 65f64fe

• fit: differentiated between ti and noneti models

  MkSibeko

  committed
  2ab3abe

  Copy full SHA for 2ab3abe

• fit: added the rest of the manufactures

  MkSibeko

  committed
  6cd5ae8

  Copy full SHA for 6cd5ae8

• Merge remote-tracking branch 'origin/feature-scraper-descriptions' into feature-scraper-descriptions

  Show description for e95514d
  MkSibeko

  committed
  e95514d

  Copy full SHA for e95514d

Commits on Aug 28, 2021

• feat: add intel base url

  lesedimotoma

  committed
  7286f8d

  Copy full SHA for 7286f8d

• feat: add intel base url

  lesedimotoma

  committed
  84bf71d

  Copy full SHA for 84bf71d

• Merge branch 'feature-scraper-descriptions' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper-descriptions

  lesedimotoma

  committed
  50d0466

  Copy full SHA for 50d0466

• feat: created intel search urls

  lesedimotoma

  committed
  ad3feb6

  Copy full SHA for ad3feb6

• feat: created intel search urls

  lesedimotoma

  committed
  36486f0

  Copy full SHA for 36486f0

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  1b4d7d2

  Copy full SHA for 1b4d7d2

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  3b21333

  Copy full SHA for 3b21333

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  6a16f30

  Copy full SHA for 6a16f30

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  bbe418d

  Copy full SHA for bbe418d

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  5596820

  Copy full SHA for 5596820

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  9726a37

  Copy full SHA for 9726a37

• feat: adding conditional description assignments due to the different structures of the manufactures websites

  MkSibeko

  committed
  23c8098

  Copy full SHA for 23c8098

• feat: implemented the intel selector

  MkSibeko

  committed
  dab6ef9

  Copy full SHA for dab6ef9

• feat: implemented the intel selector

  MkSibeko

  committed
  94d3f2e

  Copy full SHA for 94d3f2e

• feat: implemented the intel selector

  MkSibeko

  committed
  748e9c6

  Copy full SHA for 748e9c6

• feat: implemented the intel selector

  MkSibeko

  committed
  3a9cc6b

  Copy full SHA for 3a9cc6b

• feat: implemented the intel selector

  MkSibeko

  committed
  cb779c9

  Copy full SHA for cb779c9

• chor: simplified the description scraper

  MkSibeko

  committed
  1211bfd

  Copy full SHA for 1211bfd

• chor: simplified the description scraper

  MkSibeko

  committed
  d082cdb

  Copy full SHA for d082cdb

• chor: simplified the description scraper

  MkSibeko

  committed
  7d6efc8

  Copy full SHA for 7d6efc8

• chor: simplified the description scraper

  MkSibeko

  committed
  d5de520

  Copy full SHA for d5de520

• fix: updated the description type from string to dynamic object

  MkSibeko

  committed
  d989ee0

  Copy full SHA for d989ee0

• fix: constant variable naming implemented in the parser module

  MkSibeko

  committed
  18b6418

  Copy full SHA for 18b6418

• chor: general cleaning up codebase and structure

  MkSibeko

  committed
  b5112a3

  Copy full SHA for b5112a3

Commits on Aug 29, 2021

• fix: fixed title parser for intel processors

  lesedimotoma

  committed
  3d186b2

  Copy full SHA for 3d186b2

• fix: fixed title parser for intel processors

  lesedimotoma

  committed
  59d1b5f

  Copy full SHA for 59d1b5f

• refactor: remove statement in title parser causing errors

  lesedimotoma

  committed
  a612919

  Copy full SHA for a612919

• refactor: remove statement in title parser causing errors

  lesedimotoma

  committed
  e60cff3

  Copy full SHA for e60cff3

• feat: added conditions for edge cases

  lesedimotoma

  committed
  cf15368

  Copy full SHA for cf15368

• feat: added conditions for edge cases

  lesedimotoma

  committed
  fda1d24

  Copy full SHA for fda1d24

• feat: added conditions for edge cases

  lesedimotoma

  committed
  f293f4f

  Copy full SHA for f293f4f

• refactor: changed selectors

  lesedimotoma

  committed
  9333957

  Copy full SHA for 9333957

Commits on Aug 30, 2021

• re: changed scraping method for dynamic websites

  MkSibeko

  committed
  70d4be8

  Copy full SHA for 70d4be8

• refactor: changed selectors

  lesedimotoma

  committed
  47e6229

  Copy full SHA for 47e6229

• Merge branch 'feature-scraper-descriptions' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper-descriptions

  lesedimotoma

  committed
  77ab974

  Copy full SHA for 77ab974

Commits on Sep 6, 2021

• refactor: page evaluation function

  lesedimotoma

  committed
  eedb199

  Copy full SHA for eedb199

• refactor: added parameter to page fucntion

  lesedimotoma

  committed
  92c1e41

  Copy full SHA for 92c1e41

• refactor: changed page evaluation function

  lesedimotoma

  committed
  152b066

  Copy full SHA for 152b066

• refactor: changed selector

  lesedimotoma

  committed
  866f3be

  Copy full SHA for 866f3be

• refactor: changed page evaluation function

  lesedimotoma

  committed
  6d77031

  Copy full SHA for 6d77031

• refactor: changed page evaluation function

  lesedimotoma

  committed
  5915055

  Copy full SHA for 5915055

• refactor: changed evaluation function parameters

  lesedimotoma

  committed
  270d6d3

  Copy full SHA for 270d6d3

• refactor: changed evaluation function parameters

  lesedimotoma

  committed
  4c8cf9b

  Copy full SHA for 4c8cf9b

• chore: cleaned up code

  lesedimotoma

  committed
  66b7f70

  Copy full SHA for 66b7f70

Commits on Sep 9, 2021

• test: added description tests

  lesedimotoma

  committed
  54aa8a2

  Copy full SHA for 54aa8a2

• test: added parser tests

  lesedimotoma

  committed
  1f796cb

  Copy full SHA for 1f796cb

• chore: added comments

  lesedimotoma

  committed
  91ba463

  Copy full SHA for 91ba463

Commits on Sep 10, 2021

• refactor: changed amd urls

  lesedimotoma

  committed
  a86f90e

  Copy full SHA for a86f90e

• refactor: changed amd urls

  lesedimotoma

  committed
  cdbaa4a

  Copy full SHA for cdbaa4a

Commits on Sep 14, 2021

• refactor: changed intel selectors

  lesedimotoma

  committed
  c30c8df

  Copy full SHA for c30c8df

• refactor: changed intel selectors

  lesedimotoma

  committed
  84bcd91

  Copy full SHA for 84bcd91

• feat: added intel search query

  lesedimotoma

  committed
  06ba580

  Copy full SHA for 06ba580

Commits on Sep 15, 2021

• refactor: changed if condition

  lesedimotoma

  committed
  9170a35

  Copy full SHA for 9170a35

• refactor: added function to expose selector variable

  lesedimotoma

  committed
  860e571

  Copy full SHA for 860e571

• refactor: fixed page evalute function

  lesedimotoma

  committed
  05b56e3

  Copy full SHA for 05b56e3

• refactor: fixed page evalute function

  lesedimotoma

  committed
  efad839

  Copy full SHA for efad839

Commits on Sep 16, 2021

• refactor: moved content to separate function

  lesedimotoma

  committed
  3f6406a

  Copy full SHA for 3f6406a

Commits on Sep 17, 2021

• refactor: changed page evalute function

  lesedimotoma

  committed
  2736e5d

  Copy full SHA for 2736e5d

Commits on Sep 18, 2021

• fix: updated title parser

  

  ThatoJadezweni

  committed
  e361d48

  Copy full SHA for e361d48

• fix: fixed title parser

  lesedimotoma

  committed
  68f2f70

  Copy full SHA for 68f2f70

• refactor: changed url

  lesedimotoma

  committed
  d828623

  Copy full SHA for d828623

• fix: fixed manufacture url

  lesedimotoma

  committed
  5baddbf

  Copy full SHA for 5baddbf

• fix: fixed manufacture url

  lesedimotoma

  committed
  9fac6e5

  Copy full SHA for 9fac6e5

• fix: fixed manufacture url

  lesedimotoma

  committed
  4b4c618

  Copy full SHA for 4b4c618

Commits on Sep 19, 2021

• refactor: trimmed title

  lesedimotoma

  committed
  b3a0ee4

  Copy full SHA for b3a0ee4

• feat : scrape descriptions

  MkSibeko

  committed
  a021816

  Copy full SHA for a021816

• chore: removed console log

  lesedimotoma

  committed
  3e73875

  Copy full SHA for 3e73875

• Merge branch 'feature-scraper-descriptions' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper-descriptions

  lesedimotoma

  committed
  4d73957

  Copy full SHA for 4d73957

• ffix: fixed title parser

  lesedimotoma

  committed
  314a628

  Copy full SHA for 314a628

Commits on Sep 20, 2021

• fix: fixed descriptions

  lesedimotoma

  committed
  5a0d081

  Copy full SHA for 5a0d081

• fix: fixed selectors

  lesedimotoma

  committed
  c573187

  Copy full SHA for c573187

• fix: fixed descriptions

  lesedimotoma

  committed
  b500ba2

  Copy full SHA for b500ba2

• fix: fixed descriptions

  lesedimotoma

  committed
  b65b7bf

  Copy full SHA for b65b7bf

• chore: removed file

  lesedimotoma

  committed
  9fbcf36

  Copy full SHA for 9fbcf36

Commits on Oct 5, 2021

• Merge pull request #92 from COS301-SE-2021/feature-scraper-descriptions

  Show description for 2e004b6
  lesedimotoma

  authored
  2e004b6

  Copy full SHA for 2e004b6

• fix: fixed tests

  lesedimotoma

  committed
  f24aee6

  Copy full SHA for f24aee6

• Merge branch 'feature-scraper' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper

  lesedimotoma

  committed
  5d54cc3

  Copy full SHA for 5d54cc3

• chore: removed file

  lesedimotoma

  committed
  e4355cc

  Copy full SHA for e4355cc

• fix: fixed install issues

  lesedimotoma

  committed
  5f901ba

  Copy full SHA for 5f901ba

• fix: fixed spelling in product model

  lesedimotoma

  committed
  7ec60a5

  Copy full SHA for 7ec60a5

• fix: fixed import issues

  lesedimotoma

  committed
  226c6e7

  Copy full SHA for 226c6e7

• tests: added puppeteer mocks

  lesedimotoma

  committed
  361a6a5

  Copy full SHA for 361a6a5

• chore: changed jest config

  lesedimotoma

  committed
  9610c48

  Copy full SHA for 9610c48

• chore: installed types

  lesedimotoma

  committed
  e1a29dd

  Copy full SHA for e1a29dd

• chore: installed types

  lesedimotoma

  committed
  66bb654

  Copy full SHA for 66bb654

• chore: installed types

  lesedimotoma

  committed
  41a6936

  Copy full SHA for 41a6936

Commits on Oct 6, 2021

• chore - installed types

  MkSibeko

  committed
  159f379

  Copy full SHA for 159f379

• chore - installed types

  MkSibeko

  committed
  1f36cf6

  Copy full SHA for 1f36cf6

• chore - installed types

  MkSibeko

  committed
  1875235

  Copy full SHA for 1875235

• chore - installed types

  MkSibeko

  committed
  3e4a253

  Copy full SHA for 3e4a253

• chore - installed types

  MkSibeko

  committed
  2f03ab2

  Copy full SHA for 2f03ab2

• chore - installed types

  MkSibeko

  committed
  36a1083

  Copy full SHA for 36a1083

• chore - installed types

  MkSibeko

  committed
  42a8f86

  Copy full SHA for 42a8f86

• chore - installed types

  MkSibeko

  committed
  f8ddef0

  Copy full SHA for f8ddef0

• chore: updated dependencies

  lesedimotoma

  committed
  d08f541

  Copy full SHA for d08f541

• chore: updated dependencies

  lesedimotoma

  committed
  ebd9b32

  Copy full SHA for ebd9b32

• chore: updated dependencies

  lesedimotoma

  committed
  95a0969

  Copy full SHA for 95a0969

• chore: updated dependencies

  lesedimotoma

  committed
  f0ff0d1

  Copy full SHA for f0ff0d1

• test: fixed test conditions

  lesedimotoma

  committed
  977e2b9

  Copy full SHA for 977e2b9

• fix: fixed test config

  lesedimotoma

  committed
  bdffc0d

  Copy full SHA for bdffc0d

• tests: fixed tests

  lesedimotoma

  committed
  e49782d

  Copy full SHA for e49782d

• Merge branch 'feature-scraper' of github.com:COS301-SE-2021/Silicon-Scraper into feature-scraper

  lesedimotoma

  committed
  631f442

  Copy full SHA for 631f442

• tests: fixed tests

  lesedimotoma

  committed
  1835ccf

  Copy full SHA for 1835ccf