Skip to content

Commit

Permalink
Add binary handlers in DbApiHandler.cs
Browse files Browse the repository at this point in the history
Short changes:
	Add supporting of binary requests, and binary-responses.
	Add binary handlers in _handlers_bin dictionary, in DbApiHandler.cs
		This handlers assept byte[] from binary-requests, sent with "Content-Type: application/octet-stream".
		A text-handlers in _handlers dictionary, accept strings only.
		
		The reversive encoding "iso-8859-1" (latin1), can encode each one byte into each one char in range "\u0000-\u00FF",
		so this encoding can be used, to send bytes, as string on the text-handlers.
		But, by default, there is an utf-8 bytes, so this decoded as UTF-8 text.
		
	Add an example text-handlers
		(text-handler) "echo-string"
			accept utf-8 encoded string, and echo this as string.
			this can accept AllBytes as latin-1 string, but return this as utf-8 encoded string.
			
		(text-handler) "echo-binary-string"
			accept AllBytes, as latin1-encoded string,
			and echo this as binary-response (with "Content-Type: application/octet-stream")

		(binary-handler)	"echo-bytes"
			accept AllBytes, as binary request (with "Content-Type: application/octet-stream")
			and echo this as binary-response (with "Content-Type: application/octet-stream")
				
	Add test file: \pages\test_echo_handlers.html
		44 tests, with example code,
			how to send the data with different types,
			and how to decode the responses back.

Full changes:
	\nanodb.exe-source\Server\DbApiHandler.cs
		Lines 28		:	Add Dictionary<string, Func<string,byte[],HttpResponse>> _handlers_bin
		Lines 34		:	Set this as new Dictionary<string, Func<string, byte[], HttpResponse>>();
		Lines 96 - 99	:	Att three test-handlers for full-server:
								"echo-string"			//return the same string, utf-8
								"echo-binary-string"	//return bytes of binary string
								"echo-bytes"			//return bytes of binary request
		Lines 170 - 173	:	Add the same handlers for lite-server, but comment this, to prevent DDoS-attack, with large requests.
		Lines 174 - 181	:	Rewrite long line with code.
		Lines 183 - 187	:	Add private HttpResponse EchoString to echo text-string in utf-8 encoding.
		Lines 189 - 204	:	Add private HttpResponse EchoBinaryString to echo text-string in iso-8859-1 (latin1) encoding.
		Lines 206 - 214	:	Add private HttpResponse EchoBytes to echo bytes, that was been send with "Content-Type: application/octet-stream"
		Lines 216 - 232	:	Add public static bool isLatin1String + commented javascript-line. Use _lockObj there.
		Lines 1136		:	Made this _lockObj the private static, because isLatin1String is public static.
		Lines 1268		:	Add comment for HttpResponse Handle
		Lines 1269-1213	:	Rewrite HttpResponse Handle, and add supporting of binary-handlers: "_handlers_bin", that accept byte[], not string.

	\nanodb.exe-source\Server\HttpConnection.cs
		Lines 20		:	Rename string Request to string RequestText, because this is a text of request.
		Lines 21		:	Rename byte[] Raw to byte[] RequestBytes, because this is a bytes for binary-requests.
		Lines 24		:	string RequestHeadersString, to save headers from TcpServer.cs
		Lines 25		:	Add bool isBinaryRequest = false
		Lines 27 - 34	:	Rewrite arguments one per line.
								Add two optional arguments: string headers = null, and bool set_binary_request = false
		Lines 36		:	Rename Raw to RequestBytes, because this is a bytes of binary-requests.
		Lines 37		:	Rename Request to RequestText, because this is a text of text-request.
		Lines 40 - 44	:	Set "RequestHeadersString" and "isBinaryRequest" from optional arguments, if this was been specified.
		Lines 53		:	Change condition from (response.IsBinary) to (response.IsBinary == true), to show this must to be true.
		Lines 54, 56	:	Using GetResponseHeaders() instead of GetResponse(); because this is a headers.

	\nanodb.exe-source\Server\HttpRequest.cs
		Lines 26		:	Add public readonly bool isBinary = false;
								to mark binary requests.
		Lines 27		:	Add public readonly byte[] Content_bytes;
								to save bytes of binary-request.
		Lines 28		:	Add public readonly int MaxHeadersSize = 48*1024;
								to read partial request with headers, from large POST-request.
		Lines 29		:	Add public readonly int MaxBinaryContentStringSize = 4*1024*1024;
								to limit size of binary string, that can be writted from binary-request.
		Lines 31		:	Add private readonly object _lockObj
								to lock process
		Lines 33 - 49	:	Move the code to extract headers from "he", into separate method:
								public Dictionary<string, string> GetHeadersFromString(ref string he).
								Use lock, there.
		Lines 51		:	Rewrite public HttpRequest(HttpConnection conn), use conn only, there.
		Lines 53		:	Use lock, in public HttpRequest(HttpConnection conn)
		Lines 55 - 56	:	Add two lines, with comments, about state of text-request, and binary-request.
		Lines 57 - 60	:	Extract four values from conn, add commented default values of this, and comments.
		Lines 62 - 210	:	Rewrite the rest code.
								Add supporting of binary-reqests.
								Write Content_bytes from bytes for binary-requests, and leave Content = null.
								use headers, parsed in TcpServer.cs, but do extracting the headers optionally,
								if this was been not specified, in conn.HeadersString
		Lines 212		:	use method: Headers = GetHeadersFromString(ref he);	//get Dictionary <string, string> Headers from "he"

	\nanodb.exe-source\Server\HttpServer.cs
		Lines 52		:	OnConnectionAdded: Remove old test-code with Console.WriteLine,
								and use connection only, in arguments of running HttpRequest.
		Lines 63		:	OnConnectionAdded_liet: the same change.
		Lines 84		:	Add line to show request.Headers["Referrer"],
								but leave this commented.
		Lines 139		:	Rename connection.Request to connection.RequestText, because this was been renamed in HttpConnection.cs

	\nanodb.exe-source\Server\HttpResponse.cs
		Lines 15		:	Rename private readonly string _response; to private readonly string _headers;
								because really, this is a headers of response.
		Lines 18		:	Add private readonly bool	_isBinary;
								to set this flag true, when response is binary.
		Lines 28 - 43	:	Rewrite HttpResponse for binary responses, that accept "byte[] bytes"
		Lines 45 - 75	:	Rewrite HttpResponse for text-responses, that accept "string content"
								Add optional parameters, there.
								Now this can return bytes, if isBinaryResponse and encoding was been specified.
		Lines 77 - 82	:	Rewrite short version of HttpResponse(string code, string content)
								Add there an optional parameters bool isBinaryResponse = false, System.Text.Encoding encoding = null
								Now this return binary responses, as bytes of text, encoded with specified encoding.
		Lines 84		:	Rename GetResponse() to GetResponseHeaders(), because really this return headers.
		Lines 86		:	Rename _response to _headers, because really, this is a headers of response.
								
	\nanodb.exe-source\Server\TcpServer.cs
		Lines 555		:	Comment test Console.WriteLine.
		Lines 557		:	Comment test Console.WriteLine.
		Lines 561		:	Add comment
		Lines 824 - 827	:	Add two test Console.WriteLine, but comment this.
		Lines 861		:	Add optional argument "headers", readed in "headers"-variable, to save this in HttpConnection.HeadersString
		Lines 862		:	Add test value of null, there, to test the case when headers not specified.
								HttpRequest, must to parse this from if this not exists in conn.HeadersString.
		Lines 864 - 865	:	Add two comments about state of requests, when "is_binary_request" not specified.
		Lines 866		:	Add optional argument "is_binary_request" that got value in TcpServer.cs
		Lines 867		:	Add commented the test case, "true", and "false", when this value specified for any request (seems, like ok, in both cases.

	\pages\test_echo_handlers.html
		Add this file with 44 tests.
		This available on http://IP:PORT/pages/test_echo_handlers.html
		See source code and comments.
		Make this file readonly, to prevent modifications of the source code.
		This is just an example of JavaScript code, to send the different types of data on the API Handlers of DbApiHandler.cs.

	\Changes.txt - Update this file "Changes.txt"
  • Loading branch information
username1565 authored Jun 26, 2021
1 parent 36708e9 commit beb5bdb
Show file tree
Hide file tree
Showing 8 changed files with 3,090 additions and 116 deletions.
132 changes: 132 additions & 0 deletions Changes.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,138 @@ BEGIN CHANGELOG:
__________________________________________________________________________________________________________________________________________________________________
(XX.XX.XXXX) commit (latest, unknown hash) ? files changed with ? additions and ? deletions.
__________________________________________________________________________________________________________________________________________________________________
(27.06.2021) commit (latest, unknown hash) 8 files changed with ? additions and ? deletions.

Add binary handlers in DbApiHandler.cs

Short changes:
Add supporting of binary requests, and binary-responses.
Add binary handlers in _handlers_bin dictionary, in DbApiHandler.cs
This handlers assept byte[] from binary-requests, sent with "Content-Type: application/octet-stream".
A text-handlers in _handlers dictionary, accept strings only.

The reversive encoding "iso-8859-1" (latin1), can encode each one byte into each one char in range "\u0000-\u00FF",
so this encoding can be used, to send bytes, as string on the text-handlers.
But, by default, there is an utf-8 bytes, so this decoded as UTF-8 text.

Add an example text-handlers
(text-handler) "echo-string"
accept utf-8 encoded string, and echo this as string.
this can accept AllBytes as latin-1 string, but return this as utf-8 encoded string.

(text-handler) "echo-binary-string"
accept AllBytes, as latin1-encoded string,
and echo this as binary-response (with "Content-Type: application/octet-stream")

(binary-handler) "echo-bytes"
accept AllBytes, as binary request (with "Content-Type: application/octet-stream")
and echo this as binary-response (with "Content-Type: application/octet-stream")

Add test file: \pages\test_echo_handlers.html
44 tests, with example code,
how to send the data with different types,
and how to decode the responses back.

Full changes:
\nanodb.exe-source\Server\DbApiHandler.cs
Lines 28 : Add Dictionary<string, Func<string,byte[],HttpResponse>> _handlers_bin
Lines 34 : Set this as new Dictionary<string, Func<string, byte[], HttpResponse>>();
Lines 96 - 99 : Att three test-handlers for full-server:
"echo-string" //return the same string, utf-8
"echo-binary-string" //return bytes of binary string
"echo-bytes" //return bytes of binary request
Lines 170 - 173 : Add the same handlers for lite-server, but comment this, to prevent DDoS-attack, with large requests.
Lines 174 - 181 : Rewrite long line with code.
Lines 183 - 187 : Add private HttpResponse EchoString to echo text-string in utf-8 encoding.
Lines 189 - 204 : Add private HttpResponse EchoBinaryString to echo text-string in iso-8859-1 (latin1) encoding.
Lines 206 - 214 : Add private HttpResponse EchoBytes to echo bytes, that was been send with "Content-Type: application/octet-stream"
Lines 216 - 232 : Add public static bool isLatin1String + commented javascript-line. Use _lockObj there.
Lines 1136 : Made this _lockObj the private static, because isLatin1String is public static.
Lines 1268 : Add comment for HttpResponse Handle
Lines 1269-1213 : Rewrite HttpResponse Handle, and add supporting of binary-handlers: "_handlers_bin", that accept byte[], not string.

\nanodb.exe-source\Server\HttpConnection.cs
Lines 20 : Rename string Request to string RequestText, because this is a text of request.
Lines 21 : Rename byte[] Raw to byte[] RequestBytes, because this is a bytes for binary-requests.
Lines 24 : string RequestHeadersString, to save headers from TcpServer.cs
Lines 25 : Add bool isBinaryRequest = false
Lines 27 - 34 : Rewrite arguments one per line.
Add two optional arguments: string headers = null, and bool set_binary_request = false
Lines 36 : Rename Raw to RequestBytes, because this is a bytes of binary-requests.
Lines 37 : Rename Request to RequestText, because this is a text of text-request.
Lines 40 - 44 : Set "RequestHeadersString" and "isBinaryRequest" from optional arguments, if this was been specified.
Lines 53 : Change condition from (response.IsBinary) to (response.IsBinary == true), to show this must to be true.
Lines 54, 56 : Using GetResponseHeaders() instead of GetResponse(); because this is a headers.

\nanodb.exe-source\Server\HttpRequest.cs
Lines 26 : Add public readonly bool isBinary = false;
to mark binary requests.
Lines 27 : Add public readonly byte[] Content_bytes;
to save bytes of binary-request.
Lines 28 : Add public readonly int MaxHeadersSize = 48*1024;
to read partial request with headers, from large POST-request.
Lines 29 : Add public readonly int MaxBinaryContentStringSize = 4*1024*1024;
to limit size of binary string, that can be writted from binary-request.
Lines 31 : Add private readonly object _lockObj
to lock process
Lines 33 - 49 : Move the code to extract headers from "he", into separate method:
public Dictionary<string, string> GetHeadersFromString(ref string he).
Use lock, there.
Lines 51 : Rewrite public HttpRequest(HttpConnection conn), use conn only, there.
Lines 53 : Use lock, in public HttpRequest(HttpConnection conn)
Lines 55 - 56 : Add two lines, with comments, about state of text-request, and binary-request.
Lines 57 - 60 : Extract four values from conn, add commented default values of this, and comments.
Lines 62 - 210 : Rewrite the rest code.
Add supporting of binary-reqests.
Write Content_bytes from bytes for binary-requests, and leave Content = null.
use headers, parsed in TcpServer.cs, but do extracting the headers optionally,
if this was been not specified, in conn.HeadersString
Lines 212 : use method: Headers = GetHeadersFromString(ref he); //get Dictionary <string, string> Headers from "he"

\nanodb.exe-source\Server\HttpServer.cs
Lines 52 : OnConnectionAdded: Remove old test-code with Console.WriteLine,
and use connection only, in arguments of running HttpRequest.
Lines 63 : OnConnectionAdded_liet: the same change.
Lines 84 : Add line to show request.Headers["Referrer"],
but leave this commented.
Lines 139 : Rename connection.Request to connection.RequestText, because this was been renamed in HttpConnection.cs

\nanodb.exe-source\Server\HttpResponse.cs
Lines 15 : Rename private readonly string _response; to private readonly string _headers;
because really, this is a headers of response.
Lines 18 : Add private readonly bool _isBinary;
to set this flag true, when response is binary.
Lines 28 - 43 : Rewrite HttpResponse for binary responses, that accept "byte[] bytes"
Lines 45 - 75 : Rewrite HttpResponse for text-responses, that accept "string content"
Add optional parameters, there.
Now this can return bytes, if isBinaryResponse and encoding was been specified.
Lines 77 - 82 : Rewrite short version of HttpResponse(string code, string content)
Add there an optional parameters bool isBinaryResponse = false, System.Text.Encoding encoding = null
Now this return binary responses, as bytes of text, encoded with specified encoding.
Lines 84 : Rename GetResponse() to GetResponseHeaders(), because really this return headers.
Lines 86 : Rename _response to _headers, because really, this is a headers of response.

\nanodb.exe-source\Server\TcpServer.cs
Lines 555 : Comment test Console.WriteLine.
Lines 557 : Comment test Console.WriteLine.
Lines 561 : Add comment
Lines 824 - 827 : Add two test Console.WriteLine, but comment this.
Lines 861 : Add optional argument "headers", readed in "headers"-variable, to save this in HttpConnection.HeadersString
Lines 862 : Add test value of null, there, to test the case when headers not specified.
HttpRequest, must to parse this from if this not exists in conn.HeadersString.
Lines 864 - 865 : Add two comments about state of requests, when "is_binary_request" not specified.
Lines 866 : Add optional argument "is_binary_request" that got value in TcpServer.cs
Lines 867 : Add commented the test case, "true", and "false", when this value specified for any request (seems, like ok, in both cases.

\pages\test_echo_handlers.html
Add this file with 44 tests.
This available on http://IP:PORT/pages/test_echo_handlers.html
See source code and comments.
Make this file readonly, to prevent modifications of the source code.
This is just an example of JavaScript code, to send the different types of data on the API Handlers of DbApiHandler.cs.

\Changes.txt - Update this file "Changes.txt"
__________________________________________________________________________________________________________________________________________________________________
(23.06.2021) commit (latest, unknown hash) 2 files changed with ? additions and ? deletions.

Rewrite TcpServer.cs
Expand Down
109 changes: 99 additions & 10 deletions nanodb.exe-source/Server/DbApiHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,13 @@ class DbApiHandler : IRequestHandler
{
private PostDb _db;
private Dictionary<string, Func<string,string,HttpResponse>> _handlers;
private Dictionary<string, Func<string,byte[],HttpResponse>> _handlers_bin;
private bool is_lite = false; //True, when this is request on lite-server, or false, if this is request on full-server.
public DbApiHandler(PostDb db, bool lite = false, bool SetAllowReput = false, bool SetByPassValidation = false)
{
_db = db;
_handlers = new Dictionary<string, Func<string, string, HttpResponse>>();
_handlers_bin = new Dictionary<string, Func<string, byte[], HttpResponse>>();
is_lite = lite; allowReput = SetAllowReput; bypassValidation = SetByPassValidation; //set this once, for current Instance.
if(lite == false){ //all actions allowed for full server
// filling handlers dictionary with actions that will be called with (request address, request content) args:
Expand Down Expand Up @@ -91,6 +93,10 @@ public DbApiHandler(PostDb db, bool lite = false, bool SetAllowReput = false, bo
_handlers["delete_reports"] = DeleteReportsForPostHash;
_handlers["undelete_post"] = UndeletePost; //undelete post, after this was been "deleted_once" on full-server: http://127.0.0.1:7346/api/undelete_post/{POST_HASH}
_handlers["random_captcha"] = RandomCaptcha; //GET random captcha or solve this (POST)

_handlers ["echo-string"] = EchoString; //return the same string, utf-8
_handlers ["echo-binary-string"] = EchoBinaryString; //return bytes of binary string
_handlers_bin ["echo-bytes"] = EchoBytes; //return bytes back
}
else if(lite == true){ //not all actions allowed for lite-server to run this by another anonymous users.

Expand Down Expand Up @@ -161,9 +167,70 @@ public DbApiHandler(PostDb db, bool lite = false, bool SetAllowReput = false, bo
// _handlers["png-collect-avail"] = (a,b)=>new HttpResponse(_collectAvail ? StatusCode.Ok : StatusCode.NotFound, _collectAvail ? "Finished." : "Collect...");
_handlers["png-create-avail"] = (a,b)=>new HttpResponse(_createAvail ? StatusCode.Ok : StatusCode.NotFound, _createAvail ? "Finished." : "Creating PNG...");
_handlers["random_captcha"] = RandomCaptcha; //GET random captcha or solve this (POST)
} Set_Timer_To_Delete_Generated_Images();
}private static bool allowReput = false; private static bool bypassValidation = false; /*//true if need to make bypassValidation for posts in containers, while png-collect is processing.*/

// _handlers ["echo-string"] = EchoString; //return the same string, utf-8
// _handlers ["echo-binary-string"] = EchoBinaryString; //return bytes of binary string encoded with iso-8859-1
// _handlers_bin ["echo-bytes"] = EchoBytes; //return bytes back
}

Set_Timer_To_Delete_Generated_Images();
}

private static bool allowReput = false;
private static bool bypassValidation = false;
/*//true if need to make bypassValidation for posts in containers, while png-collect is processing.*/

//echo string, in any encoding.
private HttpResponse EchoString(string stringGET = null, string stringPOST = null){
string data = param_GET_POST(stringGET, stringPOST);
return new HttpResponse(StatusCode.Ok, data);
}

//echo binary string, as binary response
private HttpResponse EchoBinaryString(string stringGET = null, string stringPOST = null){
string data = param_GET_POST(stringGET, stringPOST);
//Console.WriteLine("EchoBinaryString: data.Length: "+data.Length);

System.Text.Encoding encoding = null;
if(isLatin1String(data)){
encoding = System.Text.Encoding.GetEncoding("iso-8859-1");
}
else{
encoding = System.Text.Encoding.UTF8;
}

return new HttpResponse(StatusCode.Ok, data, MimeType.Mime("App", encoding), true, encoding); //just return back, the binary string with bytes, in HTTP response.
//"true", means response is binary, even if this is a string.
}

//echo bytes from binary request.
private HttpResponse EchoBytes(string path = null, byte[] bytes = null){
try{
return new HttpResponse(StatusCode.Ok, bytes, MimeType.Mime("App")); //just return binary data in HTTP response.
}catch(Exception ex){
Console.WriteLine(ex);
return new HttpResponse(StatusCode.Ok, ex.Message, MimeType.Mime("Html")); //just return binary data in HTTP response.
}
}

//C#: check is string "iso-8859-1"-encoded or not (true/false)
public static bool isLatin1String(string str){
lock (_lockObj)
{
return (
Regex.Match(
str //for each char in str
, @"[^\u0000-\u00FF]" //try to match not latin1-char, and stop
, RegexOptions.None
)
.Length //then take Length of result
) == 0 //and compare this to 0
; //true/false (non-latin character was not found or was found)
}
}
// JavaScript:
//function isLatin1String(str){return (str.match(/[^\u0000-\u00FF]/) === null);} //check is string "iso-8859-1"-encoded or not (true/false)

/*
This method return string with parameters, from request parameters.
It return first or second parameter of request query (if second is available).
Expand Down Expand Up @@ -1066,7 +1133,7 @@ private HttpResponse ReAddPost(string replyTo, string content)
return new HttpResponse(StatusCode.Ok, JsonConvert.SerializeObject(post));
}

private object _lockObj = new object();
private static readonly object _lockObj = new object();

//Delete post "delete_once", or "delete_forever", if post was already "deleted_once".
private HttpResponse DeletePost(string hash, string notUsed = null)
Expand Down Expand Up @@ -1198,6 +1265,7 @@ private HttpResponse UndeletePost(string hash, string notUsed = null)
);
}

//Handle API-request, and return HttpResponse.
public HttpResponse Handle(HttpRequest request)
{
try
Expand All @@ -1206,20 +1274,41 @@ public HttpResponse Handle(HttpRequest request)
var cmd = splitted.Length < 2 ? "" : splitted[1];
var arg = splitted.Length < 3 ? "" : splitted[2];

if (_handlers.ContainsKey(cmd))
{
return _handlers[cmd](arg, request.Content);
if( //if
_handlers.ContainsKey(cmd) //handler contains in _handlers Dictionary
// && request.isBinary == false //and if it's not binary-request
//this was been commented, because binary requests can be accepted by handlers, as iso-8859-1 string, too.
){
return _handlers[cmd](arg, request.Content); //send request to handler, and return response.
//for binary-request, send request to "handler" with "iso-8859-1"-encoded string, and return response.
}

else
else if ( //else if
_handlers_bin.ContainsKey(cmd) //and if handler contains in binary handlers
)
{
return new ErrorHandler(StatusCode.BadRequest, "No such command: " + cmd + ". Available commands: " + JsonConvert.SerializeObject(_handlers.Keys.ToArray())).Handle(request);
if( //if
request.Content_bytes == null //but content not found
|| ( //or
(request.Content_bytes != null) //if it's found
&& (request.Content_bytes.Length == 0) //but empty
)
&& request.Content.Length != 0 //and if text found
){
// Console.WriteLine("request.Content"+request.Content); //it seems, like "iso-8859-1"-text sent, as string.
byte[] request_bytes = MimeType.DefaultBinaryEncoding.GetBytes(request.Content); //decode this to bytes, using reversive latin1 encoding.
return _handlers_bin[cmd](arg, request_bytes); //and send bytes to binary-handler, and return the binary response
}
else{ //else if it's was been bytes (Content-Type: application/octet-stream), and request.isBinary == true
return _handlers_bin[cmd](arg, request.Content_bytes); //send bytes to binary-handler, and return response
}
}
//else if "_handlers" or "_handlers_bin" dictionaries not contains handler "cmd" - return an error:
return new ErrorHandler(StatusCode.BadRequest, "No such command: " + cmd + ". Available commands: " + JsonConvert.SerializeObject(_handlers.Keys.ToArray())).Handle(request);
}

catch (Exception e)
{
return new ErrorHandler(StatusCode.InternalServerError, e.Message).Handle(request);
return new ErrorHandler(StatusCode.InternalServerError, e.Message).Handle(request); //or return an exception-message
}
}
}
Expand Down
Loading

0 comments on commit beb5bdb

Please sign in to comment.