feat: Attachment of Files to Documents using S3. #280

chanchadsahil7 · 2020-09-10T11:21:08Z

Background:
At Qure.ai we have been using the self-hosted outline and its feature for maintaining our internal documentation and knowledge base.

Problem:
We have been having a hard time uploading and maintaining files except for images on the outline that we are using.

Solution:

So we have built a draft of the files attachment using the original image upload feature and supported all kinds of files to be upload in the outline.
We understand that this isssue has been already requested.
The above issue requested Google Drive and Dropbox integration. However, we believe it is not a great direction to integrate third party for a feature as simple as file upload.
S3 is a files service provider over the cloud and is already an integral part of the outline service.
So we thought why not use the already existing S3 upload integration and support all kinds of files to be stored inside a document.

Thanks for reviewing my first open-source PR🙂

cc: @chsasank

tommoor · 2020-09-14T06:26:07Z

Hey @chanchadsahil7 – this is interesting and I'm glad you were able to fork to handle the functionality you needed. As it stands I don't think we'll be able to merge this one as it's kind of "abusing" the markdown image syntax to host files – we'd need a new way of representing files in the markdown that maintains the current image behavior.

chsasank · 2020-09-14T06:43:37Z

As it stands I don't think we'll be able to merge this one

Of course, this is only a proof of concept. We wanted to get feedback early as we develop so that the PR has higher chance of merging. We didn't want to complete the feature in a way that's not in line with your vision.

it's kind of "abusing" the markdown image syntax to host files

We used this to get a quick and dirty patch to demonstrate the feature. We mean to replace it with better code following your guidance.

we'd need a new way of representing files in the markdown

Indeed! How about GitHub's way of handling files as links? We can use some metadata to differentiate the files from links. We can also render the files the slack way.

Markdown also allows for embedded html so that's another option.

chanchadsahil7 · 2020-09-14T06:47:46Z

Hey @tommoor thanks for the feedback but as it stands this was just for the POC and not to disturb the current markdown images syntax to host files.

The idea is to create a similar functionality as markdown image with a file doc node in the markdown so that whenever a file is attached to the editor it creates a link to the doc uploaded.
Attaching the idea of the file doc node.

tommoor · 2020-09-16T01:28:57Z

Indeed! How about GitHub's way of handling files as links? We can use some metadata to differentiate the files from links. We can also render the files the slack way.

As links makes sense for a first version both for storage in the markdown and the visual display. Id suggest adding a callback with the same signature as the image callback and only show the option in the block menu when the callback is passed in. That way users of the library can opt in to handling file uploads.

We should also provide a prop that allows setting the accepted file types

chanchadsahil7 · 2020-09-18T12:02:55Z

I am glad that we are on the same page for the first version.

Below are the points that will be taken into consideration for this feature:

Build a file drop option on the block menu.
Upload files to S3 bucket.
Display files as links in the document.
Pass a callback similar to upload the image and have a prop to validate the file upload option to be displayed inside the block menu.

@tommoor @chsasank

I think we are good to go for the development of the first version.

Full Fledge File Upload to S3

chanchadsahil7 · 2020-10-06T05:18:16Z

@tommoor @chsasank

I have pushed the code for file upload which replicates the image upload functionality for all file types.

Features added.

Added Upload file callback.
Added Insert all files function.
Added a file node similar to the image node which adds files as a link in the doc.

Issues:

File node-link getting converted to an image node-link after saving the document.

tommoor · 2020-10-07T06:10:40Z

File node-link getting converted to an image node-link after saving the document.

I don't think you need the FileDoc component at all, just insert the uploaded url as a regular link and store it as a markdown link instead of a markdown image. Would this work for your usecase?

chanchadsahil7 · 2020-10-07T06:51:23Z

File node-link getting converted to an image node-link after saving the document.

I don't think you need the FileDoc component at all, just insert the uploaded url as a regular link and store it as a markdown link instead of a markdown image. Would this work for your usecase?

Makes Sense @tommoor Thanks will make the changes.

chanchadsahil7 · 2020-10-07T11:38:32Z

@tommoor

Used the markdown link for the file upload and showing in the document.
Removed the file doc node that I created previously and used the existing createAndInsertLink function for inserting the link to the file.

Issue:
After the file is created the href attr inside the link is wrongly parsing the URL of the file and creating a document as a draft instead of just giving a file URL as output.

tommoor · 2020-10-08T02:20:43Z

the href attr inside the link is wrongly parsing the URL of the file and creating a document as a draft

I'm afraid I don't know what you mean – make a gif?

chanchadsahil7 · 2020-10-08T05:04:10Z

I think I got the issue!

the href attr inside the link is wrongly parsing the URL of the file and creating a document as a draft

I'm afraid I don't know what you mean – make a gif?

I think I got the fix for it.

chanchadsahil7 · 2020-10-13T05:17:03Z

Any thoughts on the PR?

tommoor · 2020-10-13T05:36:38Z

example/src/index.js

+            // Delay to simulate time taken to upload
+            return new Promise(resolve => {
+              setTimeout(
+                () => resolve("https://loremflickr.com/1000/1000"),


Probably best to use a non-image for this example, otherwise it's just confusing.

yep got it.

tommoor · 2020-10-13T05:38:54Z

src/commands/insertAllFiles.ts

+    // start uploading the file file to the server. Using "then" syntax
+    // to allow all placeholders to be entered at once with the uploads
+    // happening in the background in parallel.
+    uploadFile(file)


The delay between the user starting the upload and the files being inserted will result in strange and potentially broken scenarios – the document content can have changed substantially in that time.

With the image extension a placeholder widget is used to hold the position for this reason, but a better comparison might be the logic when creating a document from the link menu – a "placeholder" link is created and then updated when the document has been successfully created. A similar approach seems necessary here.

let me know if I got this right:

You are suggesting that as soon as a user selects a file using the menu option you want me to insert a place holder to the document and save it. And then start the file upload process. Once the file is uploaded just replace the link.

is that what you are suggesting?

Yes, here's the existing code:

rich-markdown-editor/src/components/LinkToolbar.tsx

Lines 71 to 88 in 7b03109

const href = `creating#${title}…`;

// Insert a placeholder link

dispatch(

view.state.tr

.insertText(title, from, to)

.addMark(

from,

to + title.length,

state.schema.marks.link.create({ href })

)

);

createAndInsertLink(view, title, href, {

onCreateLink,

onShowToast,

dictionary,

});

A "placeholder" link is created with a temporary url while the document is being created. Once it's uploaded the href is swapped out. We could also target the temporary link in css to maybe fade it out or similar while the upload is happening.

tommoor · 2020-10-13T05:39:54Z

src/components/BlockMenu.tsx

+                type="file"
+                ref={this.fileInputRef}
+                onChange={this.handleFilePicked}
+                accept="*"


Makes sense to make this a prop

are you talking about the default accept prop?

tommoor · 2020-10-13T05:40:08Z

src/index.tsx

+  .file {
+    a {
+      pointer-events: ${props => (props.readOnly ? "initial" : "none")};
+    }
+  }
+
+  .file.placeholder {
+    position: relative;
+    background: ${props => props.theme.background};
+
+    a {
+      opacity: 0.5;
+    }
+  }


This seems to be unused

sure will remove this

tommoor · 2020-10-13T05:40:26Z

src/lib/uploadFilePlaceholder.ts

@@ -0,0 +1,51 @@
+import { Plugin } from "prosemirror-state";


The entire thing here seems to be unused now it's no longer based on an image?

sure will remove this

tommoor · 2020-10-13T05:40:47Z

src/menus/block.ts

+    {
+      name: "file",
+      title: "File",
+      icon: ImageIcon,


Going to need a unique icon here – there isn't a simple "file" icon in the set, but I can get one added.

Yes please get it added will use the same Icon here

tommoor · 2020-10-13T05:41:30Z

src/menus/block.ts

+      name: "file",
+      title: "File",
+      icon: ImageIcon,
+      keywords: "file doc pdf",


Suggested change

keywords: "file doc pdf",

keywords: "doc pdf",

No need to include "file" as the name and title are included in the keywords automatically

Okay got it

marconunnari · 2020-12-14T14:27:18Z

Hello 👋 Great work on outline and this, are there any news about this PR?

chanchadsahil7 · 2020-12-14T14:55:13Z

Hey, I will be soon pushing the final code for the Feature.

…

On Mon, 14 Dec, 2020, 7:57 pm Marco Nunnari, ***@***.***> wrote: Hello 👋 Great work on outline and this, do we have any news about this PR? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQMDQLPE7GCVBBQSH6L5N3SUYOFNANCNFSM4RE2OJXQ> .

malekhijazi · 2021-03-10T10:51:02Z

@chanchadsahil7 any updates on this? Need any help💪 ?

warnus · 2021-08-13T03:33:11Z

Thanks for sharing a great project.
Do you have any plans to merge this PR?

tommoor · 2021-08-13T04:18:17Z

Honestly review was never requested again so it disappeared off my GitHub "reviews requested" list, definitely open to this.

chanchadsahil7 changed the title ~~Attachment Files : S3 Bucket~~ Attachment of Files to Documents using S3. Sep 10, 2020

chanchadsahil7 force-pushed the develop branch 2 times, most recently from af511c6 to 757eded Compare September 12, 2020 13:53

First draft for files upload to S3

bc94f6f

chanchadsahil7 force-pushed the develop branch from 757eded to bc94f6f Compare September 12, 2020 14:03

All file Type support without hendering Image.

29e77ce

sahilchanchad and others added 2 commits October 6, 2020 10:34

Added fildoc node similar to image node

cec2dfd

Merge pull request #1 from chanchadsahil7/sahil/full-fledge-file-upload

cc1a066

Full Fledge File Upload to S3

tommoor changed the title ~~Attachment of Files to Documents using S3.~~ feat: Attachment of Files to Documents using S3. Oct 7, 2020

Using mark link for adding files in document

49a0b72

removed createlink for removing creation of doc.

f305d33

chanchadsahil7 mentioned this pull request Oct 8, 2020

feat: Attachment of Files to Documents using S3 outline/outline#1588

Closed

tommoor reviewed Oct 13, 2020

View reviewed changes

Minor Code review changes

bd2971d

Base automatically changed from develop to main February 3, 2021 16:49

FeralMib mentioned this pull request Dec 2, 2021

File Attachments outline/outline#859

Closed

FeralMib mentioned this pull request Dec 15, 2021

feat: Attachment of Files to Documents using S3. #603

Open

warnus mentioned this pull request Jan 13, 2022

feat: Attachment of files as block to documents using s3 #624

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Attachment of Files to Documents using S3. #280

feat: Attachment of Files to Documents using S3. #280

chanchadsahil7 commented Sep 10, 2020 •

edited

Loading

tommoor commented Sep 14, 2020

chsasank commented Sep 14, 2020 •

edited

Loading

chanchadsahil7 commented Sep 14, 2020 •

edited

Loading

tommoor commented Sep 16, 2020

chanchadsahil7 commented Sep 18, 2020

chanchadsahil7 commented Oct 6, 2020

tommoor commented Oct 7, 2020

chanchadsahil7 commented Oct 7, 2020

chanchadsahil7 commented Oct 7, 2020 •

edited

Loading

tommoor commented Oct 8, 2020

chanchadsahil7 commented Oct 8, 2020

chanchadsahil7 commented Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

tommoor Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

tommoor Oct 13, 2020

chanchadsahil7 Oct 13, 2020

marconunnari commented Dec 14, 2020 •

edited

Loading

chanchadsahil7 commented Dec 14, 2020 via email

malekhijazi commented Mar 10, 2021

warnus commented Aug 13, 2021

tommoor commented Aug 13, 2021

	const href = `creating#${title}…`;

	// Insert a placeholder link
	dispatch(
	view.state.tr
	.insertText(title, from, to)
	.addMark(
	from,
	to + title.length,
	state.schema.marks.link.create({ href })
	)
	);

	createAndInsertLink(view, title, href, {
	onCreateLink,
	onShowToast,
	dictionary,
	});

feat: Attachment of Files to Documents using S3. #280

Are you sure you want to change the base?

feat: Attachment of Files to Documents using S3. #280

Conversation

chanchadsahil7 commented Sep 10, 2020 • edited Loading

tommoor commented Sep 14, 2020

chsasank commented Sep 14, 2020 • edited Loading

chanchadsahil7 commented Sep 14, 2020 • edited Loading

tommoor commented Sep 16, 2020

chanchadsahil7 commented Sep 18, 2020

chanchadsahil7 commented Oct 6, 2020

tommoor commented Oct 7, 2020

chanchadsahil7 commented Oct 7, 2020

chanchadsahil7 commented Oct 7, 2020 • edited Loading

tommoor commented Oct 8, 2020

chanchadsahil7 commented Oct 8, 2020

chanchadsahil7 commented Oct 13, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marconunnari commented Dec 14, 2020 • edited Loading

chanchadsahil7 commented Dec 14, 2020 via email

malekhijazi commented Mar 10, 2021

warnus commented Aug 13, 2021

tommoor commented Aug 13, 2021

chanchadsahil7 commented Sep 10, 2020 •

edited

Loading

chsasank commented Sep 14, 2020 •

edited

Loading

chanchadsahil7 commented Sep 14, 2020 •

edited

Loading

chanchadsahil7 commented Oct 7, 2020 •

edited

Loading

marconunnari commented Dec 14, 2020 •

edited

Loading