Commit Graph

52 Commits

Author SHA1 Message Date
Mark Felder
e8bf060e6e Move construction of S3 base URL with optional namespace and bucket to Upload.base_url/0
Now we should have a correct base URL for S3 hosted objects throughout the codebase.
2021-01-08 17:32:42 -06:00
Mark Felder
3c936061d5 Apply Upload.base_url for S3 2021-01-08 17:32:42 -06:00
Alexander Strizhakov
a5987155f7
UploadedMedia module name 2020-10-13 16:42:51 +03:00
Alexander Strizhakov
84fbf16161
timeout option moved to gun adapter helper 2020-09-02 10:50:51 +03:00
Alexander Strizhakov
c17d83cd73
improvements and fixes for http requests
- fix for gun worker termination in some circumstances
- pool for http clients (ex_aws, tzdata)
- default pool timeouts for gun
- gun retries on gun_down messages
- s3 upload timeout if streaming enabled
2020-09-02 09:04:23 +03:00
Haelwenn (lanodan) Monnier
6da6540036
Bump copyright years of files changed after 2020-01-07
Done via the following command:
git diff fcd5dd259a --stat --name-only | xargs sed -i '/Pleroma Authors/c# Copyright © 2017-2020 Pleroma Authors <https:\/\/pleroma.social\/>'
2020-03-02 06:08:45 +01:00
Roman Chvanikov
88f0eed0f2 Delete attachments when status is deleted 2020-01-12 18:48:58 +00:00
Hakaba Hitoyo
0c9c62509d Remove MDII uploader 2020-01-11 17:19:54 +00:00
Rachel Fae Fox
63af6951fa add tunable for stream uploads, as needed for jortage to work. 2019-09-23 20:38:53 +00:00
Maksim
11d08c2de0 tests for Pleroma.Uploaders 2019-08-10 18:46:26 +00:00
Maksim
bb9c539580 Uploader.S3 added support stream uploads 2019-08-10 11:27:59 +00:00
Maksim
b6b748d3e7 tests for Uploader with webhook 2019-07-18 12:30:18 +00:00
Egor Kislitsyn
5104f65b69 Wrap error messages into gettext helpers 2019-07-10 18:10:09 +07:00
Egor Kislitsyn
4f42093220 Remove Uploaders.Swift 2019-07-03 14:11:04 +07:00
William Pitcock
9bec891eb4 kill @httpoison 2019-05-25 04:24:21 +00:00
Mark Felder
498bfdf403 Switch to Jason over Poison 2019-05-13 15:37:38 -05:00
Austin Lanari
6c96d68f77 add overriding truncated_namespace condition for truncating paths for digital ocean 2019-03-24 11:32:09 -07:00
Haelwenn (lanodan) Monnier
4b73ca638e
[Credo][CI] Add readability as it’s fixed 2019-03-13 04:26:56 +01:00
Haelwenn (lanodan) Monnier
c42d34b2ec
[Credo] fix Credo.Check.Readability.MaxLineLength 2019-03-13 04:26:56 +01:00
Haelwenn (lanodan) Monnier
fb82f6fc7c
[Credo] Remove parentesis on argument-less functions 2019-03-13 04:26:56 +01:00
hakabahitoyo
b7bc666200 bugfix mdii uploader 2019-02-13 15:46:42 +09:00
Haelwenn (lanodan) Monnier
106f4e7a0f
Credo fixes: parameter consistency 2019-02-09 14:59:20 +01:00
href
4aff4efa8d
Use multiple hackney pools
* federation (ap, salmon)
* media (rich media, media proxy)
* upload (uploader proxy)

Each "part" will stop fighting others ones -- a huge federation outbound
could before make the media proxy fail to checkout a connection in time.

splitted media and uploaded media for the good reason than an upload
pool will have all connections to the same host (the uploader upstream).
it also has a longer default retention period for connections.
2019-01-30 15:06:46 +01:00
kaniini
4a278cd80a Merge branch 's3-namespace' into 'develop'
S3 uploader: support for namespaced bucket

Closes #532

See merge request pleroma/pleroma!699
2019-01-23 13:25:04 +00:00
href
f9a3269099
Uploader callback controller 2019-01-21 22:44:14 +01:00
href
97412d9f94
S3 Namespace 2019-01-21 14:42:16 +01:00
William Pitcock
980b5288ed update copyright years to 2019 2018-12-31 15:41:47 +00:00
William Pitcock
2791ce9a1f add license boilerplate to pleroma core 2018-12-23 20:56:42 +00:00
Maksim Pechnikov
074fa790ba fix compile warnings 2018-12-09 20:50:08 +03:00
Maksim Pechnikov
87109482f3 status_code -> status 2018-12-04 14:04:06 +03:00
href
02d3dc6869
Uploads fun, part. 2 2018-11-30 18:02:37 +01:00
href
b19597f602
reverse proxy / uploads 2018-11-30 18:00:47 +01:00
hakabahitoyo
59e079f641 fallbacking into local uploader 2018-11-17 20:16:25 +09:00
hakabahitoyo
8fd0556c78 better config reading 2018-11-17 18:14:42 +09:00
hakabahitoyo
55abd8482e better config 2018-11-16 20:41:12 +09:00
hakabahitoyo
52224de39f better extension detection 2018-11-16 20:22:36 +09:00
hakabahitoyo
4fbfacf5e1 debug 2018-11-15 16:08:55 +09:00
hakabahitoyo
8e707aba29 format 2018-11-15 15:11:59 +09:00
Hakaba Hitoyo
ebe658c169 debuf 2018-11-15 14:46:43 +09:00
Hakaba Hitoyo
698cb3587c omplement mdii uploader 2018-11-15 14:38:45 +09:00
Hakaba Hitoyo
58af0787be add mdii uploader 2018-11-15 14:19:10 +09:00
href
5bb88fd174
Runtime configuration
Related to #85

Everything should now be configured at runtime, with the exception of
the `Pleroma.HTML` scrubbers (the scrubbers used can be
changed at runtime, but their configuration is compile-time) because
it's building a module with a macro.
2018-11-06 19:41:15 +01:00
William Pitcock
36825932eb s3 uploader: add new feature to force public attachment URIs to go through media proxy 2018-10-29 18:00:59 +00:00
Martin Kühl
f77ec96707 Uploaders.S3: Replace unsafe characters in object key
According to [the S3 docs][s3], the characters safe for use in object keys are:

* 0-9
* a-z
* A-Z
* !
* -
* _
* .
* *
* '
* (
* )

(The / character is not listed but mentioned being safe outside of the list.)

Several characters that are valid in filenames can cause problems, for example
spaces are not valid in URLs and need to be escaped,
sequences of spaces can become squeezed by S3,
some characters like \ are documented to require “significant special handling”.

To avoid these problems, this change encodes the filename
before using it as part of the S3 object name
by replacing all characters except those documented as “safe” with dashes.

[s3]: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html
2018-09-24 15:38:32 +02:00
Haelwenn (lanodan) Monnier
754deb26dd
[Pleroma.Uploaders.Local]: Add configuration for custom url path
One use-case being an external caching proxy
2018-09-02 19:00:16 +02:00
Thurloat
adffad5502 increase uploader behaviour documentation accuracy. 2018-08-30 09:20:29 -03:00
Thurloat
af01f0196a Add backend failure handling with :ok | :error so the uploader can handle it.
defaulting to :ok, since that's the currently level of error handling.
2018-08-29 22:07:28 -03:00
Thurloat
2ff25ac0ce A hobbldey-working swift client.
apparently, all elixir openstack libraries are trash
luckily, the APIs are stupid easy.
2018-08-28 22:32:24 -03:00
Thurloat
dad39b24a1 add the behaviour, work on actually making it work. 2018-08-28 19:48:03 -03:00
Thurloat
8d2d7a8859 Implement uploader behaviour
run formatter <#
2018-08-28 09:57:41 -03:00