sergiotarxz/Pleroma
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'feature/delete-validator' of git.pleroma.social:pleroma/pleroma into feature/delete-validator

Browse Source
This commit is contained in:
lain 2020-05-01 13:35:42 +02:00
commit ab01863abf
17 changed files with 1580 additions and 947 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
CHANGELOG.md
View File

@ -4,6 +4,13 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [unreleased]
### Changed
<details>
<summary>API Changes</summary>
- **Breaking:** Emoji API: changed methods and renamed routes.
</details>
### Removed
- **Breaking:** removed `with_move` parameter from notifications timeline.
@ -27,6 +34,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Support pagination in conversations API
- **Breaking**: SimplePolicy `:reject` and `:accept` allow deletions again
- Fix follower/blocks import when nicknames starts with @
- Filtering of push notifications on activities from blocked domains
## [unreleased-patch]
### Fixed
@ -129,7 +137,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- **Breaking:** Admin API: Return link alongside with token on password reset
- **Breaking:** Admin API: `PUT /api/pleroma/admin/reports/:id` is now `PATCH /api/pleroma/admin/reports`, see admin_api.md for details
- **Breaking:** `/api/pleroma/admin/users/invite_token` now uses `POST`, changed accepted params and returns full invite in json instead of only token string.
- **Breaking** replying to reports is now "report notes", enpoint changed from `POST /api/pleroma/admin/reports/:id/respond` to `POST /api/pleroma/admin/reports/:id/notes`
- **Breaking** replying to reports is now "report notes", endpoint changed from `POST /api/pleroma/admin/reports/:id/respond` to `POST /api/pleroma/admin/reports/:id/notes`
- Mastodon API: stopped sanitizing display names, field names and subject fields since they are supposed to be treated as plaintext
- Admin API: Return `total` when querying for reports
- Mastodon API: Return `pleroma.direct_conversation_id` when creating a direct message (`POST /api/v1/statuses`)

118
docs/API/pleroma_api.md
View File

@ -323,20 +323,54 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
* Params: None
* Response: JSON, returns a list of Mastodon Conversation entities that were marked as read (200 - healthy, 503 unhealthy).
## `GET /api/pleroma/emoji/packs`
### Lists the custom emoji packs on the server
## `GET /api/pleroma/emoji/packs/import`
### Imports packs from filesystem
* Method `GET`
* Authentication: not required
* Authentication: required
* Params: None
* Response: JSON, "ok" and 200 status and the JSON hashmap of "pack name" to "pack contents"
* Response: JSON, returns a list of imported packs.
## `PUT /api/pleroma/emoji/packs/:name`
### Creates an empty custom emoji pack
* Method `PUT`
## `GET /api/pleroma/emoji/packs/remote`
### Make request to another instance for packs list
* Method `GET`
* Authentication: required
* Params:
* `url`: url of the instance to get packs from
* Response: JSON with the pack list, hashmap with pack name and pack contents
## `POST /api/pleroma/emoji/packs/download`
### Download pack from another instance
* Method `POST`
* Authentication: required
* Params:
* `url`: url of the instance to download from
* `name`: pack to download from that instance
* `as`: (*optional*) name how to save pack
* Response: JSON, "ok" with 200 status if the pack was downloaded, or 500 if there were
errors downloading the pack
## `POST /api/pleroma/emoji/packs/:name`
### Creates an empty pack
* Method `POST`
* Authentication: required
* Params: None
* Response: JSON, "ok" and 200 status or 409 if the pack with that name already exists
## `PATCH /api/pleroma/emoji/packs/:name`
### Updates (replaces) pack metadata
* Method `PATCH`
* Authentication: required
* Params:
* `metadata`: metadata to replace the old one
* `license`: Pack license
* `homepage`: Pack home page url
* `description`: Pack description
* `fallback-src`: Fallback url to download pack from
* `fallback-src-sha256`: SHA256 encoded for fallback pack archive
* `share-files`: is pack allowed for sharing (boolean)
* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
problem with the new metadata (the error is specified in the "error" part of the response JSON)
## `DELETE /api/pleroma/emoji/packs/:name`
### Delete a custom emoji pack
* Method `DELETE`
@ -344,53 +378,51 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
* Params: None
* Response: JSON, "ok" and 200 status or 500 if there was an error deleting the pack
## `POST /api/pleroma/emoji/packs/:name/update_file`
### Update a file in a custom emoji pack
## `POST /api/pleroma/emoji/packs/:name/files`
### Add new file to the pack
* Method `POST`
* Authentication: required
* Params:
* if the `action` is `add`, adds an emoji named `shortcode` to the pack `pack_name`,
that means that the emoji file needs to be uploaded with the request
(thus requiring it to be a multipart request) and be named `file`.
There can also be an optional `filename` that will be the new emoji file name
(if it's not there, the name will be taken from the uploaded file).
* if the `action` is `update`, changes emoji shortcode
(from `shortcode` to `new_shortcode` or moves the file (from the current filename to `new_filename`)
* if the `action` is `remove`, removes the emoji named `shortcode` and it's associated file
* Response: JSON, updated "files" section of the pack and 200 status, 409 if the trying to use a shortcode
that is already taken, 400 if there was an error with the shortcode, filename or file (additional info
in the "error" part of the response JSON)
* `file`: file needs to be uploaded with the multipart request or link to remote file.
* `shortcode`: (*optional*) shortcode for new emoji, must be uniq for all emoji. If not sended, shortcode will be taken from original filename.
* `filename`: (*optional*) new emoji file name. If not specified will be taken from original filename.
* Response: JSON, list of files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
## `POST /api/pleroma/emoji/packs/:name/update_metadata`
### Updates (replaces) pack metadata
* Method `POST`
## `PATCH /api/pleroma/emoji/packs/:name/files`
### Update emoji file from pack
* Method `PATCH`
* Authentication: required
* Params:
* `new_data`: new metadata to replace the old one
* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
problem with the new metadata (the error is specified in the "error" part of the response JSON)
* `shortcode`: emoji file shortcode
* `new_shortcode`: new emoji file shortcode
* `new_filename`: new filename for emoji file
* `force`: (*optional*) with true value to overwrite existing emoji with new shortcode
* Response: JSON, list with updated files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
## `POST /api/pleroma/emoji/packs/download_from`
### Requests the instance to download the pack from another instance
* Method `POST`
## `DELETE /api/pleroma/emoji/packs/:name/files`
### Delete emoji file from pack
* Method `DELETE`
* Authentication: required
* Params:
* `instance_address`: the address of the instance to download from
* `pack_name`: the pack to download from that instance
* Response: JSON, "ok" and 200 status if the pack was downloaded, or 500 if there were
errors downloading the pack
* `shortcode`: emoji file shortcode
* Response: JSON, list with updated files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
## `POST /api/pleroma/emoji/packs/list_from`
### Requests the instance to list the packs from another instance
* Method `POST`
* Authentication: required
* Params:
* `instance_address`: the address of the instance to download from
* Response: JSON with the pack list, same as if the request was made to that instance's
list endpoint directly + 200 status
## `GET /api/pleroma/emoji/packs`
### Lists local custom emoji packs
* Method `GET`
* Authentication: not required
* Params: None
* Response: JSON, "ok" and 200 status and the JSON hashmap of pack name to pack contents
## `GET /api/pleroma/emoji/packs/:name/download_shared`
### Requests a local pack from the instance
## `GET /api/pleroma/emoji/packs/:name`
### Get pack.json for the pack
* Method `GET`
* Authentication: not required
* Params: None
* Response: JSON, pack json with `files` and `pack` keys with 200 status or 404 if the pack does not exist
## `GET /api/pleroma/emoji/packs/:name/archive`
### Requests a local pack archive from the instance
* Method `GET`
* Authentication: not required
* Params: None

507
lib/pleroma/emoji/pack.ex Normal file
View File

@ -0,0 +1,507 @@
defmodule Pleroma.Emoji.Pack do
@derive {Jason.Encoder, only: [:files, :pack]}
defstruct files: %{},
pack_file: nil,
path: nil,
pack: %{},
name: nil
@type t() :: %__MODULE__{
files: %{String.t() => Path.t()},
pack_file: Path.t(),
path: Path.t(),
pack: map(),
name: String.t()
}
alias Pleroma.Emoji
@spec emoji_path() :: Path.t()
def emoji_path do
static = Pleroma.Config.get!([:instance, :static_dir])
Path.join(static, "emoji")
end
@spec create(String.t()) :: :ok | {:error, File.posix()} | {:error, :empty_values}
def create(name) when byte_size(name) > 0 do
dir = Path.join(emoji_path(), name)
with :ok <- File.mkdir(dir) do
%__MODULE__{
pack_file: Path.join(dir, "pack.json")
}
|> save_pack()
end
end
def create(_), do: {:error, :empty_values}
@spec show(String.t()) :: {:ok, t()} | {:loaded, nil} | {:error, :empty_values}
def show(name) when byte_size(name) > 0 do
with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
{_, pack} <- validate_pack(pack) do
{:ok, pack}
end
end
def show(_), do: {:error, :empty_values}
@spec delete(String.t()) ::
{:ok, [binary()]} | {:error, File.posix(), binary()} | {:error, :empty_values}
def delete(name) when byte_size(name) > 0 do
emoji_path()
|> Path.join(name)
|> File.rm_rf()
end
def delete(_), do: {:error, :empty_values}
@spec add_file(String.t(), String.t(), Path.t(), Plug.Upload.t() | String.t()) ::
{:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
def add_file(name, shortcode, filename, file)
when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(filename) > 0 do
with {_, nil} <- {:exists, Emoji.get(shortcode)},
{_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)} do
file_path = Path.join(pack.path, filename)
create_subdirs(file_path)
case file do
%Plug.Upload{path: upload_path} ->
# Copy the uploaded file from the temporary directory
File.copy!(upload_path, file_path)
url when is_binary(url) ->
# Download and write the file
file_contents = Tesla.get!(url).body
File.write!(file_path, file_contents)
end
files = Map.put(pack.files, shortcode, filename)
updated_pack = %{pack | files: files}
case save_pack(updated_pack) do
:ok ->
Emoji.reload()
{:ok, updated_pack}
e ->
e
end
end
end
def add_file(_, _, _, _), do: {:error, :empty_values}
defp create_subdirs(file_path) do
if String.contains?(file_path, "/") do
file_path
|> Path.dirname()
|> File.mkdir_p!()
end
end
@spec delete_file(String.t(), String.t()) ::
{:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
def delete_file(name, shortcode) when byte_size(name) > 0 and byte_size(shortcode) > 0 do
with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
{_, {filename, files}} when not is_nil(filename) <-
{:exists, Map.pop(pack.files, shortcode)},
emoji <- Path.join(pack.path, filename),
{_, true} <- {:exists, File.exists?(emoji)} do
emoji_dir = Path.dirname(emoji)
File.rm!(emoji)
if String.contains?(filename, "/") and File.ls!(emoji_dir) == [] do
File.rmdir!(emoji_dir)
end
updated_pack = %{pack | files: files}
case save_pack(updated_pack) do
:ok ->
Emoji.reload()
{:ok, updated_pack}
e ->
e
end
end
end
def delete_file(_, _), do: {:error, :empty_values}
@spec update_file(String.t(), String.t(), String.t(), String.t(), boolean()) ::
{:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
def update_file(name, shortcode, new_shortcode, new_filename, force)
when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(new_shortcode) > 0 and
byte_size(new_filename) > 0 do
with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
{_, {filename, files}} when not is_nil(filename) <-
{:exists, Map.pop(pack.files, shortcode)},
{_, true} <- {:not_used, force or is_nil(Emoji.get(new_shortcode))} do
old_path = Path.join(pack.path, filename)
old_dir = Path.dirname(old_path)
new_path = Path.join(pack.path, new_filename)
create_subdirs(new_path)
:ok = File.rename(old_path, new_path)
if String.contains?(filename, "/") and File.ls!(old_dir) == [] do
File.rmdir!(old_dir)
end
files = Map.put(files, new_shortcode, new_filename)
updated_pack = %{pack | files: files}
case save_pack(updated_pack) do
:ok ->
Emoji.reload()
{:ok, updated_pack}
e ->
e
end
end
end
def update_file(_, _, _, _, _), do: {:error, :empty_values}
@spec import_from_filesystem() :: {:ok, [String.t()]} | {:error, atom()}
def import_from_filesystem do
emoji_path = emoji_path()
with {:ok, %{access: :read_write}} <- File.stat(emoji_path),
{:ok, results} <- File.ls(emoji_path) do
names =
results
|> Enum.map(&Path.join(emoji_path, &1))
|> Enum.reject(fn path ->
File.dir?(path) and File.exists?(Path.join(path, "pack.json"))
end)
|> Enum.map(&write_pack_contents/1)
|> Enum.filter(& &1)
{:ok, names}
else
{:ok, %{access: _}} -> {:error, :no_read_write}
e -> e
end
end
defp write_pack_contents(path) do
pack = %__MODULE__{
files: files_from_path(path),
path: path,
pack_file: Path.join(path, "pack.json")
}
case save_pack(pack) do
:ok -> Path.basename(path)
_ -> nil
end
end
defp files_from_path(path) do
txt_path = Path.join(path, "emoji.txt")
if File.exists?(txt_path) do
# There's an emoji.txt file, it's likely from a pack installed by the pack manager.
# Make a pack.json file from the contents of that emoji.txt file
# FIXME: Copy-pasted from Pleroma.Emoji/load_from_file_stream/2
# Create a map of shortcodes to filenames from emoji.txt
File.read!(txt_path)
|> String.split("\n")
|> Enum.map(&String.trim/1)
|> Enum.map(fn line ->
case String.split(line, ~r/,\s*/) do
# This matches both strings with and without tags
# and we don't care about tags here
[name, file | _] ->
file_dir_name = Path.dirname(file)
file =
if String.ends_with?(path, file_dir_name) do
Path.basename(file)
else
file
end
{name, file}
_ ->
nil
end
end)
|> Enum.filter(& &1)
|> Enum.into(%{})
else
# If there's no emoji.txt, assume all files
# that are of certain extensions from the config are emojis and import them all
pack_extensions = Pleroma.Config.get!([:emoji, :pack_extensions])
Emoji.Loader.make_shortcode_to_file_map(path, pack_extensions)
end
end
@spec list_remote(String.t()) :: {:ok, map()}
def list_remote(url) do
uri =
url
|> String.trim()
|> URI.parse()
with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
packs =
uri
|> URI.merge("/api/pleroma/emoji/packs")
|> to_string()
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
{:ok, packs}
end
end
@spec list_local() :: {:ok, map()}
def list_local do
emoji_path = emoji_path()
# Create the directory first if it does not exist. This is probably the first request made
# with the API so it should be sufficient
with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_path)},
{:ls, {:ok, results}} <- {:ls, File.ls(emoji_path)} do
packs =
results
|> Enum.map(&load_pack/1)
|> Enum.filter(& &1)
|> Enum.map(&validate_pack/1)
|> Map.new()
{:ok, packs}
end
end
defp validate_pack(pack) do
if downloadable?(pack) do
archive = fetch_archive(pack)
archive_sha = :crypto.hash(:sha256, archive) |> Base.encode16()
info =
pack.pack
|> Map.put("can-download", true)
|> Map.put("download-sha256", archive_sha)
{pack.name, Map.put(pack, :pack, info)}
else
info = Map.put(pack.pack, "can-download", false)
{pack.name, Map.put(pack, :pack, info)}
end
end
defp downloadable?(pack) do
# If the pack is set as shared, check if it can be downloaded
# That means that when asked, the pack can be packed and sent to the remote
# Otherwise, they'd have to download it from external-src
pack.pack["share-files"] &&
Enum.all?(pack.files, fn {_, file} ->
File.exists?(Path.join(pack.path, file))
end)
end
@spec get_archive(String.t()) :: {:ok, binary()}
def get_archive(name) do
with {_, %__MODULE__{} = pack} <- {:exists?, load_pack(name)},
{_, true} <- {:can_download?, downloadable?(pack)} do
{:ok, fetch_archive(pack)}
end
end
defp fetch_archive(pack) do
hash = :crypto.hash(:md5, File.read!(pack.pack_file))
case Cachex.get!(:emoji_packs_cache, pack.name) do
%{hash: ^hash, pack_data: archive} ->
archive
_ ->
create_archive_and_cache(pack, hash)
end
end
defp create_archive_and_cache(pack, hash) do
files = ['pack.json' | Enum.map(pack.files, fn {_, file} -> to_charlist(file) end)]
{:ok, {_, result}} =
:zip.zip('#{pack.name}.zip', files, [:memory, cwd: to_charlist(pack.path)])
ttl_per_file = Pleroma.Config.get!([:emoji, :shared_pack_cache_seconds_per_file])
overall_ttl = :timer.seconds(ttl_per_file * Enum.count(files))
Cachex.put!(
:emoji_packs_cache,
pack.name,
# if pack.json MD5 changes, the cache is not valid anymore
%{hash: hash, pack_data: result},
# Add a minute to cache time for every file in the pack
ttl: overall_ttl
)
result
end
@spec download(String.t(), String.t(), String.t()) :: :ok
def download(name, url, as) do
uri =
url
|> String.trim()
|> URI.parse()
with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
remote_pack =
uri
|> URI.merge("/api/pleroma/emoji/packs/#{name}")
|> to_string()
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
result =
case remote_pack["pack"] do
%{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
{:ok,
%{
sha: sha,
url: URI.merge(uri, "/api/pleroma/emoji/packs/#{name}/archive") |> to_string()
}}
%{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
{:ok,
%{
sha: sha,
url: src,
fallback: true
}}
_ ->
{:error,
"The pack was not set as shared and there is no fallback src to download from"}
end
with {:ok, %{sha: sha, url: url} = pinfo} <- result,
%{body: archive} <- Tesla.get!(url),
{_, true} <- {:checksum, Base.decode16!(sha) == :crypto.hash(:sha256, archive)} do
local_name = as || name
path = Path.join(emoji_path(), local_name)
pack = %__MODULE__{
name: local_name,
path: path,
files: remote_pack["files"],
pack_file: Path.join(path, "pack.json")
}
File.mkdir_p!(pack.path)
files = Enum.map(remote_pack["files"], fn {_, path} -> to_charlist(path) end)
# Fallback cannot contain a pack.json file
files = if pinfo[:fallback], do: files, else: ['pack.json' | files]
{:ok, _} = :zip.unzip(archive, cwd: to_charlist(pack.path), file_list: files)
# Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
# in it to depend on itself
if pinfo[:fallback] do
save_pack(pack)
end
:ok
end
end
end
defp save_pack(pack), do: File.write(pack.pack_file, Jason.encode!(pack, pretty: true))
@spec save_metadata(map(), t()) :: {:ok, t()} | {:error, File.posix()}
def save_metadata(metadata, %__MODULE__{} = pack) do
pack = Map.put(pack, :pack, metadata)
with :ok <- save_pack(pack) do
{:ok, pack}
end
end
@spec update_metadata(String.t(), map()) :: {:ok, t()} | {:error, File.posix()}
def update_metadata(name, data) do
pack = load_pack(name)
fb_sha_changed? =
not is_nil(data["fallback-src"]) and data["fallback-src"] != pack.pack["fallback-src"]
with {_, true} <- {:update?, fb_sha_changed?},
{:ok, %{body: zip}} <- Tesla.get(data["fallback-src"]),
{:ok, f_list} <- :zip.unzip(zip, [:memory]),
{_, true} <- {:has_all_files?, has_all_files?(pack.files, f_list)} do
fallback_sha = :crypto.hash(:sha256, zip) |> Base.encode16()
data
|> Map.put("fallback-src-sha256", fallback_sha)
|> save_metadata(pack)
else
{:update?, _} -> save_metadata(data, pack)
e -> e
end
end
# Check if all files from the pack.json are in the archive
defp has_all_files?(files, f_list) do
Enum.all?(files, fn {_, from_manifest} ->
List.keyfind(f_list, to_charlist(from_manifest), 0)
end)
end
@spec load_pack(String.t()) :: t() | nil
def load_pack(name) do
pack_file = Path.join([emoji_path(), name, "pack.json"])
if File.exists?(pack_file) do
pack_file
|> File.read!()
|> from_json()
|> Map.put(:pack_file, pack_file)
|> Map.put(:path, Path.dirname(pack_file))
|> Map.put(:name, name)
end
end
defp from_json(json) do
map = Jason.decode!(json)
struct(__MODULE__, %{files: map["files"], pack: map["pack"]})
end
defp shareable_packs_available?(uri) do
uri
|> URI.merge("/.well-known/nodeinfo")
|> to_string()
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
|> Map.get("links")
|> List.last()
|> Map.get("href")
# Get the actual nodeinfo address and fetch it
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
|> get_in(["metadata", "features"])
|> Enum.member?("shareable_emoji_packs")
end
end

63
lib/pleroma/following_relationship.ex
View File

@ -10,11 +10,12 @@ defmodule Pleroma.FollowingRelationship do
alias Ecto.Changeset
alias FlakeId.Ecto.CompatType
alias Pleroma.FollowingRelationship.State
alias Pleroma.Repo
alias Pleroma.User
schema "following_relationships" do
field(:state, Pleroma.FollowingRelationship.State, default: :follow_pending)
field(:state, State, default: :follow_pending)
belongs_to(:follower, User, type: CompatType)
belongs_to(:following, User, type: CompatType)
@ -22,6 +23,11 @@ defmodule Pleroma.FollowingRelationship do
timestamps()
end
@doc "Returns underlying integer code for state atom"
def state_int_code(state_atom), do: State.__enum_map__() |> Keyword.fetch!(state_atom)
def accept_state_code, do: state_int_code(:follow_accept)
def changeset(%__MODULE__{} = following_relationship, attrs) do
following_relationship
|> cast(attrs, [:state])
@ -82,6 +88,29 @@ def follower_count(%User{} = user) do
|> Repo.aggregate(:count, :id)
end
def followers_query(%User{} = user) do
__MODULE__
|> join(:inner, [r], u in User, on: r.follower_id == u.id)
|> where([r], r.following_id == ^user.id)
|> where([r], r.state == ^:follow_accept)
end
def followers_ap_ids(%User{} = user, from_ap_ids \\ nil) do
query =
user
|> followers_query()
|> select([r, u], u.ap_id)
query =
if from_ap_ids do
where(query, [r, u], u.ap_id in ^from_ap_ids)
else
query
end
Repo.all(query)
end
def following_count(%User{id: nil}), do: 0
def following_count(%User{} = user) do
@ -105,12 +134,16 @@ def following?(%User{id: follower_id}, %User{id: followed_id}) do
|> Repo.exists?()
end
def following(%User{} = user) do
following =
def following_query(%User{} = user) do
__MODULE__
|> join(:inner, [r], u in User, on: r.following_id == u.id)
|> where([r], r.follower_id == ^user.id)
|> where([r], r.state == ^:follow_accept)
end
def following(%User{} = user) do
following =
following_query(user)
|> select([r, u], u.follower_address)
|> Repo.all()
@ -171,6 +204,30 @@ def find(following_relationships, follower, following) do
end)
end
@doc """
For a query with joined activity,
keeps rows where activity's actor is followed by user -or- is NOT domain-blocked by user.
"""
def keep_following_or_not_domain_blocked(query, user) do
where(
query,
[_, activity],
fragment(
# "(actor's domain NOT in domain_blocks) OR (actor IS in followed AP IDs)"
"""
NOT (substring(? from '.*://([^/]*)') = ANY(?)) OR
? = ANY(SELECT ap_id FROM users AS u INNER JOIN following_relationships AS fr
ON u.id = fr.following_id WHERE fr.follower_id = ? AND fr.state = ?)
""",
activity.actor,
^user.domain_blocks,
activity.actor,
^User.binary_id(user.id),
^accept_state_code()
)
)
end
defp validate_not_self_relationship(%Changeset{} = changeset) do
changeset
|> validate_follower_id_following_id_inequality()

61
lib/pleroma/notification.ex
View File

@ -6,6 +6,7 @@ defmodule Pleroma.Notification do
use Ecto.Schema
alias Pleroma.Activity
alias Pleroma.FollowingRelationship
alias Pleroma.Notification
alias Pleroma.Object
alias Pleroma.Pagination
@ -81,15 +82,13 @@ def for_user_query(user, opts \\ %{}) do
|> exclude_visibility(opts)
end
# Excludes blocked users and non-followed domain-blocked users
defp exclude_blocked(query, user, opts) do
blocked_ap_ids = opts[:blocked_users_ap_ids] || User.blocked_users_ap_ids(user)
query
|> where([n, a], a.actor not in ^blocked_ap_ids)
|> where(
[n, a],
fragment("substring(? from '.*://([^/]*)')", a.actor) not in ^user.domain_blocks
)
|> FollowingRelationship.keep_following_or_not_domain_blocked(user)
end
defp exclude_notification_muted(query, _, %{@include_muted_option => true}) do
@ -262,6 +261,16 @@ def destroy_multiple(%{id: user_id} = _user, ids) do
|> Repo.delete_all()
end
def dismiss(%Pleroma.Activity{} = activity) do
Notification
|> where([n], n.activity_id == ^activity.id)
|> Repo.delete_all()
|> case do
{_, notifications} -> {:ok, notifications}
_ -> {:error, "Cannot dismiss notification"}
end
end
def dismiss(%{id: user_id} = _user, id) do
notification = Repo.get(Notification, id)
@ -330,10 +339,11 @@ def create_notification(%Activity{} = activity, %User{} = user, do_send \\ true)
@doc """
Returns a tuple with 2 elements:
{enabled notification receivers, currently disabled receivers (blocking / [thread] muting)}
{notification-enabled receivers, currently disabled receivers (blocking / [thread] muting)}
NOTE: might be called for FAKE Activities, see ActivityPub.Utils.get_notified_from_object/1
"""
@spec get_notified_from_activity(Activity.t(), boolean()) :: {list(User.t()), list(User.t())}
def get_notified_from_activity(activity, local_only \\ true)
def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, local_only)
@ -346,17 +356,14 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
|> Utils.maybe_notify_followers(activity)
|> Enum.uniq()
# Since even subscribers and followers can mute / thread-mute, filtering all above AP IDs
potential_receivers = User.get_users_from_set(potential_receiver_ap_ids, local_only)
notification_enabled_ap_ids =
potential_receiver_ap_ids
|> exclude_domain_blocker_ap_ids(activity, potential_receivers)
|> exclude_relationship_restricted_ap_ids(activity)
|> exclude_thread_muter_ap_ids(activity)
potential_receivers =
potential_receiver_ap_ids
|> Enum.uniq()
|> User.get_users_from_set(local_only)
notification_enabled_users =
Enum.filter(potential_receivers, fn u -> u.ap_id in notification_enabled_ap_ids end)
@ -365,6 +372,38 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
def get_notified_from_activity(_, _local_only), do: {[], []}
@doc "Filters out AP IDs domain-blocking and not following the activity's actor"
def exclude_domain_blocker_ap_ids(ap_ids, activity, preloaded_users \\ [])
def exclude_domain_blocker_ap_ids([], _activity, _preloaded_users), do: []
def exclude_domain_blocker_ap_ids(ap_ids, %Activity{} = activity, preloaded_users) do
activity_actor_domain = activity.actor && URI.parse(activity.actor).host
users =
ap_ids
|> Enum.map(fn ap_id ->
Enum.find(preloaded_users, &(&1.ap_id == ap_id)) ||
User.get_cached_by_ap_id(ap_id)
end)
|> Enum.filter(& &1)
domain_blocker_ap_ids = for u <- users, activity_actor_domain in u.domain_blocks, do: u.ap_id
domain_blocker_follower_ap_ids =
if Enum.any?(domain_blocker_ap_ids) do
activity
|> Activity.user_actor()
|> FollowingRelationship.followers_ap_ids(domain_blocker_ap_ids)
else
[]
end
ap_ids
|> Kernel.--(domain_blocker_ap_ids)
|> Kernel.++(domain_blocker_follower_ap_ids)
end
@doc "Filters out AP IDs of users basing on their relationships with activity actor user"
def exclude_relationship_restricted_ap_ids([], _activity), do: []

1
lib/pleroma/web/api_spec/operations/account_operation.ex
View File

@ -131,6 +131,7 @@ def statuses_operation do
"Include statuses from muted acccounts."
),
Operation.parameter(:exclude_reblogs, :query, BooleanLike, "Exclude reblogs"),
Operation.parameter(:exclude_replies, :query, BooleanLike, "Exclude replies"),
Operation.parameter(
:exclude_visibilities,
:query,

2
lib/pleroma/web/common_api/common_api.ex
View File

@ -7,6 +7,7 @@ defmodule Pleroma.Web.CommonAPI do
alias Pleroma.ActivityExpiration
alias Pleroma.Conversation.Participation
alias Pleroma.FollowingRelationship
alias Pleroma.Notification
alias Pleroma.Object
alias Pleroma.ThreadMute
alias Pleroma.User
@ -61,6 +62,7 @@ def reject_follow_request(follower, followed) do
with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
{:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
{:ok, _relationship} <- FollowingRelationship.update(follower, followed, :follow_reject),
{:ok, _notifications} <- Notification.dismiss(follow_activity),
{:ok, _activity} <-
ActivityPub.reject(%{
to: [follower.ap_id],

675
lib/pleroma/web/pleroma_api/controllers/emoji_api_controller.ex
View File

@ -1,195 +1,93 @@
defmodule Pleroma.Web.PleromaAPI.EmojiAPIController do
use Pleroma.Web, :controller
alias Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug
alias Pleroma.Plugs.OAuthScopesPlug
require Logger
alias Pleroma.Emoji.Pack
plug(
OAuthScopesPlug,
Pleroma.Plugs.OAuthScopesPlug,
%{scopes: ["write"], admin: true}
when action in [
:import_from_filesystem,
:remote,
:download,
:create,
:update,
:delete,
:save_from,
:import_from_fs,
:add_file,
:update_file,
:update_metadata
:delete_file
]
)
plug(
:skip_plug,
[OAuthScopesPlug, ExpectPublicOrAuthenticatedCheckPlug]
when action in [:download_shared, :list_packs, :list_from]
[Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug]
when action in [:archive, :show, :list]
)
defp emoji_dir_path do
Path.join(
Pleroma.Config.get!([:instance, :static_dir]),
"emoji"
)
end
@doc """
Lists packs from the remote instance.
Since JS cannot ask remote instances for their packs due to CPS, it has to
be done by the server
"""
def list_from(conn, %{"instance_address" => address}) do
address = String.trim(address)
if shareable_packs_available(address) do
list_resp =
"#{address}/api/pleroma/emoji/packs" |> Tesla.get!() |> Map.get(:body) |> Jason.decode!()
json(conn, list_resp)
def remote(conn, %{"url" => url}) do
with {:ok, packs} <- Pack.list_remote(url) do
json(conn, packs)
else
{:shareable, _} ->
conn
|> put_status(:internal_server_error)
|> json(%{error: "The requested instance does not support sharing emoji packs"})
end
end
@doc """
Lists the packs available on the instance as JSON.
def list(conn, _params) do
emoji_path =
Path.join(
Pleroma.Config.get!([:instance, :static_dir]),
"emoji"
)
The information is public and does not require authentication. The format is
a map of "pack directory name" to pack.json contents.
"""
def list_packs(conn, _params) do
# Create the directory first if it does not exist. This is probably the first request made
# with the API so it should be sufficient
with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_dir_path())},
{:ls, {:ok, results}} <- {:ls, File.ls(emoji_dir_path())} do
pack_infos =
results
|> Enum.filter(&has_pack_json?/1)
|> Enum.map(&load_pack/1)
# Check if all the files are in place and can be sent
|> Enum.map(&validate_pack/1)
# Transform into a map of pack-name => pack-data
|> Enum.into(%{})
json(conn, pack_infos)
with {:ok, packs} <- Pack.list_local() do
json(conn, packs)
else
{:create_dir, {:error, e}} ->
conn
|> put_status(:internal_server_error)
|> json(%{error: "Failed to create the emoji pack directory at #{emoji_dir_path()}: #{e}"})
|> json(%{error: "Failed to create the emoji pack directory at #{emoji_path}: #{e}"})
{:ls, {:error, e}} ->
conn
|> put_status(:internal_server_error)
|> json(%{
error:
"Failed to get the contents of the emoji pack directory at #{emoji_dir_path()}: #{e}"
error: "Failed to get the contents of the emoji pack directory at #{emoji_path}: #{e}"
})
end
end
defp has_pack_json?(file) do
dir_path = Path.join(emoji_dir_path(), file)
# Filter to only use the pack.json packs
File.dir?(dir_path) and File.exists?(Path.join(dir_path, "pack.json"))
end
def show(conn, %{"name" => name}) do
name = String.trim(name)
defp load_pack(pack_name) do
pack_path = Path.join(emoji_dir_path(), pack_name)
pack_file = Path.join(pack_path, "pack.json")
{pack_name, Jason.decode!(File.read!(pack_file))}
end
defp validate_pack({name, pack}) do
pack_path = Path.join(emoji_dir_path(), name)
if can_download?(pack, pack_path) do
archive_for_sha = make_archive(name, pack, pack_path)
archive_sha = :crypto.hash(:sha256, archive_for_sha) |> Base.encode16()
pack =
pack
|> put_in(["pack", "can-download"], true)
|> put_in(["pack", "download-sha256"], archive_sha)
{name, pack}
with {:ok, pack} <- Pack.show(name) do
json(conn, pack)
else
{name, put_in(pack, ["pack", "can-download"], false)}
{:loaded, _} ->
conn
|> put_status(:not_found)
|> json(%{error: "Pack #{name} does not exist"})
{:error, :empty_values} ->
conn
|> put_status(:bad_request)
|> json(%{error: "pack name cannot be empty"})
end
end
defp can_download?(pack, pack_path) do
# If the pack is set as shared, check if it can be downloaded
# That means that when asked, the pack can be packed and sent to the remote
# Otherwise, they'd have to download it from external-src
pack["pack"]["share-files"] &&
Enum.all?(pack["files"], fn {_, path} ->
File.exists?(Path.join(pack_path, path))
end)
end
defp create_archive_and_cache(name, pack, pack_dir, md5) do
files =
['pack.json'] ++
(pack["files"] |> Enum.map(fn {_, path} -> to_charlist(path) end))
{:ok, {_, zip_result}} = :zip.zip('#{name}.zip', files, [:memory, cwd: to_charlist(pack_dir)])
cache_seconds_per_file = Pleroma.Config.get!([:emoji, :shared_pack_cache_seconds_per_file])
cache_ms = :timer.seconds(cache_seconds_per_file * Enum.count(files))
Cachex.put!(
:emoji_packs_cache,
name,
# if pack.json MD5 changes, the cache is not valid anymore
%{pack_json_md5: md5, pack_data: zip_result},
# Add a minute to cache time for every file in the pack
ttl: cache_ms
)
Logger.debug("Created an archive for the '#{name}' emoji pack, \
keeping it in cache for #{div(cache_ms, 1000)}s")
zip_result
end
defp make_archive(name, pack, pack_dir) do
# Having a different pack.json md5 invalidates cache
pack_file_md5 = :crypto.hash(:md5, File.read!(Path.join(pack_dir, "pack.json")))
case Cachex.get!(:emoji_packs_cache, name) do
%{pack_file_md5: ^pack_file_md5, pack_data: zip_result} ->
Logger.debug("Using cache for the '#{name}' shared emoji pack")
zip_result
_ ->
create_archive_and_cache(name, pack, pack_dir, pack_file_md5)
end
end
@doc """
An endpoint for other instances (via admin UI) or users (via browser)
to download packs that the instance shares.
"""
def download_shared(conn, %{"name" => name}) do
pack_dir = Path.join(emoji_dir_path(), name)
pack_file = Path.join(pack_dir, "pack.json")
with {_, true} <- {:exists?, File.exists?(pack_file)},
pack = Jason.decode!(File.read!(pack_file)),
{_, true} <- {:can_download?, can_download?(pack, pack_dir)} do
zip_result = make_archive(name, pack, pack_dir)
send_download(conn, {:binary, zip_result}, filename: "#{name}.zip")
def archive(conn, %{"name" => name}) do
with {:ok, archive} <- Pack.get_archive(name) do
send_download(conn, {:binary, archive}, filename: "#{name}.zip")
else
{:can_download?, _} ->
conn
|> put_status(:forbidden)
|> json(%{
error: "Pack #{name} cannot be downloaded from this instance, either pack sharing\
was disabled for this pack or some files are missing"
error:
"Pack #{name} cannot be downloaded from this instance, either pack sharing was disabled for this pack or some files are missing"
})
{:exists?, _} ->
@ -199,133 +97,67 @@ def download_shared(conn, %{"name" => name}) do
end
end
defp shareable_packs_available(address) do
"#{address}/.well-known/nodeinfo"
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
|> Map.get("links")
|> List.last()
|> Map.get("href")
# Get the actual nodeinfo address and fetch it
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
|> get_in(["metadata", "features"])
|> Enum.member?("shareable_emoji_packs")
end
@doc """
An admin endpoint to request downloading and storing a pack named `pack_name` from the instance
`instance_address`.
If the requested instance's admin chose to share the pack, it will be downloaded
from that instance, otherwise it will be downloaded from the fallback source, if there is one.
"""
def save_from(conn, %{"instance_address" => address, "pack_name" => name} = data) do
address = String.trim(address)
if shareable_packs_available(address) do
full_pack =
"#{address}/api/pleroma/emoji/packs/list"
|> Tesla.get!()
|> Map.get(:body)
|> Jason.decode!()
|> Map.get(name)
pack_info_res =
case full_pack["pack"] do
%{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
{:ok,
%{
sha: sha,
uri: "#{address}/api/pleroma/emoji/packs/download_shared/#{name}"
}}
%{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
{:ok,
%{
sha: sha,
uri: src,
fallback: true
}}
_ ->
{:error,
"The pack was not set as shared and there is no fallback src to download from"}
end
with {:ok, %{sha: sha, uri: uri} = pinfo} <- pack_info_res,
%{body: emoji_archive} <- Tesla.get!(uri),
{_, true} <- {:checksum, Base.decode16!(sha) == :crypto.hash(:sha256, emoji_archive)} do
local_name = data["as"] || name
pack_dir = Path.join(emoji_dir_path(), local_name)
File.mkdir_p!(pack_dir)
files = Enum.map(full_pack["files"], fn {_, path} -> to_charlist(path) end)
# Fallback cannot contain a pack.json file
files = if pinfo[:fallback], do: files, else: ['pack.json'] ++ files
{:ok, _} = :zip.unzip(emoji_archive, cwd: to_charlist(pack_dir), file_list: files)
# Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
# in it to depend on itself
if pinfo[:fallback] do
pack_file_path = Path.join(pack_dir, "pack.json")
File.write!(pack_file_path, Jason.encode!(full_pack, pretty: true))
end
def download(conn, %{"url" => url, "name" => name} = params) do
with :ok <- Pack.download(name, url, params["as"]) do
json(conn, "ok")
else
{:error, e} ->
conn |> put_status(:internal_server_error) |> json(%{error: e})
{:shareable, _} ->
conn
|> put_status(:internal_server_error)
|> json(%{error: "The requested instance does not support sharing emoji packs"})
{:checksum, _} ->
conn
|> put_status(:internal_server_error)
|> json(%{error: "SHA256 for the pack doesn't match the one sent by the server"})
end
else
{:error, e} ->
conn
|> put_status(:internal_server_error)
|> json(%{error: "The requested instance does not support sharing emoji packs"})
|> json(%{error: e})
end
end
@doc """
Creates an empty pack named `name` which then can be updated via the admin UI.
"""
def create(conn, %{"name" => name}) do
pack_dir = Path.join(emoji_dir_path(), name)
name = String.trim(name)
if not File.exists?(pack_dir) do
File.mkdir_p!(pack_dir)
pack_file_p = Path.join(pack_dir, "pack.json")
File.write!(
pack_file_p,
Jason.encode!(%{pack: %{}, files: %{}}, pretty: true)
)
conn |> json("ok")
with :ok <- Pack.create(name) do
json(conn, "ok")
else
{:error, :eexist} ->
conn
|> put_status(:conflict)
|> json(%{error: "A pack named \"#{name}\" already exists"})
{:error, :empty_values} ->
conn
|> put_status(:bad_request)
|> json(%{error: "pack name cannot be empty"})
{:error, _} ->
render_error(
conn,
:internal_server_error,
"Unexpected error occurred while creating pack."
)
end
end
@doc """
Deletes the pack `name` and all it's files.
"""
def delete(conn, %{"name" => name}) do
pack_dir = Path.join(emoji_dir_path(), name)
name = String.trim(name)
case File.rm_rf(pack_dir) do
{:ok, _} ->
conn |> json("ok")
with {:ok, deleted} when deleted != [] <- Pack.delete(name) do
json(conn, "ok")
else
{:ok, []} ->
conn
|> put_status(:not_found)
|> json(%{error: "Pack #{name} does not exist"})
{:error, :empty_values} ->
conn
|> put_status(:bad_request)
|> json(%{error: "pack name cannot be empty"})
{:error, _, _} ->
conn
@ -334,265 +166,128 @@ def delete(conn, %{"name" => name}) do
end
end
@doc """
An endpoint to update `pack_names`'s metadata.
`new_data` is the new metadata for the pack, that will replace the old metadata.
"""
def update_metadata(conn, %{"pack_name" => name, "new_data" => new_data}) do
pack_file_p = Path.join([emoji_dir_path(), name, "pack.json"])
full_pack = Jason.decode!(File.read!(pack_file_p))
# The new fallback-src is in the new data and it's not the same as it was in the old data
should_update_fb_sha =
not is_nil(new_data["fallback-src"]) and
new_data["fallback-src"] != full_pack["pack"]["fallback-src"]
with {_, true} <- {:should_update?, should_update_fb_sha},
%{body: pack_arch} <- Tesla.get!(new_data["fallback-src"]),
{:ok, flist} <- :zip.unzip(pack_arch, [:memory]),
{_, true} <- {:has_all_files?, has_all_files?(full_pack, flist)} do
fallback_sha = :crypto.hash(:sha256, pack_arch) |> Base.encode16()
new_data = Map.put(new_data, "fallback-src-sha256", fallback_sha)
update_metadata_and_send(conn, full_pack, new_data, pack_file_p)
def update(conn, %{"name" => name, "metadata" => metadata}) do
with {:ok, pack} <- Pack.update_metadata(name, metadata) do
json(conn, pack.pack)
else
{:should_update?, _} ->
update_metadata_and_send(conn, full_pack, new_data, pack_file_p)
{:has_all_files?, _} ->
conn
|> put_status(:bad_request)
|> json(%{error: "The fallback archive does not have all files specified in pack.json"})
end
end
# Check if all files from the pack.json are in the archive
defp has_all_files?(%{"files" => files}, flist) do
Enum.all?(files, fn {_, from_manifest} ->
Enum.find(flist, fn {from_archive, _} ->
to_string(from_archive) == from_manifest
end)
end)
end
defp update_metadata_and_send(conn, full_pack, new_data, pack_file_p) do
full_pack = Map.put(full_pack, "pack", new_data)
File.write!(pack_file_p, Jason.encode!(full_pack, pretty: true))
# Send new data back with fallback sha filled
json(conn, new_data)
end
defp get_filename(%{"filename" => filename}), do: filename
defp get_filename(%{"file" => file}) do
case file do
%Plug.Upload{filename: filename} -> filename
url when is_binary(url) -> Path.basename(url)
end
end
defp empty?(str), do: String.trim(str) == ""
defp update_file_and_send(conn, updated_full_pack, pack_file_p) do
# Write the emoji pack file
File.write!(pack_file_p, Jason.encode!(updated_full_pack, pretty: true))
# Return the modified file list
json(conn, updated_full_pack["files"])
end
@doc """
Updates a file in a pack.
Updating can mean three things:
- `add` adds an emoji named `shortcode` to the pack `pack_name`,
that means that the emoji file needs to be uploaded with the request
(thus requiring it to be a multipart request) and be named `file`.
There can also be an optional `filename` that will be the new emoji file name
(if it's not there, the name will be taken from the uploaded file).
- `update` changes emoji shortcode (from `shortcode` to `new_shortcode` or moves the file
(from the current filename to `new_filename`)
- `remove` removes the emoji named `shortcode` and it's associated file
"""
# Add
def update_file(
{:error, _} ->
render_error(
conn,
%{"pack_name" => pack_name, "action" => "add", "shortcode" => shortcode} = params
) do
pack_dir = Path.join(emoji_dir_path(), pack_name)
pack_file_p = Path.join(pack_dir, "pack.json")
full_pack = Jason.decode!(File.read!(pack_file_p))
with {_, false} <- {:has_shortcode, Map.has_key?(full_pack["files"], shortcode)},
filename <- get_filename(params),
false <- empty?(shortcode),
false <- empty?(filename) do
file_path = Path.join(pack_dir, filename)
# If the name contains directories, create them
if String.contains?(file_path, "/") do
File.mkdir_p!(Path.dirname(file_path))
:internal_server_error,
"Unexpected error occurred while updating pack metadata."
)
end
end
case params["file"] do
%Plug.Upload{path: upload_path} ->
# Copy the uploaded file from the temporary directory
File.copy!(upload_path, file_path)
def add_file(conn, %{"name" => name} = params) do
filename = params["filename"] || get_filename(params["file"])
shortcode = params["shortcode"] || Path.basename(filename, Path.extname(filename))
url when is_binary(url) ->
# Download and write the file
file_contents = Tesla.get!(url).body
File.write!(file_path, file_contents)
end
updated_full_pack = put_in(full_pack, ["files", shortcode], filename)
update_file_and_send(conn, updated_full_pack, pack_file_p)
with {:ok, pack} <- Pack.add_file(name, shortcode, filename, params["file"]) do
json(conn, pack.files)
else
{:has_shortcode, _} ->
{:exists, _} ->
conn
|> put_status(:conflict)
|> json(%{error: "An emoji with the \"#{shortcode}\" shortcode already exists"})
true ->
{:loaded, _} ->
conn
|> put_status(:bad_request)
|> json(%{error: "shortcode or filename cannot be empty"})
end
end
|> json(%{error: "pack \"#{name}\" is not found"})
# Remove
def update_file(conn, %{
"pack_name" => pack_name,
"action" => "remove",
"shortcode" => shortcode
}) do
pack_dir = Path.join(emoji_dir_path(), pack_name)
pack_file_p = Path.join(pack_dir, "pack.json")
full_pack = Jason.decode!(File.read!(pack_file_p))
if Map.has_key?(full_pack["files"], shortcode) do
{emoji_file_path, updated_full_pack} = pop_in(full_pack, ["files", shortcode])
emoji_file_path = Path.join(pack_dir, emoji_file_path)
# Delete the emoji file
File.rm!(emoji_file_path)
# If the old directory has no more files, remove it
if String.contains?(emoji_file_path, "/") do
dir = Path.dirname(emoji_file_path)
if Enum.empty?(File.ls!(dir)) do
File.rmdir!(dir)
end
end
update_file_and_send(conn, updated_full_pack, pack_file_p)
else
{:error, :empty_values} ->
conn
|> put_status(:bad_request)
|> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
end
end
|> json(%{error: "pack name, shortcode or filename cannot be empty"})
# Update
def update_file(
{:error, _} ->
render_error(
conn,
%{"pack_name" => pack_name, "action" => "update", "shortcode" => shortcode} = params
) do
pack_dir = Path.join(emoji_dir_path(), pack_name)
pack_file_p = Path.join(pack_dir, "pack.json")
full_pack = Jason.decode!(File.read!(pack_file_p))
with {_, true} <- {:has_shortcode, Map.has_key?(full_pack["files"], shortcode)},
%{"new_shortcode" => new_shortcode, "new_filename" => new_filename} <- params,
false <- empty?(new_shortcode),
false <- empty?(new_filename) do
# First, remove the old shortcode, saving the old path
{old_emoji_file_path, updated_full_pack} = pop_in(full_pack, ["files", shortcode])
old_emoji_file_path = Path.join(pack_dir, old_emoji_file_path)
new_emoji_file_path = Path.join(pack_dir, new_filename)
# If the name contains directories, create them
if String.contains?(new_emoji_file_path, "/") do
File.mkdir_p!(Path.dirname(new_emoji_file_path))
end
# Move/Rename the old filename to a new filename
# These are probably on the same filesystem, so just rename should work
:ok = File.rename(old_emoji_file_path, new_emoji_file_path)
# If the old directory has no more files, remove it
if String.contains?(old_emoji_file_path, "/") do
dir = Path.dirname(old_emoji_file_path)
if Enum.empty?(File.ls!(dir)) do
File.rmdir!(dir)
:internal_server_error,
"Unexpected error occurred while adding file to pack."
)
end
end
# Then, put in the new shortcode with the new path
updated_full_pack = put_in(updated_full_pack, ["files", new_shortcode], new_filename)
update_file_and_send(conn, updated_full_pack, pack_file_p)
def update_file(conn, %{"name" => name, "shortcode" => shortcode} = params) do
new_shortcode = params["new_shortcode"]
new_filename = params["new_filename"]
force = params["force"] == true
with {:ok, pack} <- Pack.update_file(name, shortcode, new_shortcode, new_filename, force) do
json(conn, pack.files)
else
{:has_shortcode, _} ->
{:exists, _} ->
conn
|> put_status(:bad_request)
|> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
true ->
{:not_used, _} ->
conn
|> put_status(:conflict)
|> json(%{
error:
"New shortcode \"#{new_shortcode}\" is already used. If you want to override emoji use 'force' option"
})
{:loaded, _} ->
conn
|> put_status(:bad_request)
|> json(%{error: "pack \"#{name}\" is not found"})
{:error, :empty_values} ->
conn
|> put_status(:bad_request)
|> json(%{error: "new_shortcode or new_filename cannot be empty"})
_ ->
conn
|> put_status(:bad_request)
|> json(%{error: "new_shortcode or new_file were not specified"})
{:error, _} ->
render_error(
conn,
:internal_server_error,
"Unexpected error occurred while updating file in pack."
)
end
end
def update_file(conn, %{"action" => action}) do
conn
|> put_status(:bad_request)
|> json(%{error: "Unknown action: #{action}"})
end
@doc """
Imports emoji from the filesystem.
Importing means checking all the directories in the
`$instance_static/emoji/` for directories which do not have
`pack.json`. If one has an emoji.txt file, that file will be used
to create a `pack.json` file with it's contents. If the directory has
neither, all the files with specific configured extenstions will be
assumed to be emojis and stored in the new `pack.json` file.
"""
def import_from_fs(conn, _params) do
emoji_path = emoji_dir_path()
with {:ok, %{access: :read_write}} <- File.stat(emoji_path),
{:ok, results} <- File.ls(emoji_path) do
imported_pack_names =
results
|> Enum.filter(fn file ->
dir_path = Path.join(emoji_path, file)
# Find the directories that do NOT have pack.json
File.dir?(dir_path) and not File.exists?(Path.join(dir_path, "pack.json"))
end)
|> Enum.map(&write_pack_json_contents/1)
json(conn, imported_pack_names)
def delete_file(conn, %{"name" => name, "shortcode" => shortcode}) do
with {:ok, pack} <- Pack.delete_file(name, shortcode) do
json(conn, pack.files)
else
{:ok, %{access: _}} ->
{:exists, _} ->
conn
|> put_status(:bad_request)
|> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
{:loaded, _} ->
conn
|> put_status(:bad_request)
|> json(%{error: "pack \"#{name}\" is not found"})
{:error, :empty_values} ->
conn
|> put_status(:bad_request)
|> json(%{error: "pack name or shortcode cannot be empty"})
{:error, _} ->
render_error(
conn,
:internal_server_error,
"Unexpected error occurred while removing file from pack."
)
end
end
def import_from_filesystem(conn, _params) do
with {:ok, names} <- Pack.import_from_filesystem() do
json(conn, names)
else
{:error, :no_read_write} ->
conn
|> put_status(:internal_server_error)
|> json(%{error: "Error: emoji pack directory must be writable"})
@ -604,44 +299,6 @@ def import_from_fs(conn, _params) do
end
end
defp write_pack_json_contents(dir) do
dir_path = Path.join(emoji_dir_path(), dir)
emoji_txt_path = Path.join(dir_path, "emoji.txt")
files_for_pack = files_for_pack(emoji_txt_path, dir_path)
pack_json_contents = Jason.encode!(%{pack: %{}, files: files_for_pack})
File.write!(Path.join(dir_path, "pack.json"), pack_json_contents)
dir
end
defp files_for_pack(emoji_txt_path, dir_path) do
if File.exists?(emoji_txt_path) do
# There's an emoji.txt file, it's likely from a pack installed by the pack manager.
# Make a pack.json file from the contents of that emoji.txt fileh
# FIXME: Copy-pasted from Pleroma.Emoji/load_from_file_stream/2
# Create a map of shortcodes to filenames from emoji.txt
File.read!(emoji_txt_path)
|> String.split("\n")
|> Enum.map(&String.trim/1)
|> Enum.map(fn line ->
case String.split(line, ~r/,\s*/) do
# This matches both strings with and without tags
# and we don't care about tags here
[name, file | _] -> {name, file}
_ -> nil
end
end)
|> Enum.filter(fn x -> not is_nil(x) end)
|> Enum.into(%{})
else
# If there's no emoji.txt, assume all files
# that are of certain extensions from the config are emojis and import them all
pack_extensions = Pleroma.Config.get!([:emoji, :pack_extensions])
Pleroma.Emoji.Loader.make_shortcode_to_file_map(dir_path, pack_extensions)
end
end
defp get_filename(%Plug.Upload{filename: filename}), do: filename
defp get_filename(url) when is_binary(url), do: Path.basename(url)
end

24
lib/pleroma/web/router.ex
View File

@ -214,24 +214,24 @@ defmodule Pleroma.Web.Router do
scope "/packs" do
pipe_through(:admin_api)
post("/import_from_fs", EmojiAPIController, :import_from_fs)
post("/:pack_name/update_file", EmojiAPIController, :update_file)
post("/:pack_name/update_metadata", EmojiAPIController, :update_metadata)
put("/:name", EmojiAPIController, :create)
get("/import", EmojiAPIController, :import_from_filesystem)
get("/remote", EmojiAPIController, :remote)
post("/download", EmojiAPIController, :download)
post("/:name", EmojiAPIController, :create)
patch("/:name", EmojiAPIController, :update)
delete("/:name", EmojiAPIController, :delete)
# Note: /download_from downloads and saves to instance, not to requester
post("/download_from", EmojiAPIController, :save_from)
post("/:name/files", EmojiAPIController, :add_file)
patch("/:name/files", EmojiAPIController, :update_file)
delete("/:name/files", EmojiAPIController, :delete_file)
end
# Pack info / downloading
scope "/packs" do
get("/", EmojiAPIController, :list_packs)
get("/:name/download_shared/", EmojiAPIController, :download_shared)
get("/list_from", EmojiAPIController, :list_from)
# Deprecated: POST /api/pleroma/emoji/packs/list_from (use GET instead)
post("/list_from", EmojiAPIController, :list_from)
get("/", EmojiAPIController, :list)
get("/:name", EmojiAPIController, :show)
get("/:name/archive", EmojiAPIController, :archive)
end
end

BIN
test/instance_static/add/shortcode.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 95 B

BIN
test/instance_static/emoji/pack_bad_sha/blank.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 95 B

13
test/instance_static/emoji/pack_bad_sha/pack.json Normal file
View File

@ -0,0 +1,13 @@
{
"pack": {
"license": "Test license",
"homepage": "https://pleroma.social",
"description": "Test description",
"can-download": true,
"share-files": true,
"download-sha256": "57482F30674FD3DE821FF48C81C00DA4D4AF1F300209253684ABA7075E5FC238"
},
"files": {
"blank": "blank.png"
}
}

BIN
test/instance_static/emoji/pack_bad_sha/pack_bad_sha.zip Normal file
View File

Binary file not shown.

14
test/instance_static/emoji/test_pack/pack.json
View File

@ -1,13 +1,11 @@
{
"pack": {
"license": "Test license",
"homepage": "https://pleroma.social",
"description": "Test description",
"share-files": true
},
"files": {
"blank": "blank.png"
},
"pack": {
"description": "Test description",
"homepage": "https://pleroma.social",
"license": "Test license",
"share-files": true
}
}

3
test/instance_static/emoji/test_pack_nonshared/pack.json
View File

@ -3,13 +3,10 @@
"license": "Test license",
"homepage": "https://pleroma.social",
"description": "Test description",
"fallback-src": "https://nonshared-pack",
"fallback-src-sha256": "74409E2674DAA06C072729C6C8426C4CB3B7E0B85ED77792DB7A436E11D76DAF",
"share-files": false
},
"files": {
"blank": "blank.png"
}

58
test/notification_test.exs
View File

@ -362,6 +362,16 @@ test "it doesn't create a notification for follow-unfollow-follow chains" do
notification_id = notification.id
assert [%{id: ^notification_id}] = Notification.for_user(followed_user)
end
test "dismisses the notification on follow request rejection" do
clear_config([:notifications, :enable_follow_request_notifications], true)
user = insert(:user, locked: true)
follower = insert(:user)
{:ok, _, _, _follow_activity} = CommonAPI.follow(follower, user)
assert [notification] = Notification.for_user(user)
{:ok, _follower} = CommonAPI.reject_follow_request(follower, user)
assert [] = Notification.for_user(user)
end
end
describe "get notification" do
@ -669,6 +679,37 @@ test "it returns thread-muting recipient in disabled recipients list" do
assert [other_user] == disabled_receivers
refute other_user in enabled_receivers
end
test "it returns non-following domain-blocking recipient in disabled recipients list" do
blocked_domain = "blocked.domain"
user = insert(:user, %{ap_id: "https://#{blocked_domain}/@actor"})
other_user = insert(:user)
{:ok, other_user} = User.block_domain(other_user, blocked_domain)
{:ok, activity} = CommonAPI.post(user, %{"status" => "hey @#{other_user.nickname}!"})
{enabled_receivers, disabled_receivers} = Notification.get_notified_from_activity(activity)
assert [] == enabled_receivers
assert [other_user] == disabled_receivers
end
test "it returns following domain-blocking recipient in enabled recipients list" do
blocked_domain = "blocked.domain"
user = insert(:user, %{ap_id: "https://#{blocked_domain}/@actor"})
other_user = insert(:user)
{:ok, other_user} = User.block_domain(other_user, blocked_domain)
{:ok, other_user} = User.follow(other_user, user)
{:ok, activity} = CommonAPI.post(user, %{"status" => "hey @#{other_user.nickname}!"})
{enabled_receivers, disabled_receivers} = Notification.get_notified_from_activity(activity)
assert [other_user] == enabled_receivers
assert [] == disabled_receivers
end
end
describe "notification lifecycle" do
@ -931,7 +972,7 @@ test "it doesn't return notifications for blocked user" do
assert Notification.for_user(user) == []
end
test "it doesn't return notifications for blocked domain" do
test "it doesn't return notifications for domain-blocked non-followed user" do
user = insert(:user)
blocked = insert(:user, ap_id: "http://some-domain.com")
{:ok, user} = User.block_domain(user, "some-domain.com")
@ -941,6 +982,18 @@ test "it doesn't return notifications for blocked domain" do
assert Notification.for_user(user) == []
end
test "it returns notifications for domain-blocked but followed user" do
user = insert(:user)
blocked = insert(:user, ap_id: "http://some-domain.com")
{:ok, user} = User.block_domain(user, "some-domain.com")
{:ok, _} = User.follow(user, blocked)
{:ok, _activity} = CommonAPI.post(blocked, %{"status" => "hey @#{user.nickname}"})
assert length(Notification.for_user(user)) == 1
end
test "it doesn't return notifications for muted thread" do
user = insert(:user)
another_user = insert(:user)
@ -971,7 +1024,8 @@ test "it doesn't return notifications from a blocked user when with_muted is set
assert Enum.empty?(Notification.for_user(user, %{with_muted: true}))
end
test "it doesn't return notifications from a domain-blocked user when with_muted is set" do
test "when with_muted is set, " <>
"it doesn't return notifications from a domain-blocked non-followed user" do
user = insert(:user)
blocked = insert(:user, ap_id: "http://some-domain.com")
{:ok, user} = User.block_domain(user, "some-domain.com")

754
test/web/pleroma_api/controllers/emoji_api_controller_test.exs
View File

@ -8,41 +8,44 @@ defmodule Pleroma.Web.PleromaAPI.EmojiAPIControllerTest do
import Tesla.Mock
import Pleroma.Factory
@emoji_dir_path Path.join(
@emoji_path Path.join(
Pleroma.Config.get!([:instance, :static_dir]),
"emoji"
)
setup do: clear_config([:auth, :enforce_oauth_admin_scope_usage], false)
test "shared & non-shared pack information in list_packs is ok" do
conn = build_conn()
resp = conn |> get(emoji_api_path(conn, :list_packs)) |> json_response(200)
setup do
admin = insert(:user, is_admin: true)
token = insert(:oauth_admin_token, user: admin)
assert Map.has_key?(resp, "test_pack")
admin_conn =
build_conn()
|> assign(:user, admin)
|> assign(:token, token)
pack = resp["test_pack"]
assert Map.has_key?(pack["pack"], "download-sha256")
assert pack["pack"]["can-download"]
assert pack["files"] == %{"blank" => "blank.png"}
# Non-shared pack
assert Map.has_key?(resp, "test_pack_nonshared")
pack = resp["test_pack_nonshared"]
refute pack["pack"]["shared"]
refute pack["pack"]["can-download"]
Pleroma.Emoji.reload()
{:ok, %{admin_conn: admin_conn}}
end
test "listing remote packs" do
conn = build_conn()
test "GET /api/pleroma/emoji/packs", %{conn: conn} do
resp = conn |> get("/api/pleroma/emoji/packs") |> json_response(200)
shared = resp["test_pack"]
assert shared["files"] == %{"blank" => "blank.png"}
assert Map.has_key?(shared["pack"], "download-sha256")
assert shared["pack"]["can-download"]
assert shared["pack"]["share-files"]
non_shared = resp["test_pack_nonshared"]
assert non_shared["pack"]["share-files"] == false
assert non_shared["pack"]["can-download"] == false
end
describe "GET /api/pleroma/emoji/packs/remote" do
test "shareable instance", %{admin_conn: admin_conn, conn: conn} do
resp =
build_conn()
|> get(emoji_api_path(conn, :list_packs))
conn
|> get("/api/pleroma/emoji/packs")
|> json_response(200)
mock(fn
@ -56,17 +59,35 @@ test "listing remote packs" do
json(resp)
end)
assert conn
|> post(emoji_api_path(conn, :list_from), %{instance_address: "https://example.com"})
assert admin_conn
|> get("/api/pleroma/emoji/packs/remote", %{
url: "https://example.com"
})
|> json_response(200) == resp
end
test "downloading a shared pack from download_shared" do
conn = build_conn()
test "non shareable instance", %{admin_conn: admin_conn} do
mock(fn
%{method: :get, url: "https://example.com/.well-known/nodeinfo"} ->
json(%{links: [%{href: "https://example.com/nodeinfo/2.1.json"}]})
%{method: :get, url: "https://example.com/nodeinfo/2.1.json"} ->
json(%{metadata: %{features: []}})
end)
assert admin_conn
|> get("/api/pleroma/emoji/packs/remote", %{url: "https://example.com"})
|> json_response(500) == %{
"error" => "The requested instance does not support sharing emoji packs"
}
end
end
describe "GET /api/pleroma/emoji/packs/:name/archive" do
test "download shared pack", %{conn: conn} do
resp =
conn
|> get(emoji_api_path(conn, :download_shared, "test_pack"))
|> get("/api/pleroma/emoji/packs/test_pack/archive")
|> response(200)
{:ok, arch} = :zip.unzip(resp, [:memory])
@ -75,19 +96,30 @@ test "downloading a shared pack from download_shared" do
assert Enum.find(arch, fn {n, _} -> n == 'blank.png' end)
end
test "downloading shared & unshared packs from another instance, deleting them" do
on_exit(fn ->
File.rm_rf!("#{@emoji_dir_path}/test_pack2")
File.rm_rf!("#{@emoji_dir_path}/test_pack_nonshared2")
end)
test "non existing pack", %{conn: conn} do
assert conn
|> get("/api/pleroma/emoji/packs/test_pack_for_import/archive")
|> json_response(:not_found) == %{
"error" => "Pack test_pack_for_import does not exist"
}
end
test "non downloadable pack", %{conn: conn} do
assert conn
|> get("/api/pleroma/emoji/packs/test_pack_nonshared/archive")
|> json_response(:forbidden) == %{
"error" =>
"Pack test_pack_nonshared cannot be downloaded from this instance, either pack sharing was disabled for this pack or some files are missing"
}
end
end
describe "POST /api/pleroma/emoji/packs/download" do
test "shared pack from remote and non shared from fallback-src", %{
admin_conn: admin_conn,
conn: conn
} do
mock(fn
%{method: :get, url: "https://old-instance/.well-known/nodeinfo"} ->
json(%{links: [%{href: "https://old-instance/nodeinfo/2.1.json"}]})
%{method: :get, url: "https://old-instance/nodeinfo/2.1.json"} ->
json(%{metadata: %{features: []}})
%{method: :get, url: "https://example.com/.well-known/nodeinfo"} ->
json(%{links: [%{href: "https://example.com/nodeinfo/2.1.json"}]})
@ -96,124 +128,178 @@ test "downloading shared & unshared packs from another instance, deleting them"
%{
method: :get,
url: "https://example.com/api/pleroma/emoji/packs/list"
url: "https://example.com/api/pleroma/emoji/packs/test_pack"
} ->
conn = build_conn()
conn
|> get(emoji_api_path(conn, :list_packs))
|> get("/api/pleroma/emoji/packs/test_pack")
|> json_response(200)
|> json()
%{
method: :get,
url: "https://example.com/api/pleroma/emoji/packs/download_shared/test_pack"
url: "https://example.com/api/pleroma/emoji/packs/test_pack/archive"
} ->
conn = build_conn()
conn
|> get(emoji_api_path(conn, :download_shared, "test_pack"))
|> get("/api/pleroma/emoji/packs/test_pack/archive")
|> response(200)
|> text()
%{
method: :get,
url: "https://example.com/api/pleroma/emoji/packs/test_pack_nonshared"
} ->
conn
|> get("/api/pleroma/emoji/packs/test_pack_nonshared")
|> json_response(200)
|> json()
%{
method: :get,
url: "https://nonshared-pack"
} ->
text(File.read!("#{@emoji_dir_path}/test_pack_nonshared/nonshared.zip"))
text(File.read!("#{@emoji_path}/test_pack_nonshared/nonshared.zip"))
end)
admin = insert(:user, is_admin: true)
conn =
build_conn()
|> assign(:user, admin)
|> assign(:token, insert(:oauth_admin_token, user: admin, scopes: ["admin:write"]))
assert (conn
|> put_req_header("content-type", "application/json")
|> post(
emoji_api_path(
conn,
:save_from
),
%{
instance_address: "https://old-instance",
pack_name: "test_pack",
assert admin_conn
|> post("/api/pleroma/emoji/packs/download", %{
url: "https://example.com",
name: "test_pack",
as: "test_pack2"
}
|> Jason.encode!()
)
|> json_response(500))["error"] =~ "does not support"
assert conn
|> put_req_header("content-type", "application/json")
|> post(
emoji_api_path(
conn,
:save_from
),
%{
instance_address: "https://example.com",
pack_name: "test_pack",
as: "test_pack2"
}
|> Jason.encode!()
)
})
|> json_response(200) == "ok"
assert File.exists?("#{@emoji_dir_path}/test_pack2/pack.json")
assert File.exists?("#{@emoji_dir_path}/test_pack2/blank.png")
assert File.exists?("#{@emoji_path}/test_pack2/pack.json")
assert File.exists?("#{@emoji_path}/test_pack2/blank.png")
assert conn
|> delete(emoji_api_path(conn, :delete, "test_pack2"))
assert admin_conn
|> delete("/api/pleroma/emoji/packs/test_pack2")
|> json_response(200) == "ok"
refute File.exists?("#{@emoji_dir_path}/test_pack2")
refute File.exists?("#{@emoji_path}/test_pack2")
# non-shared, downloaded from the fallback URL
assert conn
|> put_req_header("content-type", "application/json")
assert admin_conn
|> post(
emoji_api_path(
conn,
:save_from
),
"/api/pleroma/emoji/packs/download",
%{
instance_address: "https://example.com",
pack_name: "test_pack_nonshared",
url: "https://example.com",
name: "test_pack_nonshared",
as: "test_pack_nonshared2"
}
|> Jason.encode!()
)
|> json_response(200) == "ok"
assert File.exists?("#{@emoji_dir_path}/test_pack_nonshared2/pack.json")
assert File.exists?("#{@emoji_dir_path}/test_pack_nonshared2/blank.png")
assert File.exists?("#{@emoji_path}/test_pack_nonshared2/pack.json")
assert File.exists?("#{@emoji_path}/test_pack_nonshared2/blank.png")
assert conn
|> delete(emoji_api_path(conn, :delete, "test_pack_nonshared2"))
assert admin_conn
|> delete("/api/pleroma/emoji/packs/test_pack_nonshared2")
|> json_response(200) == "ok"
refute File.exists?("#{@emoji_dir_path}/test_pack_nonshared2")
refute File.exists?("#{@emoji_path}/test_pack_nonshared2")
end
describe "updating pack metadata" do
test "nonshareable instance", %{admin_conn: admin_conn} do
mock(fn
%{method: :get, url: "https://old-instance/.well-known/nodeinfo"} ->
json(%{links: [%{href: "https://old-instance/nodeinfo/2.1.json"}]})
%{method: :get, url: "https://old-instance/nodeinfo/2.1.json"} ->
json(%{metadata: %{features: []}})
end)
assert admin_conn
|> post(
"/api/pleroma/emoji/packs/download",
%{
url: "https://old-instance",
name: "test_pack",
as: "test_pack2"
}
)
|> json_response(500) == %{
"error" => "The requested instance does not support sharing emoji packs"
}
end
test "checksum fail", %{admin_conn: admin_conn} do
mock(fn
%{method: :get, url: "https://example.com/.well-known/nodeinfo"} ->
json(%{links: [%{href: "https://example.com/nodeinfo/2.1.json"}]})
%{method: :get, url: "https://example.com/nodeinfo/2.1.json"} ->
json(%{metadata: %{features: ["shareable_emoji_packs"]}})
%{
method: :get,
url: "https://example.com/api/pleroma/emoji/packs/pack_bad_sha"
} ->
%Tesla.Env{
status: 200,
body: Pleroma.Emoji.Pack.load_pack("pack_bad_sha") |> Jason.encode!()
}
%{
method: :get,
url: "https://example.com/api/pleroma/emoji/packs/pack_bad_sha/archive"
} ->
%Tesla.Env{
status: 200,
body: File.read!("test/instance_static/emoji/pack_bad_sha/pack_bad_sha.zip")
}
end)
assert admin_conn
|> post("/api/pleroma/emoji/packs/download", %{
url: "https://example.com",
name: "pack_bad_sha",
as: "pack_bad_sha2"
})
|> json_response(:internal_server_error) == %{
"error" => "SHA256 for the pack doesn't match the one sent by the server"
}
end
test "other error", %{admin_conn: admin_conn} do
mock(fn
%{method: :get, url: "https://example.com/.well-known/nodeinfo"} ->
json(%{links: [%{href: "https://example.com/nodeinfo/2.1.json"}]})
%{method: :get, url: "https://example.com/nodeinfo/2.1.json"} ->
json(%{metadata: %{features: ["shareable_emoji_packs"]}})
%{
method: :get,
url: "https://example.com/api/pleroma/emoji/packs/test_pack"
} ->
%Tesla.Env{
status: 200,
body: Pleroma.Emoji.Pack.load_pack("test_pack") |> Jason.encode!()
}
end)
assert admin_conn
|> post("/api/pleroma/emoji/packs/download", %{
url: "https://example.com",
name: "test_pack",
as: "test_pack2"
})
|> json_response(:internal_server_error) == %{
"error" =>
"The pack was not set as shared and there is no fallback src to download from"
}
end
end
describe "PATCH /api/pleroma/emoji/packs/:name" do
setup do
pack_file = "#{@emoji_dir_path}/test_pack/pack.json"
pack_file = "#{@emoji_path}/test_pack/pack.json"
original_content = File.read!(pack_file)
on_exit(fn ->
File.write!(pack_file, original_content)
end)
admin = insert(:user, is_admin: true)
%{conn: conn} = oauth_access(["admin:write"], user: admin)
{:ok,
admin: admin,
conn: conn,
pack_file: pack_file,
new_data: %{
"license" => "Test license changed",
@ -224,15 +310,8 @@ test "downloading shared & unshared packs from another instance, deleting them"
end
test "for a pack without a fallback source", ctx do
conn = ctx[:conn]
assert conn
|> post(
emoji_api_path(conn, :update_metadata, "test_pack"),
%{
"new_data" => ctx[:new_data]
}
)
assert ctx[:admin_conn]
|> patch("/api/pleroma/emoji/packs/test_pack", %{"metadata" => ctx[:new_data]})
|> json_response(200) == ctx[:new_data]
assert Jason.decode!(File.read!(ctx[:pack_file]))["pack"] == ctx[:new_data]
@ -244,7 +323,7 @@ test "for a pack with a fallback source", ctx do
method: :get,
url: "https://nonshared-pack"
} ->
text(File.read!("#{@emoji_dir_path}/test_pack_nonshared/nonshared.zip"))
text(File.read!("#{@emoji_path}/test_pack_nonshared/nonshared.zip"))
end)
new_data = Map.put(ctx[:new_data], "fallback-src", "https://nonshared-pack")
@ -256,15 +335,8 @@ test "for a pack with a fallback source", ctx do
"74409E2674DAA06C072729C6C8426C4CB3B7E0B85ED77792DB7A436E11D76DAF"
)
conn = ctx[:conn]
assert conn
|> post(
emoji_api_path(conn, :update_metadata, "test_pack"),
%{
"new_data" => new_data
}
)
assert ctx[:admin_conn]
|> patch("/api/pleroma/emoji/packs/test_pack", %{metadata: new_data})
|> json_response(200) == new_data_with_sha
assert Jason.decode!(File.read!(ctx[:pack_file]))["pack"] == new_data_with_sha
@ -282,181 +354,377 @@ test "when the fallback source doesn't have all the files", ctx do
new_data = Map.put(ctx[:new_data], "fallback-src", "https://nonshared-pack")
conn = ctx[:conn]
assert (conn
|> post(
emoji_api_path(conn, :update_metadata, "test_pack"),
%{
"new_data" => new_data
assert ctx[:admin_conn]
|> patch("/api/pleroma/emoji/packs/test_pack", %{metadata: new_data})
|> json_response(:bad_request) == %{
"error" => "The fallback archive does not have all files specified in pack.json"
}
)
|> json_response(:bad_request))["error"] =~ "does not have all"
end
end
test "updating pack files" do
pack_file = "#{@emoji_dir_path}/test_pack/pack.json"
describe "POST/PATCH/DELETE /api/pleroma/emoji/packs/:name/files" do
setup do
pack_file = "#{@emoji_path}/test_pack/pack.json"
original_content = File.read!(pack_file)
on_exit(fn ->
File.write!(pack_file, original_content)
File.rm_rf!("#{@emoji_dir_path}/test_pack/blank_url.png")
File.rm_rf!("#{@emoji_dir_path}/test_pack/dir")
File.rm_rf!("#{@emoji_dir_path}/test_pack/dir_2")
end)
admin = insert(:user, is_admin: true)
%{conn: conn} = oauth_access(["admin:write"], user: admin)
:ok
end
same_name = %{
"action" => "add",
"shortcode" => "blank",
"filename" => "dir/blank.png",
"file" => %Plug.Upload{
test "create shortcode exists", %{admin_conn: admin_conn} do
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank",
filename: "dir/blank.png",
file: %Plug.Upload{
filename: "blank.png",
path: "#{@emoji_dir_path}/test_pack/blank.png"
path: "#{@emoji_path}/test_pack/blank.png"
}
})
|> json_response(:conflict) == %{
"error" => "An emoji with the \"blank\" shortcode already exists"
}
end
test "don't rewrite old emoji", %{admin_conn: admin_conn} do
on_exit(fn -> File.rm_rf!("#{@emoji_path}/test_pack/dir/") end)
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank2",
filename: "dir/blank.png",
file: %Plug.Upload{
filename: "blank.png",
path: "#{@emoji_path}/test_pack/blank.png"
}
})
|> json_response(200) == %{"blank" => "blank.png", "blank2" => "dir/blank.png"}
assert File.exists?("#{@emoji_path}/test_pack/dir/blank.png")
assert admin_conn
|> patch("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank",
new_shortcode: "blank2",
new_filename: "dir_2/blank_3.png"
})
|> json_response(:conflict) == %{
"error" =>
"New shortcode \"blank2\" is already used. If you want to override emoji use 'force' option"
}
end
test "rewrite old emoji with force option", %{admin_conn: admin_conn} do
on_exit(fn -> File.rm_rf!("#{@emoji_path}/test_pack/dir_2/") end)
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank2",
filename: "dir/blank.png",
file: %Plug.Upload{
filename: "blank.png",
path: "#{@emoji_path}/test_pack/blank.png"
}
})
|> json_response(200) == %{"blank" => "blank.png", "blank2" => "dir/blank.png"}
assert File.exists?("#{@emoji_path}/test_pack/dir/blank.png")
assert admin_conn
|> patch("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank2",
new_shortcode: "blank3",
new_filename: "dir_2/blank_3.png",
force: true
})
|> json_response(200) == %{
"blank" => "blank.png",
"blank3" => "dir_2/blank_3.png"
}
different_name = %{same_name | "shortcode" => "blank_2"}
assert File.exists?("#{@emoji_path}/test_pack/dir_2/blank_3.png")
end
assert (conn
|> post(emoji_api_path(conn, :update_file, "test_pack"), same_name)
|> json_response(:conflict))["error"] =~ "already exists"
assert conn
|> post(emoji_api_path(conn, :update_file, "test_pack"), different_name)
|> json_response(200) == %{"blank" => "blank.png", "blank_2" => "dir/blank.png"}
assert File.exists?("#{@emoji_dir_path}/test_pack/dir/blank.png")
assert conn
|> post(emoji_api_path(conn, :update_file, "test_pack"), %{
"action" => "update",
"shortcode" => "blank_2",
"new_shortcode" => "blank_3",
"new_filename" => "dir_2/blank_3.png"
test "with empty filename", %{admin_conn: admin_conn} do
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank2",
filename: "",
file: %Plug.Upload{
filename: "blank.png",
path: "#{@emoji_path}/test_pack/blank.png"
}
})
|> json_response(200) == %{"blank" => "blank.png", "blank_3" => "dir_2/blank_3.png"}
|> json_response(:bad_request) == %{
"error" => "pack name, shortcode or filename cannot be empty"
}
end
refute File.exists?("#{@emoji_dir_path}/test_pack/dir/")
assert File.exists?("#{@emoji_dir_path}/test_pack/dir_2/blank_3.png")
assert conn
|> post(emoji_api_path(conn, :update_file, "test_pack"), %{
"action" => "remove",
"shortcode" => "blank_3"
test "add file with not loaded pack", %{admin_conn: admin_conn} do
assert admin_conn
|> post("/api/pleroma/emoji/packs/not_loaded/files", %{
shortcode: "blank2",
filename: "dir/blank.png",
file: %Plug.Upload{
filename: "blank.png",
path: "#{@emoji_path}/test_pack/blank.png"
}
})
|> json_response(:bad_request) == %{
"error" => "pack \"not_loaded\" is not found"
}
end
test "remove file with not loaded pack", %{admin_conn: admin_conn} do
assert admin_conn
|> delete("/api/pleroma/emoji/packs/not_loaded/files", %{shortcode: "blank3"})
|> json_response(:bad_request) == %{"error" => "pack \"not_loaded\" is not found"}
end
test "remove file with empty shortcode", %{admin_conn: admin_conn} do
assert admin_conn
|> delete("/api/pleroma/emoji/packs/not_loaded/files", %{shortcode: ""})
|> json_response(:bad_request) == %{
"error" => "pack name or shortcode cannot be empty"
}
end
test "update file with not loaded pack", %{admin_conn: admin_conn} do
assert admin_conn
|> patch("/api/pleroma/emoji/packs/not_loaded/files", %{
shortcode: "blank4",
new_shortcode: "blank3",
new_filename: "dir_2/blank_3.png"
})
|> json_response(:bad_request) == %{"error" => "pack \"not_loaded\" is not found"}
end
test "new with shortcode as file with update", %{admin_conn: admin_conn} do
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank4",
filename: "dir/blank.png",
file: %Plug.Upload{
filename: "blank.png",
path: "#{@emoji_path}/test_pack/blank.png"
}
})
|> json_response(200) == %{"blank" => "blank.png", "blank4" => "dir/blank.png"}
assert File.exists?("#{@emoji_path}/test_pack/dir/blank.png")
assert admin_conn
|> patch("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank4",
new_shortcode: "blank3",
new_filename: "dir_2/blank_3.png"
})
|> json_response(200) == %{"blank3" => "dir_2/blank_3.png", "blank" => "blank.png"}
refute File.exists?("#{@emoji_path}/test_pack/dir/")
assert File.exists?("#{@emoji_path}/test_pack/dir_2/blank_3.png")
assert admin_conn
|> delete("/api/pleroma/emoji/packs/test_pack/files", %{shortcode: "blank3"})
|> json_response(200) == %{"blank" => "blank.png"}
refute File.exists?("#{@emoji_dir_path}/test_pack/dir_2/")
refute File.exists?("#{@emoji_path}/test_pack/dir_2/")
on_exit(fn -> File.rm_rf!("#{@emoji_path}/test_pack/dir") end)
end
test "new with shortcode from url", %{admin_conn: admin_conn} do
mock(fn
%{
method: :get,
url: "https://test-blank/blank_url.png"
} ->
text(File.read!("#{@emoji_dir_path}/test_pack/blank.png"))
text(File.read!("#{@emoji_path}/test_pack/blank.png"))
end)
# The name should be inferred from the URL ending
from_url = %{
"action" => "add",
"shortcode" => "blank_url",
"file" => "https://test-blank/blank_url.png"
}
assert conn
|> post(emoji_api_path(conn, :update_file, "test_pack"), from_url)
|> json_response(200) == %{
"blank" => "blank.png",
"blank_url" => "blank_url.png"
}
assert File.exists?("#{@emoji_dir_path}/test_pack/blank_url.png")
assert conn
|> post(emoji_api_path(conn, :update_file, "test_pack"), %{
"action" => "remove",
"shortcode" => "blank_url"
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank_url",
file: "https://test-blank/blank_url.png"
})
|> json_response(200) == %{"blank" => "blank.png"}
|> json_response(200) == %{
"blank_url" => "blank_url.png",
"blank" => "blank.png"
}
refute File.exists?("#{@emoji_dir_path}/test_pack/blank_url.png")
assert File.exists?("#{@emoji_path}/test_pack/blank_url.png")
on_exit(fn -> File.rm_rf!("#{@emoji_path}/test_pack/blank_url.png") end)
end
test "creating and deleting a pack" do
on_exit(fn ->
File.rm_rf!("#{@emoji_dir_path}/test_created")
end)
test "new without shortcode", %{admin_conn: admin_conn} do
on_exit(fn -> File.rm_rf!("#{@emoji_path}/test_pack/shortcode.png") end)
admin = insert(:user, is_admin: true)
%{conn: conn} = oauth_access(["admin:write"], user: admin)
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_pack/files", %{
file: %Plug.Upload{
filename: "shortcode.png",
path: "#{Pleroma.Config.get([:instance, :static_dir])}/add/shortcode.png"
}
})
|> json_response(200) == %{"shortcode" => "shortcode.png", "blank" => "blank.png"}
end
assert conn
|> put_req_header("content-type", "application/json")
|> put(
emoji_api_path(
conn,
:create,
"test_created"
)
)
test "remove non existing shortcode in pack.json", %{admin_conn: admin_conn} do
assert admin_conn
|> delete("/api/pleroma/emoji/packs/test_pack/files", %{shortcode: "blank2"})
|> json_response(:bad_request) == %{"error" => "Emoji \"blank2\" does not exist"}
end
test "update non existing emoji", %{admin_conn: admin_conn} do
assert admin_conn
|> patch("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank2",
new_shortcode: "blank3",
new_filename: "dir_2/blank_3.png"
})
|> json_response(:bad_request) == %{"error" => "Emoji \"blank2\" does not exist"}
end
test "update with empty shortcode", %{admin_conn: admin_conn} do
assert admin_conn
|> patch("/api/pleroma/emoji/packs/test_pack/files", %{
shortcode: "blank",
new_filename: "dir_2/blank_3.png"
})
|> json_response(:bad_request) == %{
"error" => "new_shortcode or new_filename cannot be empty"
}
end
end
describe "POST/DELETE /api/pleroma/emoji/packs/:name" do
test "creating and deleting a pack", %{admin_conn: admin_conn} do
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_created")
|> json_response(200) == "ok"
assert File.exists?("#{@emoji_dir_path}/test_created/pack.json")
assert File.exists?("#{@emoji_path}/test_created/pack.json")
assert Jason.decode!(File.read!("#{@emoji_dir_path}/test_created/pack.json")) == %{
assert Jason.decode!(File.read!("#{@emoji_path}/test_created/pack.json")) == %{
"pack" => %{},
"files" => %{}
}
assert conn
|> delete(emoji_api_path(conn, :delete, "test_created"))
assert admin_conn
|> delete("/api/pleroma/emoji/packs/test_created")
|> json_response(200) == "ok"
refute File.exists?("#{@emoji_dir_path}/test_created/pack.json")
refute File.exists?("#{@emoji_path}/test_created/pack.json")
end
test "filesystem import" do
test "if pack exists", %{admin_conn: admin_conn} do
path = Path.join(@emoji_path, "test_created")
File.mkdir(path)
pack_file = Jason.encode!(%{files: %{}, pack: %{}})
File.write!(Path.join(path, "pack.json"), pack_file)
assert admin_conn
|> post("/api/pleroma/emoji/packs/test_created")
|> json_response(:conflict) == %{
"error" => "A pack named \"test_created\" already exists"
}
on_exit(fn -> File.rm_rf(path) end)
end
test "with empty name", %{admin_conn: admin_conn} do
assert admin_conn
|> post("/api/pleroma/emoji/packs/ ")
|> json_response(:bad_request) == %{"error" => "pack name cannot be empty"}
end
end
test "deleting nonexisting pack", %{admin_conn: admin_conn} do
assert admin_conn
|> delete("/api/pleroma/emoji/packs/non_existing")
|> json_response(:not_found) == %{"error" => "Pack non_existing does not exist"}
end
test "deleting with empty name", %{admin_conn: admin_conn} do
assert admin_conn
|> delete("/api/pleroma/emoji/packs/ ")
|> json_response(:bad_request) == %{"error" => "pack name cannot be empty"}
end
test "filesystem import", %{admin_conn: admin_conn, conn: conn} do
on_exit(fn ->
File.rm!("#{@emoji_dir_path}/test_pack_for_import/emoji.txt")
File.rm!("#{@emoji_dir_path}/test_pack_for_import/pack.json")
File.rm!("#{@emoji_path}/test_pack_for_import/emoji.txt")
File.rm!("#{@emoji_path}/test_pack_for_import/pack.json")
end)
conn = build_conn()
resp = conn |> get(emoji_api_path(conn, :list_packs)) |> json_response(200)
resp = conn |> get("/api/pleroma/emoji/packs") |> json_response(200)
refute Map.has_key?(resp, "test_pack_for_import")
admin = insert(:user, is_admin: true)
%{conn: conn} = oauth_access(["admin:write"], user: admin)
assert conn
|> post(emoji_api_path(conn, :import_from_fs))
assert admin_conn
|> get("/api/pleroma/emoji/packs/import")
|> json_response(200) == ["test_pack_for_import"]
resp = conn |> get(emoji_api_path(conn, :list_packs)) |> json_response(200)
resp = conn |> get("/api/pleroma/emoji/packs") |> json_response(200)
assert resp["test_pack_for_import"]["files"] == %{"blank" => "blank.png"}
File.rm!("#{@emoji_dir_path}/test_pack_for_import/pack.json")
refute File.exists?("#{@emoji_dir_path}/test_pack_for_import/pack.json")
File.rm!("#{@emoji_path}/test_pack_for_import/pack.json")
refute File.exists?("#{@emoji_path}/test_pack_for_import/pack.json")
emoji_txt_content = "blank, blank.png, Fun\n\nblank2, blank.png"
emoji_txt_content = """
blank, blank.png, Fun
blank2, blank.png
foo, /emoji/test_pack_for_import/blank.png
bar
"""
File.write!("#{@emoji_dir_path}/test_pack_for_import/emoji.txt", emoji_txt_content)
File.write!("#{@emoji_path}/test_pack_for_import/emoji.txt", emoji_txt_content)
assert conn
|> post(emoji_api_path(conn, :import_from_fs))
assert admin_conn
|> get("/api/pleroma/emoji/packs/import")
|> json_response(200) == ["test_pack_for_import"]
resp = build_conn() |> get(emoji_api_path(conn, :list_packs)) |> json_response(200)
resp = conn |> get("/api/pleroma/emoji/packs") |> json_response(200)
assert resp["test_pack_for_import"]["files"] == %{
"blank" => "blank.png",
"blank2" => "blank.png"
"blank2" => "blank.png",
"foo" => "blank.png"
}
end
describe "GET /api/pleroma/emoji/packs/:name" do
test "shows pack.json", %{conn: conn} do
assert %{
"files" => %{"blank" => "blank.png"},
"pack" => %{
"can-download" => true,
"description" => "Test description",
"download-sha256" => _,
"homepage" => "https://pleroma.social",
"license" => "Test license",
"share-files" => true
}
} =
conn
|> get("/api/pleroma/emoji/packs/test_pack")
|> json_response(200)
end
test "non existing pack", %{conn: conn} do
assert conn
|> get("/api/pleroma/emoji/packs/non_existing")
|> json_response(:not_found) == %{"error" => "Pack non_existing does not exist"}
end
test "error name", %{conn: conn} do
assert conn
|> get("/api/pleroma/emoji/packs/ ")
|> json_response(:bad_request) == %{"error" => "pack name cannot be empty"}
end
end
end