Mark Felder 372d68a55e RichMedia refactor ... Rich Media parsing was previously handled on-demand with a 2 second HTTP request timeout and retained only in Cachex. Every time a Pleroma instance is restarted it will have to request and parse the data for each status with a URL detected. When fetching a batch of statuses they were processed in parallel to attempt to keep the maximum latency at 2 seconds, but often resulted in a timeline appearing to hang during loading due to a URL that could not be successfully reached. URLs which had images links that expire (Amazon AWS) were parsed and inserted with a TTL to ensure the image link would not break. Rich Media data is now cached in the database and fetched asynchronously. Cachex is used as a read-through cache. When the data becomes available we stream an update to the clients. If the result is returned quickly the experience is almost seamless. Activities were already processed for their Rich Media data during ingestion to warm the cache, so users should not normally encounter the asynchronous loading of the Rich Media data. Implementation notes: - The async worker is a Task with a globally unique process name to prevent duplicate processing of the same URL - The Task will attempt to fetch the data 3 times with increasing sleep time between attempts - The HTTP request obeys the default HTTP request timeout value instead of 2 seconds - URLs that cannot be successfully parsed due to an unexpected error receives a negative cache entry for 15 minutes - URLs that fail with an expected error will receive a negative cache with no TTL - Activities that have no detected URLs insert a nil value in the Cachex :scrubber_cache so we do not repeat parsing the object content with Floki every time the activity is rendered - Expiring image URLs are handled with an Oban job - There is no automatic cleanup of the Rich Media data in the database, but it is safe to delete at any time - The post draft/preview feature makes the URL processing synchronous so the rendered post preview will have an accurate rendering Overall performance of timelines and creating new posts which contain URLs is greatly improved.		2024-03-19 12:12:07 -04:00
..
2.6.0-mergeback.skip	Bump package version for mergeback	2023-10-31 20:49:31 -04:00
2.6.1-mergeback.skip	Skip changelog for 2.6.1 mergeback	2023-12-15 19:45:23 -05:00
3900.change	Update to Phoenix 1.7	2023-06-03 14:06:28 -04:00
3987.fix	add changelog	2023-12-07 20:33:23 +09:00
account-rendering-auth-check.fix	Fix authentication check on account rendering when bio is defined	2024-01-04 16:57:48 +03:00
add-outbox.fix	ap userview: add outbox field.	2023-12-15 09:55:06 +04:00
anonymous-exception-else.fix	Fix #strip_report_status_data	2023-11-27 17:56:43 +04:00
api-docs.skip	Update API docs for my changes	2024-01-17 17:13:21 +01:00
atom-leak.skip	Changelog	2024-02-14 18:04:39 -05:00
authorize-interaction.add	Support /authorize-interaction route used by Mastodon	2023-11-02 13:47:23 +01:00
backups-follows.add	Include following/followers in backups	2024-03-10 23:53:12 +01:00
bad_inbox_request.change	Return a 400 from a bad delivery attempt to the inbox	2023-12-19 13:56:17 -05:00
bandit.change	Revert "Add support for a "deps" changelog type and document deps changes since 2.6.1 release"	2024-02-16 12:53:18 -05:00
bare_uri_test.skip	ObjectValidators.BareUriTest: Replace calls of SafeText to BareUri	2023-11-08 02:17:50 +01:00
benchee.skip	Ensure benchee doesn't run unless we are executing benchmarks	2023-11-08 12:44:57 -05:00
blurhash.change	Docs: Remove rust references	2023-11-28 15:21:34 +04:00
bookmark-folders.add	Allow to group bookmarks in folders	2024-03-01 11:04:01 +01:00
bugfix-ccworks.fix	Test incoming federation from Convergence AP Bridge	2024-01-26 17:05:18 +01:00
build-release-with-local-libvips.skip	Changelog	2023-12-11 12:48:53 +04:00
chat-attachment-empty-array.fix	ChatMessage: Tolerate attachment field set to an empty array	2023-12-29 05:32:47 +01:00
config-stat-symlink.fix	Config: Check the permissions of the linked file instead of the symlink↵	2024-02-14 18:19:50 +01:00
content-length.fix	MediaProxy RFC compliance	2024-02-14 13:25:52 -05:00
deprecations.skip	Chase deprecations/warnings for Elixir 1.15	2023-12-20 20:17:25 +00:00
deprecations2.skip	More deprecation fixes	2023-12-20 16:47:58 -05:00
deps-bump-2024-01-25.skip	Bump dependencies	2024-01-26 15:50:45 -05:00
dialyzer.skip	Dialyzer fixes	2024-01-20 19:29:29 -05:00
dialyzer2.skip	Changelog	2024-01-26 21:05:43 -05:00
dialyzer3.skip	Changelog	2024-01-31 10:13:02 -05:00
dialyzer4.skip	Changelog	2024-02-02 12:14:21 -05:00
digest_emails.fix	Fix digest email processing, consolidate Oban queues	2023-11-07 15:14:36 -05:00
doc-fix.skip	Add changelog	2023-11-29 19:42:40 +04:00
docs-max-elixir-erlang.change	Bump version to 2.6.1	2023-12-14 20:14:41 -05:00
emoji-download-paginate.fix	Fetch count before downloading the pack and use that as page size	2024-01-07 17:07:50 +03:00
emoji-use-v1.fix	Make remote emoji packs API use specifically the V1 URL	2024-01-07 15:30:52 +03:00
exile-bsds.skip	Exile: switch to fork with BSD compile fix	2024-01-22 13:44:20 -05:00
exile-freebsd.skip	Exile: change to upstream pre-release commit that fixes build on FreeBSD	2024-02-23 15:36:37 -05:00
exile-macos.skip	Exile: fix for MacOS dev environments	2024-01-22 18:37:13 -05:00
exile.skip	Replace custom fifo implementation with Exile	2024-01-22 10:13:17 -05:00
favicon.add	Add support for configuring a favicon and embed PWA manifest in server-generated-meta	2023-11-14 11:05:23 +01:00
federation_status-access.change	router: Make /federation_status publicly available	2023-11-14 10:48:30 +01:00
federator-modules.remove	Changelog	2023-12-28 14:55:26 -05:00
federator.skip	Changelog	2023-12-28 23:32:21 -05:00
finch_redirects.fix	Fix following redirects with Finch	2023-12-29 00:50:50 -05:00
fix-bookmark-folder-tests.skip	Add changelog	2024-03-19 15:03:16 +04:00
fix-dockerfile.skip	Dockerfile: Use platform libvips.	2023-12-12 11:36:34 +04:00
fix-duplicate-inbox-deliveries.fix	Fix duplicate inbox deliveries	2024-01-11 20:48:35 +03:00
fix-otp-comparison.skip	Fix invalid string comparison for OTP versions and replace with config	2023-12-20 23:13:33 +00:00
fix-tests.skip	StatusViewTest: Fix tests.	2023-12-13 11:19:08 +04:00
force-mention-mrf.add	Add ForceMention mrf	2024-03-01 18:16:09 +01:00
framegrabs.fix	Fix ffmpeg framegrabs with Exile	2024-03-07 17:38:21 -05:00
frontend-management.add	Bump version to 2.6.1	2023-12-14 20:14:41 -05:00
generate-unset-user-keys-migration.skip	Fix GenerateUnsetUserKeys migration	2023-11-13 14:18:22 +01:00
group-actor.add	Add changelog for group actors	2023-12-27 12:28:11 -05:00
gun-logs.skip	Change some Gun connection pool logs to debug level	2024-02-14 13:28:32 -05:00
gun_pool.fix	Fix Gun connection supervisor logic error	2024-02-09 10:48:40 -05:00
gun_pool2.fix	Prevent publisher jobs from erroring if the connection pool is full	2024-02-22 14:14:40 -05:00
gun_pool3.skip	Rename variable to make the worker retry logic easier to read	2024-02-23 11:12:10 -05:00
handle_object_fetch_failures.change	Handle 401s as I have observed it in the wild	2023-12-28 23:09:33 -05:00
healthcheck-disabled-error.fix	Bump version to 2.6.1	2023-12-14 20:14:41 -05:00
instance-contact-account.add	Add contact account to InstanceView	2023-10-26 23:32:08 +02:00
instance-defdelegates.skip	These functions in Pleroma.Instances should be defdelegates to Pleroma.Instances.Instance	2023-12-27 12:44:16 -05:00
instance-v2.add	Implement api/v2/instance route	2023-08-11 00:08:05 +02:00
instance-v2.skip	Add some missing fields to instanceV2	2024-02-27 13:25:26 +01:00
issue-3241.fix	add changelog.d	2024-03-02 18:24:39 +09:00
last_status_at.change	MastoAPI AccountView: Change last_status_at to be a date	2023-11-16 15:52:01 +01:00
link-verification.add	Verify link ownership with rel="me"	2023-11-02 13:30:46 +01:00
loading-order-test-fix.skip	Add changelog.	2023-11-30 09:55:04 +04:00
local-webfinger.fix	update changelog	2023-11-02 13:31:59 +01:00
mastodon_directory.fix	Pleroma.Web.MastodonAPI.DirectoryController: dialyzer errors	2024-01-28 17:41:07 -05:00
meilisearch.add	Add changelog.	2023-11-12 15:00:51 +04:00
memleak.fix	Force more frequent full_sweep GC runs on the Websocket processes	2024-02-17 09:53:52 -05:00
mergeback-2.6.2.skip	Mergeback of security release 2.6.2	2024-02-20 09:34:11 +01:00
migration-fix.skip	Permit the index creation to run concurrently	2023-12-08 15:48:00 -05:00
mrf-regex-error.fix	MRF: Log sensible error for subdomains_regex	2024-01-02 15:37:41 +01:00
mrf-steal-emoji-extname.fix	MRF.StealEmojiPolicy: Properly add fallback extension to filenames missing one	2024-01-07 19:37:35 +01:00
mrf_hashtags.fix	MRF.HashtagPolicy: fix dialyzer error	2024-01-22 18:37:19 -05:00
nil-content-map.fix	Support objects with a nil contentMap (firefish)	2024-01-11 07:16:45 +01:00
no-async-with-clear-config.skip	Tests: Don't run tests that use clear_config asynchronously.	2023-12-10 16:27:23 +04:00
notifications-index.fix	Changelog for notifications fix pulled in from Rebased	2024-02-12 14:14:38 -05:00
notifications.fix	Notifications: filter on users rather than activities	2024-03-17 20:58:59 +01:00
oauth-nickname.skip	Use User.full_nickname/1 in oauth html template	2024-02-09 16:46:45 +01:00
opengraph-rich-media-proxy.add	Add media proxy to opengraph rich media cards	2023-12-12 18:32:02 +03:00
optimistic-inbox.change	Optimistic Inbox	2023-12-08 18:13:43 -05:00
otp26.add	Support for Erlang OTP 26	2023-12-29 23:22:31 -05:00
postgres-jit.change	Update minimum Postgres version to 11.0; disable JIT	2024-03-18 15:36:26 -04:00
prioritize-direct-recipients.add	Actually write changelog	2023-12-16 21:41:28 +04:00
promex.change	Revert "Add support for a "deps" changelog type and document deps changes since 2.6.1 release"	2024-02-16 12:53:18 -05:00
public-polls.add	Add entry to @context, tests	2023-10-29 00:02:17 +02:00
publisher_discard.change	Activity publishing failures will prevent the job from retrying if the publishing request returns a 403 or 410	2023-12-29 00:25:33 -05:00
publisher_log.change	Make the Publisher log error less noisy	2023-12-29 00:25:11 -05:00
qtfaststart.fix	Change QTFastStart to recover gracefully if it encounters an error during bitstring matching	2023-12-29 14:12:44 -05:00
quotes-count.skip	Count and display post quotes	2023-11-12 13:38:08 +00:00
reachability.change	Change set_reachable/1 to an upsert	2023-12-06 15:50:00 -05:00
remote-fetcher-error.skip	RemoteFetcherWorker: Make sure {:error, _} is returned on failure	2024-02-23 01:59:30 +01:00
rich_media.fix	Fix Rich Media Previews for updated activities	2024-02-04 23:47:04 -05:00
rich_media_refactor.change	RichMedia refactor	2024-03-19 12:12:07 -04:00
rich_media_tests.skip	RichMedia.Helpers: move the validate_page_url/1 function to the Parser module	2024-02-06 18:34:02 -05:00
scrobble-url.add	Add optional URL value for scrobbles	2023-11-23 04:51:51 -07:00
scrubbers-html4-GtS.add	scrubbers/default: Add more formatting elements from HTML4 / GoToSocial	2023-11-16 18:27:32 +01:00
system-cflags.fix	Bump version to 2.6.1	2023-12-14 20:14:41 -05:00
tesla.deps	Tesla changelog	2024-02-15 10:24:32 -05:00
test-improvements.skip	Add changelog	2024-03-19 10:38:29 +04:00
testsecrets.skip	Clarify location of test.secret.exs file	2023-12-20 14:12:06 -05:00
transient-validators-defaults.change	Update transient-validators-defaults.change	2024-03-19 12:03:43 +00:00
typo.skip	Fix some more typos	2023-12-28 00:17:04 +01:00
vips.change	Update changelogs	2023-11-17 11:21:46 -05:00
web_push.fix	Fix Web Push notification delivery	2023-12-19 10:56:55 -05:00
websocket-refactor.change	Websocket refactor changelog	2024-02-14 15:27:12 -05:00