Commit Graph

47413 Commits

Author SHA1 Message Date
Alex Vandiver a35af3f38b install/upgrade: Allow new packages during `apt-get upgrade`.
`postgresql-14.4` is a notable upgrade in the PostgreSQL series, as it
fixes potential database corruption from `CREATE INDEX CONCURRENTLY`
statements which are run while rows are modified[1].  However, it also
requires an upgrade from `libllvm9` to `libllvm10`, which means it is
not installed by a mere `apt-get upgrade`.

Add the `--with-new-pkgs` flag to all of the potentially relevant
`apt-get upgrade` calls, so that this (and similar) packages are
upgraded successfully.

[1]: https://www.postgresql.org/docs/release/14.4/
2022-06-21 11:21:49 -07:00
Anders Kaseorg 95303a9929 assets: Remove license for previously deleted emoji images.
The images themselves had been deleted by commit
cc33b68d73, and were then zanitized out
of the commit history.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-10 12:27:48 -07:00
Anders Kaseorg a1bdadfb3d audio: Remove the copy of zulip.ogg outside notification_sounds.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-10 12:27:48 -07:00
Anders Kaseorg 048b52ed5c images: Delete various unused images.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-10 12:27:48 -07:00
Anders Kaseorg 1e6bc5b36b styles: Delete some unused CSS.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-10 12:27:48 -07:00
Alex Vandiver 5bdc4b3562 upgrade-zulip-from-git: init, then add remote.
30457ecd02 removed the `--mirror` from
initial clones, but did not add back `--bare`, which `--mirror`
implies.  This leads to `/srv/zulip.git` having a working tree in it,
with a `/srv/zulip.git/.git` directory.

This is mostly harmless, and since the bug was recent, not worth
introducing additional complexity into the upgrade process to handle.

Calling `git clone --bare`, however, would clone the refs into
`refs/heads/`, not the `refs/remotes/origin/` we want.  Instead, use
`git init --bare`, followed by `git remote add origin`.  The remote
will be fetched by the usual `git fetch --all --prune` which is below.
2022-06-09 11:18:42 -07:00
Alex Vandiver 1639792e9e upgrade-zulip-from-git: Check fetch refspecs, not mirror flag.
While the `remote.origin.mirror` boolean being set is a very good
proxy for having been cloned with `--mirror`, is technically only used
when pushing into the remote[1].  What we care about is if fetches
from this remote will overwrite `refs/heads/`, or all of `refs/` --
the latter of which is most likely, from having run `git clone
--bare`.

Detect either of these fetch refspecs, and not the mirror flag.  We
let the upgrade process error out if `remote.origin.fetch` is unset,
as that represents an unexpected state.  We ignore failures to unset
the `remote.origin.mirror` flag, in case it is not set already.

[1]: https://git-scm.com/docs/git-config#Documentation/git-config.txt-remoteltnamegtmirror
2022-06-09 11:18:42 -07:00
Anders Kaseorg 0430705d13 test_tornado: Call process_event on first fetch_events return.
The 0.1 second delay was sometimes not long enough to guarantee we hit
the async response path, resulting in a nondeterministic coverage
failure.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-08 17:10:38 -07:00
Anders Kaseorg e112b619cc tornado: Fix race condition on handler._request.
Commit 6fd1a558b7 (#21469) introduced an
await point where get_events_backend calls fetch_events in order to
switch threads.  This opened the possibility that, in the window
between the connect_handler call in fetch_events and the old location
of this assignment in get_events_backend, an event could arrive,
causing ClientDescriptor.add_event to crash on missing
handler._request.  Fix this by assigning handler._request earlier.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-08 17:10:38 -07:00
Mateusz Mandera acfa55138e retention: Add docstring info on how archive cleaning works.
In particular, it's important to record the special treatment around
ArchivedAttachment rows not being deleted in this step.
2022-06-08 15:12:36 -07:00
Tim Abbott 1304c435ce populate_db: Restore test database predictable set of topics.
856eff0fe6 perturbed the initial state
of the 30 messages in the test database, which unfortunately some
tests depend on.
2022-06-08 12:34:45 -07:00
David Rosa 3474567521 help-docs: Document "Copy link to topic" mobile app feature.
Adds a "Mobile" tab with straightforward instructions for accessing
the long-press menu using two new macros.

Knowing where to access the long-press menu is intuitive, except when
the user is in a topic narrow. It's not immediately obvious that the
top bar can be long-pressed so this adds a "Tip block" using a new macro
to clarify things in this scenario.

Also, this combines the instructions for Desktop and Web into a single
tab because the numbered steps work on both platforms. So this documents
the alternate method via the browser's address bar as a "Tip block" to
avoid stacking alternative numbered steps into a single tab.
This updates the stream and message link instructions too.

Fixes: #22147.
2022-06-08 12:24:09 -07:00
David Rosa 52ef574d3e help-docs: Fix minor errors in "Link to a message or conversation".
Removes the ":" which have accidentally ended up in the "Get a link
to a specific topic" and "Get a link to a specific stream" headings.

Renames the "Via browser's address bar" tab to "Web" so that it
stays consistent with other help center articles.

Fixes part of #22147.
2022-06-08 12:20:32 -07:00
Aman Agrawal d7444f919d app-loading: Add a delay before showing the reload message.
Fixes #22182

This message often flashes on screen briefly, causing unnecessary
worry for the user (is the app likely to not load?).
To address this, we add a delay before the message is shown.

As a consequence, we change the notice to no longer suggest waiting a
few seconds, since we did that before showing it.
2022-06-08 12:17:55 -07:00
Tim Abbott 856eff0fe6 populate_db: Ensure many streams have "more topics" in development.
We increase the total number of messages, since increasing the number
of topics would otherwise have the side effect of making it hard to
find longer conversations.
2022-06-08 12:08:56 -07:00
NerdyLucifer c31ab1bcb5 recent-topics: Mark as read using unread counters in Recent topics.
The PR changes the following behaviors and UI:
1. Removes the checkmark button to mark the topic as read in
"Recent Topics".
2. Make the unread messages counter be the button for marking
all messages in the topic as read. The unread messages counter
is made clickable and tooltip is set to "Mark as read".

In "recent_topic_row.hbs", remove the checkmark button and add
classes and attributes to ".unread_counter" to give it desirable
behaviour on clicking.

In "zulip.css" set "opacity: 0.7" for ".on_hover_topic_read".

In "recent_topics.css" we set the background-color of unread counter to
hsl(105, 2%, 50%) to decrease fading of unread counter.

Fixes: #21654
2022-06-08 11:58:25 -07:00
Zixuan James Li 426f8ce385 tests: Replace `HttpResponse` with `TestHttpResponse.`
Since `HttpResponse` is an inaccurate representation of the
monkey-patched response object returned by the Django test client, we
replace it with `_MonkeyPatchedWSGIResponse` as `TestHttpResponse`.

This replaces `HttpResponse` in zerver/tests, analytics/tests, coporate/tests,
zerver/lib/test_classes.py, and zerver/lib/test_helpers.py with
`TestHttpResponse`. Several files in zerver/tests are excluded
from this substitution.

This commit is auto-generated by a script, with manual adjustments on certain
files squashed into it.

This is a part of the django-stubs refactorings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-08 11:25:03 -07:00
Alya Abbott 3e11066f9f docs: Update documentation of max number of thumbnails per message.
This was changed to 10 in https://github.com/zulip/zulip/pull/20789.
2022-06-08 11:24:27 -07:00
Alya Abbott 2e19cd1560 portico: Update Recurse Center link on /plans to point to case study. 2022-06-07 17:54:41 -07:00
Anders Kaseorg d5fea08b8a cache: Remove needless monkey patching.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-07 09:37:43 -07:00
Sahil Batra dfacd62d93 streams: Update error message returned for administrative access.
We have now decided to not continue with the stream administrator
concept as we are changing the permissions model to be based on
user groups as per #19525. So, this commit updates the error message
to "Must be an organization administrator".
2022-06-07 09:36:24 -07:00
Zixuan James Li a142fbff85 tests: Refactor away result.json() calls with helpers.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-06 23:06:00 -07:00
Anders Kaseorg a6e2453323 emojisets: Use Google modern for emoji picker with text theme.
Followup to commit f743dda008 (#19672).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-06 15:53:32 -07:00
Alya Abbott 8a26a7414f help center: Clarify and expand documentation on inviting new users. 2022-06-03 17:40:06 -07:00
Lauryn Menard 1d1a68b3db help-docs: Add information about email invite language.
Adds minimal descriptions for email and general invitation links,
and documents that email invitations are translated into the
organization's notifications language.
2022-06-03 17:40:06 -07:00
Anders Kaseorg a7f9c4f958 logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-03 12:27:23 -07:00
Anders Kaseorg dede49ad78 test_counts: Rename confusing assertCountEquals helper.
It looks too much like the standard TestCase.assertCountEqual method.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-03 12:24:57 -07:00
Alex Vandiver d0c5389d60 gitlab: Fix event_name -> object_kind defaulting.
94457732c1 changed this from:

```py
event_name = payload.get("event_name", payload.get("object_kind")).tame(check_string)
```

...to:
```py
event_name = payload.get("event_name", payload["object_kind"]).tame(check_string)
```

Which causes a failure when `event_name` exists but `object_kind` does
not, since the default is evaluated first.

Switch to an `if` statement to clarify the fallbacks better.
2022-06-02 23:23:19 -07:00
Mateusz Mandera a671ae9749 get_old_unclaimed_attachments: Add docstring explaining the logic. 2022-06-02 17:32:23 -07:00
Mateusz Mandera 09dc166b45 do_delete_old_unclaimed_attachments: Consider ArchivedAttachment rows.
This function is oblivious to the existence of ArchivedAttachment, which
is incorrect. A file can be removed if and only if it is not referenced
by any Messages or ArchivedMessages.
2022-06-02 17:32:23 -07:00
Mateusz Mandera 5ff4754090 test_upload: Fix some URLs to uploaded files.
Using http://localhost:9991 is incorrect - e.g. messages sent with file
urls constructed trigger do_claim_attachments to be called with empty
list in potential_path_ids.

realm.host should be used in all these places, like in the other tests
in the file.
2022-06-02 17:32:23 -07:00
evykassirer 72166f1b03 puppeteer: Fix get_element_text including hidden text.
Previously, this function used the `textContent` property, which
includes hidden elements. Fix this by using innerText instead.
2022-06-02 16:44:24 -07:00
evykassirer 67771f80d0 drafts: Remove 'pro tip' from draft modal.
This keyboard shortcut advertisement dates from the original version
of the overlay, before we advertised the shortcut in other ways.

Discussion here:
https://github.com/zulip/zulip/pull/22127#discussion_r882270506
2022-06-02 16:42:37 -07:00
evykassirer e039bdd1e1 narrow: Replace pm_string() with _id and _email variants.
This makes the format returned by these functions more clear.
2022-06-02 16:41:06 -07:00
Sahil Batra 6e8523015c populate_db: Add users to initial private stream in mit and lear realm.
We do not allow keeping vacant private streams as we deactivate them
when all users are unsubscribed, so it is better to add at least one
user in the initial private stream created while creating realm.
2022-06-02 13:42:40 -07:00
Alex Vandiver aa46d8d2a8 puppet: Enable strict typo checking in uwsgi. 2022-06-02 13:20:48 -07:00
Alex Vandiver 18ec3b6215 puppet: Enable background worker threads in uwsgi.
Without this, uwsgi does not release the GIL before going back into
`epoll_wait` to wait for the next request.  This results in any
background threads languishing, unserviced.[1]

Practically, this results in Sentry background reporter threads timing
out when attempting to post results -- but only in situations with low
traffic, as in those significant time is spent in `epoll_wait`.  This
is seen in logs as:

    WARN [urllib3.connectionpool] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)'))': /api/123456789/envelope/

Or:

    WARN [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response'))': /api/123456789/envelope/

Sentry attempts to detect this and warn, but due to startup ordering,
the warning is not printed without lazy-loading.

Enable threads, at a miniscule performance cost, in order to support
background workers like Sentry[2].

[1] https://github.com/unbit/uwsgi/issues/1141#issuecomment-169042767
[2] https://docs.sentry.io/clients/python/advanced/#a-note-on-uwsgi
2022-06-02 13:20:48 -07:00
Alex Vandiver 919c904091 puppet: Give the uwsgi processes a shorter process name.
Previously, the complete command line, which is quite long, is shown:

    3963143 ?        SN     0:00 /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
    3963144 ?        SN     0:03  \_ /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
    3963145 ?        SN     0:03  \_ /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
    3963146 ?        SN     0:03  \_ /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
    3963147 ?        SN     0:03  \_ /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
    3963148 ?        SN     0:03  \_ /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
    3963149 ?        SN     0:03  \_ /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini

Configure uwsgi to rename and number the processes.  This results in:

    3907613 ?        SN     0:00 zulip-django uWSGI master
    3907614 ?        SN     0:05  \_ zulip-django uWSGI worker 1
    3907615 ?        SN     0:03  \_ zulip-django uWSGI worker 2
    3907616 ?        SN     0:05  \_ zulip-django uWSGI worker 3
    3907617 ?        SN     0:05  \_ zulip-django uWSGI worker 4
    3907618 ?        SN     0:05  \_ zulip-django uWSGI worker 5
    3907619 ?        SN     0:05  \_ zulip-django uWSGI worker 6
2022-06-02 13:20:48 -07:00
Alex Vandiver a522ad1d9a puppet: Always create a uwsgi master control socket.
This is potentially useful even with rolling restarts disabled.
2022-06-02 13:20:48 -07:00
Alex Vandiver 721a101f12 puppet: Reorganize and comment uwsgi.ini file.
As the uwsgi documentation is somewhat obtuse, more comments are added
here than might usually be.
2022-06-02 13:20:48 -07:00
madrix01 b5e21ceeaf recent_topics: Prevent topic name from extending into unread column.
Previously, when there were no unread messages in a topic, the topic
name was allowed to expand into the space allocated to the unread
count. This matched the behavior of the left sidebar. However, the
left sidebar has extremely limited horizontal space, and being able to
display a few extra characters is useful; recent topics does not have
this constraint. Further, recent topics wraps long topic names on
overflow, which looks ugly when using use the unread count's space.

So we switch to having the unread count element consume space even
when there is no count, using `visiblity:hidden`.

Fixes a part of #19449
2022-06-02 12:49:02 -07:00
Mateusz Mandera 3ce61958a8 docs: Fix typo in gitlab callback url in dev setup instructions.
The endpoint is /complete/gitlab/ - with the slash.
2022-06-02 12:04:18 -07:00
Anders Kaseorg 2e9d5193b4 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00
Anders Kaseorg 79842fd228 install-shfmt: Upgrade shfmt from 3.4.3 to 3.5.0.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00
Anders Kaseorg 61c9740bbd install-yarn: Upgrade Yarn from 1.22.17 to 1.22.18.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00
Anders Kaseorg 2007c75061 install-node: Upgrade Node.js from 16.14.1 to 16.15.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-02 12:03:49 -07:00
Aman Agrawal 422b096b9a language_selection_modal: Add more padding between languages. 2022-06-01 17:08:00 -07:00
Aman Agrawal 66b80c8ae8 admin: Use language_selection_widget to set notification language.
Fixes #21948
2022-06-01 17:08:00 -07:00
Aman Agrawal 59e676b1f5 gear_menu: Allow spectators to select their default language.
Set the default_language as cookie and reload the page so that
the spectator can immediately see the language change in effect.

We can reload the page forcefully for spectators since there is
no chance of any work being lost. It is possible that the spectator
may lose the selected message on doing so.

This requires a new dependency, to be able to set cookies from
frontend JavaScript.

Fixes #21961
2022-06-01 17:08:00 -07:00
Aman Agrawal 085443c722 home: Get default_language info from cookie for spectator.
This allows us to set default_language as cookie in frontend which
can be used on server load to send the relevant
translation data to the client.
2022-06-01 17:08:00 -07:00