Commit Graph

47121 Commits

Author SHA1 Message Date
Alex Vandiver b355a0a63e log-search: Default to searching python logfiles.
These have more accurate timestamps, and have user information --
but are harder to parse, and will not show requests when Django or
Tornado is stopped.
2022-05-03 13:44:29 -07:00
Alex Vandiver ba1237119c log-search: Add a tool to search nginx logs by IP/hostname.
This is a script to search nginx log files by server hostname or
client IP address, and output matching lines, all while skipping
common and less-interesting request lines.
2022-05-03 13:44:29 -07:00
patricklauer d86b7c4ef7
provision: Add missing ENDC to avoid colouring all output. 2022-05-03 12:41:43 -07:00
Alex Vandiver e13154f089 puppet: Add ksplice support for 22.04. 2022-05-03 12:36:19 -07:00
Alex Vandiver cda55a40e7 puppet: Add teleport support for 22.04. 2022-05-03 12:36:19 -07:00
Anders Kaseorg e6d85895ca mypy: Link some upstream issues for adding library type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 10:10:06 -07:00
Anders Kaseorg f29553d809 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 10:10:06 -07:00
Anders Kaseorg a7cdcbb6e3 mypy: Use upstream types for asgiref, natsort.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 10:10:06 -07:00
Anders Kaseorg e952641013 install: Resupport Ubuntu 22.04.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 09:41:08 -07:00
Anders Kaseorg 25c87cc7da zulip-puppet-apply: Work around broken Puppet on Ubuntu 22.04.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 09:41:08 -07:00
Aman Agrawal 814abf6764 resize: Don't use `visible` selector to find element states.
This change decreases the time required to open compose
after clicking a message. The amount of time reduced varies with pc.

The time reduction was around 0.4s to 0.6s for me after using a
6x CPU slowdown. This may not sound convincing but the profile
uploaded in #21979 clearly shows the root cause of having a message
click take 10s was the `:visible` query.

Fixes #21979
2022-05-03 09:19:01 -07:00
Adam Sah dbd03b5054 email_backends: Fix concurrent backend testing for generate_emails.
Previously, this command would reliably fail:

```
tools/test-backend --skip-provision-check --parallel=3
  zerver.tests.test_email_log.EmailLogTest.test_forward_address_details
  zerver.tests.test_email_log.EmailLogTest.test_generate_and_clear_email_log
  zerver.tests.test_example.TestDevelopmentEmailsLog
```

and now it reliably succeeds. :-)

After hours of fiddling/googling/hair-tearing, I found that
mocking-away Django Connection.send_messages() was the best:

- We're testing Zulip and not Django.
- Mocking at this lower level exercises more of our code.
- EmailLogBackEnd._do_send_messages() helper method added to simplify mocking.

Fixes #21925.
2022-05-03 08:45:51 -07:00
Anders Kaseorg e9ba9b0e0d zulip-ec2-configure-interfaces: Remove.
Our current EC2 systems don’t have an interface named ‘eth0’, and if
they did, this script would do nothing but crash with ImportError
because we have never installed boto.utils for Python 3.

(The message of commit 2a4d851a7c made
an effort to document for future researchers why this script should
not have been blindly converted to Python 3.  However, commit
2dc6d09c2a (#14278) was evidently
unresearched and untested.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 02:25:59 -07:00
Anders Kaseorg 7acb642fa5 requirements: Upgrade to Tornado 6.
Fixes #8913.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg f23bfe91c0 run-dev: Fix types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 6fd1a558b7 runtornado: Switch to asyncio event loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg c263bfdb41 queue: Use a thread-local Pika connection.
According to the documentation: “Pika does not have any notion of
threading in the code. If you want to use Pika with threading, make
sure you have a Pika connection per thread, created in that thread. It
is not safe to share one Pika connection across threads, with one
exception: you may call the connection method add_callback_threadsafe
from another thread to schedule a callback within an active pika
connection.”

https://pika.readthedocs.io/en/stable/faq.html

This also means that synchronous Django code running in Tornado will
use its own synchronous SimpleQueueClient rather than sharing the
asynchronous TornadoQueueClient, which is unfortunate but necessary as
they’re about to be on different threads.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg c9faefd50e cache: Instantiate only one BMemcached cache backend.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 52b9c59875 requirements: Upgrade asgiref.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 0ef9309e92 run-dev: Switch to asyncio event loop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 5d69dafddb runtornado: Avoid deprecated IOLoop debugging methods.
IOLoop.set_blocking_log_threshold and IOLoop.handle_callback_exception
are removed in Tornado 6.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg e4bf7066f3 tornado: Remove instrument_tornado_ioloop.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg bded7180f7 tornado: Unfork tornado.autoreload.
We previously forked tornado.autoreload to work around a problem where
it would crash if you introduce a syntax error and not recover if you
fix it (https://github.com/tornadoweb/tornado/issues/2398).

A much more maintainable workaround for that issue, at least in
current Tornado, is to use tornado.autoreload as the main module.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg 1c7954b452 run-dev: Avoid deprecated tornado.gen.engine.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Alex Vandiver d891b9590a puppet: Fix non-replicated PostgreSQL 10 and 11 configuration.
6f5ae8d13d removed the `$replication` variable from the
configurations of PostgreSQL 12 and higher, but left it in the
templates for PostgreSQL 10 and 11.  Because `undef != ''`,
deployments on PostgreSQL 10 and 11 started trying to push to S3
backups, regardless of if they were configured, leaving frequent log
messages like:

```
2022-04-30 12:45:47.805 UTC [626d24ec.1f8db0]: [107-1] LOG: archiver process (PID 2086106) exited with exit code 1
2022-04-30 12:45:49.680 UTC [626d24ee.1f8dc3]: [18-1] LOG: checkpoint complete: wrote 19 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=1.910 s, sync=0.022 s, total=1.950 s; sync files=16, longest=0.018 s, average=0.002 s; distance=49 kB, estimate=373 kB
/usr/bin/timeout: failed to run command "/usr/local/bin/env-wal-g": No such file or directory
2022-04-30 12:46:17.852 UTC [626d2f99.1fd4e9]: [1-1] FATAL: archive command failed with exit code 127
2022-04-30 12:46:17.852 UTC [626d2f99.1fd4e9]: [2-1] DETAIL: The failed archive command was: /usr/bin/timeout 10m /usr/local/bin/env-wal-g wal-push pg_wal/000000010000000300000080
```

Switch the PostgreSQL 10 and 11 configuration to check
`s3_backups_bucket`, like the other versions.
2022-05-02 16:46:10 -07:00
Aman Agrawal ce62c11720 message_view_header: Add divider after stream name for spectator.
Since originally divider is a part of sub_count which is not
displayed for spectators, we need to add a new one for them.
2022-05-02 10:05:33 -07:00
Tim Abbott 88110e7188 spectators: Adjust the heading for the login-to-access UI.
The previous "Join the {realm_name} community" was awkward for
organizations that put "community" in their realm name, e.g. "Join the
Zulip development community community".
2022-04-30 13:10:05 -07:00
Anders Kaseorg 646a4d19a3 puppet: Remove quotes for enumerable values.
https://puppet.com/docs/puppet/7/style_guide.html#style_guide_module_design-quoting
“If a string is a value from an enumerable set of options, such as
present and absent, it SHOULD NOT be enclosed in quotes at all.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-29 22:06:46 -07:00
Tim Abbott 4538792517 message controls: Use login_to_access modal for reactions/stars.
Hiding these UI widgets causing layout issues -- specifically, the
position of the \vdots menu looks off with these elements missing.

Enabling this buttons (and opening the login_to_access modal on click)
provides a light advertisement for these features, seems to be the
standard practice for forum-like software, and will also be easier to
maintain.

This effectively reverts f26a76a9d8, in
addition to adding new logic.
2022-04-29 16:35:49 -07:00
Tim Abbott 75c64476b7 compose: Remove special button for spectators.
After playing with several options, it feels cleanest to just have the
closed-compose area look exactly how it would if you were logged in;
popping up the login_to_access modal when clicking those buttons feels
reasonable. The extra button felt buggy, and this customization helps
make the Zulip layout more consistent for spectators.

This effectively reverts 5ffc95f6bb.
2022-04-29 16:35:49 -07:00
Aman Agrawal 64ea9125f3 narrow_banner: Simplify empty narrow messages for spectators. 2022-04-29 16:35:49 -07:00
Tim Abbott d0b3474ef6 login_to_access: Restore paragraph margin in modal.
Now that we have two paragraphs in some settings, this looks better.
2022-04-29 16:35:49 -07:00
Aman Agrawal 43d789993f login_to_access: Change message displayed for empty narrows.
We change the generic message copy while we're at it.

Also, show login_to_access modal when a spectator tries to access
a stream that either does not exist is is not web-public.
2022-04-29 16:35:49 -07:00
Tim Abbott 362761757b spectators: Fix double modal bug with message edit history.
Previously, clicking MOVED/EDITED buttons on a message would pop up
the message edit history modal, which would (after a brief loading
indicator) get a 400 error for the server and then pop the
login_to_access modal on top of the error in that modal.

Fix this with an explicit login_to_access check. This feels like the
cleanest way to avoid churning the UI (hover behaviors, etc.) as would
be required to make this not clickable.

Fixes #21963.
2022-04-29 16:35:49 -07:00
Anders Kaseorg 080a806d60 build-pgroonga: Update PGroonga to 2.3.6.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-29 16:02:45 -07:00
Anders Kaseorg e8e0b045fc Revert "ci: Remove actions/cache@v2 steps from run due to failures."
This reverts commit ae24fe69ed.

The problem was fixed by GitHub.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-29 14:03:12 -07:00
Lauryn Menard ae24fe69ed
ci: Remove actions/cache@v2 steps from run due to failures.
Comments out the steps in 'Create cache directories' that use
`actions/cache@2` so that the CI and production build can pass
while Github support issue is processed.

See https://github.com/actions/cache/issues/794 for an upstream report.
2022-04-29 10:14:51 -07:00
Tanya Singh ac938ef2a1
user profile modal: Display user ID above role.
This makes it more convenient to interact with users via the API.

Fixes #21713.
2022-04-29 08:38:21 -07:00
Lauryn Menard a0d483a9a5 openapi: Fix reference link in `/register-queue` documentation.
Fixes a reference link in `realm_enable_spectator_access`
description in `/register-queue` endpoint documentation.
2022-04-29 08:35:25 -07:00
Alex Vandiver bcd179be0e import: Default the S3 metadata user of emoji to the first owner.
This matches the metadata that we store in the database, and means
that the S3 metadatata invariant of always having a `user_profile_id`
in the metadata.

This does not fix existing imports, which may still have missing
`user_profile_id`s.
2022-04-28 16:51:58 -07:00
Tim Abbott 066b1b58c2 github: Add visible link in pull request template. 2022-04-28 15:29:37 -07:00
Tim Abbott 627144b0c7 settings: Simplify setting organization default language.
The changes in the last few commits changed the semantics of the
organization default language to no longer be the primary source of
information for a user's language when creating a new account.

Here, we change the settings UI and /help/ documentation to reflect
this.
2022-04-28 15:03:26 -07:00
Sahil Batra 47c6100da9 onboarding: Override the language of welcome bot PMs with user setting. 2022-04-28 15:03:26 -07:00
Siddharth Asthana 9954db4b59 users: Use browser locale to set the default language of new user.
This commit reads the browser locale during user registration, and
sets it as default language of user if it is supported by Zulip.
Otherwise, it is set to realm's default language.
2022-04-28 15:03:26 -07:00
Siddharth Asthana 807c73ef17 i18n: Add function to get default language for new users.
It returns realm's default language if the browser's locale
is not available or is unsupported.
2022-04-28 15:03:26 -07:00
Sahil Batra 476a217087 i18n: Add get_browser_language_code function.
This commit adds get_browser_language_code function
which returns None if there is no Accept-language
header in the request or Accept-languge header contains
only unsupported languages or all languages (meaning
header having value of '*'). Otherwise it returns the
language with highest weight/quality-value.
2022-04-28 15:03:26 -07:00
Alex Vandiver 3476f63dca compare-settings-to-template: Handle prod_settings_template renaming. 2022-04-28 14:52:38 -07:00
Alex Vandiver b6b6faa404 compare-settings-to-template: Simplify and dedent logic. 2022-04-28 14:52:38 -07:00
Alex Vandiver d205050ab0 compare-settings-to-template: Fetch 100 per pagination. 2022-04-28 14:52:38 -07:00
Alex Vandiver d79776f80d compare-settings-to-template: Paginate through all tags.
The default page size is 30, which means this only goes back to 4.6 at
present, due to starting with `shared-...` and old `enterprise-...`
tags.
2022-04-28 14:52:38 -07:00