Commit Graph

39956 Commits

Author SHA1 Message Date
Alex Vandiver c2185a81d6 puppet: Move top-level zulip deployments into "profile" directory.
This moves the puppet configuration closer to the "roles and profiles
method"[1] which is suggested for organizing puppet classes.  Notably,
here it makes clear which classes are meant to be able to stand alone
as deployments.

Shims are left behind at the previous names, for compatibility with
existing `zulip.conf` files when upgrading.

[1] https://puppet.com/docs/pe/2019.8/the_roles_and_profiles_method
2020-10-27 13:29:19 -07:00
Alex Vandiver 27cfb14d92 puppet: Only include zulip::base for top-level deploys.
This also removes direct includes of `zulip::common`, making
`zulip::base` gatekeep the inclusion of it.  This helps enforce that
any top-level deploy only needs include a single class, and that any
configuration which is not meant to be deployed by itself will not
apply, due to lack of `zulip::common` include.

The following commit will better differentiate these top-level deploys
by moving them into a subdirectory.
2020-10-27 13:29:19 -07:00
Alex Vandiver 34e8c2c61e puppet: Move total_memory_mb from zulip::base into zulip::common.
This makes `zulip::common` used only for variable-setting, and
`zulip::base` used only for resource creation.
2020-10-27 13:29:19 -07:00
Alex Vandiver 7bb888c2ec puppet: Template supervisor.conf for redhat paths. 2020-10-27 13:29:19 -07:00
Alex Vandiver 3ab9b31d2f puppet: Purge all un-managed supervisor configuration files.
Relying on `defined(Class['...'])` makes the class sensitive to
resource evaluation ordering, and thus brittle.  It is also only
functional for a single service (thumbor).

Generalize by using `purge => true` for the directory to automatically
remove all un-managed files.  This is more general than the previous
form, and may result in additional not-managed services being removed.
2020-10-27 13:29:19 -07:00
Alex Vandiver 4cca1a2076 docs: Reword pgroonga sections.
These were somewhat stilted as originally written; reword.
2020-10-27 11:27:20 -07:00
Chris Bobbe 500f92169e openapi: Fix the "the the" instances. 2020-10-26 12:50:01 -07:00
Chris Bobbe dd43d4feb8 openapi: Note that a user's `avatar_url` may be missing.
Like it already says in the detail about
`user_avatar_url_field_optional`, but on the field itself [1].

[1] https://github.com/zulip/zulip-mobile/pull/4230#discussion_r493109645
2020-10-26 12:50:01 -07:00
Alex Vandiver abb8b2166f docs: Fix name of realm rename function. 2020-10-26 12:48:56 -07:00
Tim Abbott 6d7cd351a3 events: Optimize creating streams for new users.
During the new user creation code path, there can be no existing
active clients for the user being created, so we can skip the code to
send events to that user's clients.

The tests here reflect that we need to send fewer events, and do fewer
queries that would have been spent computing data for these..

Fixes #16503, combined with the long series of recent changes by Steve
Howell to fix super-linear behavior in this code path.
2020-10-26 12:47:15 -07:00
Steve Howell 88a7a1b002 events: Optimize peer_add/peer_remove for public streams.
We no bulk up peer_add/peer_remove events by user if the
same user has subscribed to multiple streams (and just
that single user).

This mostly optimizes the new-user codepath, but the
algorithm is a bit more general in nature.
2020-10-26 12:33:28 -07:00
Alex Vandiver 7cf737988d queue: Be more explicit about test/real queue division. 2020-10-26 12:32:47 -07:00
Anders Kaseorg 31d0141a30 python: Close opened files.
Fixes various instances of ‘ResourceWarning: unclosed file’ with
python -Wd.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Anders Kaseorg bb2db31f29 backends: Remove unnecessary deepcopy.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-26 12:31:30 -07:00
Steve Howell 3ad1335a97 tests: Clear ContentType cache for user test.
This keeps the number of queries predictable.
2020-10-26 07:18:08 -04:00
Steve Howell 5ef01b3ad8 tests: Fix test_create_user_with_multiple_streams.
This test was flaky due to some date-related
non-determinism.  I make all the Message objects
current to make add_new_user_history reliably
try to bulk-update UserMessage rows to read.
2020-10-26 07:18:08 -04:00
Steve Howell 0e7e4f1ad3 api feature level: Bump level for peer_add/peer_remove. 2020-10-26 07:18:08 -04:00
Anders Kaseorg 96dee7e40b decorator: Unfork redirect_to_login.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-25 14:58:53 -07:00
Alex Vandiver 1d54630b4e log: Rename email-deliverer.log to match other files. 2020-10-25 14:56:37 -07:00
Alex Vandiver 93d661d119 puppet: Configure logrotate for all logger files.
This adds log rotation to all /var/log/zulip files.
2020-10-25 14:56:37 -07:00
sahil839 a96811ab58 management: Replace knight command with change_user_role command.
We replace knight command with change_user_role command which
allows us to change role of a user to owner, admins, member and
guest. We can also give/revoke api_super_user permission using
this command.

Tweaked by tabbott to improve the logging output and update documentation.

Fixes #16586.
2020-10-25 14:55:08 -07:00
Anders Kaseorg 0f11e2a361 dependencies: Upgrade JavaScript dependencies.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-24 02:09:14 -07:00
Harsh Srivastava 9b31df009b openapi: Fix excessively large test_events failure output.
Because of the very large `oneOf` clause of the formats of events
possible in Zulip's `GET /events` system, we had issues with
`test-backend` failures for missing documentation for a new event
format being like 1000 lines of output, which was very much unhelpful.

Fix this by limiting the output use only the oneOf variants that are
broadly similar to the actual payload received.

Fixes #16023.
2020-10-23 17:00:17 -07:00
Alex Vandiver e3dacdadae help: Gitter imports do not use a token to fetch latest data.
This is a copy/paste error from the Slack import.
2020-10-23 16:47:28 -07:00
Alex Vandiver c296b5d819 puppet: Allow unattended-upgrades for all but servers.
Restarting servers is what can cause service interruptions, and
increase risk.  Add all of the servers that we use to the list of
ignored packages, and uncomment the default allowed-origins in order
to enable unattended upgrades.
2020-10-23 16:46:06 -07:00
Vishnu KS f1ceab69c9 activity: Speed up the main query in /activity. 2020-10-23 12:45:20 -07:00
Anders Kaseorg 72d6ff3c3b docs: Fix more capitalization issues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:46:55 -07:00
Alex Vandiver a7d1fd9ffb puppet: Remove non-working apt::source.
d2aa81858c replaced the `apt::source` to set up debathena with
`Exec['setup-apt-repo-debathena']`, but mistakenly left the
`apt::source` in place in `zmirror` (but not `zmirror_personals`).
The `apt::source` resource type was later removed in c9d54f7854,
making the manifest to apply on `zmirror`.

Remove the broken and unnecessary `apt::source` resource.
2020-10-23 11:31:20 -07:00
Anders Kaseorg e513b75e86 markdown: Remove handler for old bug with incompatible twitter library.
See commit 8b002040e0 and #86.  The
development environment bug that necessitated this handler has long
been irrelevant.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:30:26 -07:00
Anders Kaseorg b9fd49a2c6 mypy: Correct mistaken *args type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-23 11:29:13 -07:00
Anders Kaseorg d295da676b test_message_fetch: Clean up obsolete PGroonga bug workaround.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-22 23:27:23 -07:00
Anders Kaseorg 831d086110 i18n: Fix get_language_translation_data for zh_TW.
Fixes #16600.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-22 16:43:02 -07:00
Alex Vandiver 48e06c25ba puppet: Switch nagios SSH checks to id_ed25519 key.
The ssh-rsa algorithm was deprecated[1] in OpenSSH 8.2 (2020-02-14) and
will be removed in a future release.

[1] https://www.openssh.com/txt/release-8.4
2020-10-22 16:42:30 -07:00
Anders Kaseorg 99e6ec4190 favicon: Slightly increase the font size for short numbers.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-22 15:43:09 -07:00
sahil839 571bb62e3d events: Update subscriber list on peer_add for unsubscribed streams.
We update the subscriber list on peer_add event for unsubscribed
streams as well.
2020-10-22 15:12:32 -07:00
sahil839 733d26aef2 events: Update subscriber list on peer_remove for never subscribed stream.
We now update the subscriber list on peer_remove event for never
subscribed streams also.
2020-10-22 15:12:32 -07:00
sahil839 af9b153ee3 events: Update subscriber list on peer_remove for unsubscribed stream.
We update the subscriber list on peer_remove event for unsubscribed
streams also.
2020-10-22 15:12:32 -07:00
sahil839 709edd29d4 test_events: Fix comment in do_test_subscribe_events.
The comment still pointed to 'vacate' event flow, but
we have removed the vacate event in a9356508ca.
This commit fixes the comment to depict the correct
purpose of below lines, i.e. to test the remove
event flow.
2020-10-22 15:12:32 -07:00
sahil839 e578742b02 test_events: Remove 'realm_user' from event_types in subscription test.
We were including 'realm_user' in event_types along with 'subscription',
but we don't send event of type 'realm_user' when subscribing to a new
stream. This was added in 1c332f5d6a.

This commit removes 'realm_user' from event_types.
2020-10-22 15:12:32 -07:00
sahil839 465122b0df compose: Show error for wildcard messages according to settings.
This commit changes UI to show /error for sending message with
wildcard mention according to wildcard mention policy of organization.

Fixes #16211.
2020-10-22 15:00:27 -07:00
sahil839 1d5aa2e514 settings_org: Add frontend to change wildcard_mention_policy.
We add dropdown for wildcard_mention_policy in organization
permissions page.
2020-10-22 15:00:27 -07:00
sahil839 d0f5537fb2 actions: Modify check_message for handling wildcard_mention_policy setting.
This commit adds enforcement for sending messages containing wildcard
mentions according to wildcard_mention_policy.
2020-10-22 14:46:32 -07:00
sahil839 25f32d461e tests: Add tests for all the values of wildcard_mention_policy. 2020-10-22 12:08:22 -07:00
Mateusz Mandera ec8fcf3a04 docs: Fix duplication in apple-error.md config error page.
SOCIAL_AUTH_APPLE_TEAM was repeated twice.
2020-10-22 12:07:46 -07:00
Mateusz Mandera 48f80fcb0a auth: Expect name in request params in Apple auth.
The name used to be included in the id_token, but this seems to have
been changed by Apple and now it's sent in the `user` request param.

https://github.com/python-social-auth/social-core/pull/483 is the
upstream PR for this - but upstream is currently unmaintained, so we
have to monkey patch.

We also alter the tests to reflect this situation. Tests no longer put
the name in the id_token, but rather in the `user` request param in the
browser flow, just like it happens in reality.

An adaptation has to be made in the native flow - since the name won't
be included by Apple in the id_token anymore, the app, when POSTing
to the /complete/apple/ endpoint,
can (and should for better user experience)
add the `user` param formatted as json of
{"email": "hamlet@zulip.com", "name": {"firstName": "Full", "lastName": "Name"}}
dict. This is also reflected by the change in the
native flow tests.
2020-10-22 12:07:46 -07:00
Alex Vandiver 0ea20bd7d8 puppet: Move postgres_version into postgres_common.
This property is not related to the base zulip install; move it to
zulip::postgres_common, which is already used as a namespace for
various postgres variables.
2020-10-22 11:32:25 -07:00
Alex Vandiver 25e995b677 puppet: Move normal_queues to the one place that uses it. 2020-10-22 11:32:25 -07:00
Alex Vandiver 423b5c2be2 puppet: Move queue error and stats directories to just the app host. 2020-10-22 11:31:05 -07:00
Alex Vandiver 4d4c21499a puppet: Move supervisor dependency into process_fts_updates.
PostgreSQL itself has no dependency on supervisor; rather, the FTS
updates do.
2020-10-22 11:30:53 -07:00
Alex Vandiver ca971ebc59 puppet: Remove empty zulip_ops class. 2020-10-22 11:30:53 -07:00