Commit Graph

28795 Commits

Author SHA1 Message Date
Steve Howell 50e3f85557 Extract topic.py library.
We start by including functions that do custom
queries for topic history.

The goal of this library is partly to quarantine
the legacy "subject" column on Message.
2018-11-07 10:03:53 -08:00
Rohitt Vashishtha 652477daea bugdown: Refactor name to mention_text in mention.py.
This commit renames some variables to make it clear that a mention's
text is not only the name of the user in all cases.
2018-11-07 10:02:53 -08:00
Marco Burstein 207d7a8ee6 compose: Use a placeholder when quoting and replying.
Use the placeholder `[Quoting…]` when quoting and replying before the
quote has been added to the message. Also, add tests to the
`compose_actions` Node tests for the new behavior.

Fix #10705.
2018-11-07 09:56:38 -08:00
Eeshan Garg f9d867e138 webhooks/jira: Test double-escaped stream names with spaces.
A recent change to check_send_webhook_message allows webhooks to
unescape stream names before sending a message. This commit adds
a test for the edge case where the webhook URL is escaped twice by
a third-party.
2018-11-07 09:55:48 -08:00
Tim Abbott e14a35b490 import: Don't assume a last_modified key is present.
This fixes an exception when importing uploaded file data from
Slack/HipChat.
2018-11-07 09:52:35 -08:00
Tim Abbott bf77158dbc docs: Remove links to long-obsolete /plans.html.
We were previously sending users to a redirect, and that redirect
recently broke.
2018-11-07 08:11:07 -08:00
Tim Abbott db51d40f4e release: Update version strings following 1.9.0 release. 2018-11-07 07:53:12 -08:00
Tim Abbott 06d39f87b7 Release Zulip Server 1.9.0. 2018-11-07 07:24:30 -08:00
Tim Abbott aba25a5ef6 i18n: Update translation data from transifex. 2018-11-07 07:22:47 -08:00
Tim Abbott f62050212b tornado: Fix supervisord configuration for multiple processes.
Apparently, we can use the process group naming style of having dashes
in the names without using the explicit nun_procs feature of
supervisord configuration.

The new configuration is perfectly satisfactory, so there's no real
reason to prefer the old approach.
2018-11-06 17:56:06 -08:00
Tim Abbott 60ed41081b scripts: Write to upgrade.log in upgrade-zulip-from-git.
This should simplify debugging when doing Git-based upgrades; this log
has long been useful for the main upgrade-zulip tool.
2018-11-06 17:56:01 -08:00
Tim Abbott 3f03dcdf5e nagios: Support multiple tornado processes.
This allows our Tornado monitoring to correctly report whether
multiple configured Tornado processes are running.

This setup isn't ideal, in that it can't detect cases where the wrong
set of Tornado processes are running, but it's nice and simple and
should catch most actual problems.
2018-11-06 16:50:03 -08:00
Tim Abbott 5abf4dee92 nagios: Add new host groups for Tornado processes.
We also move all the existing Tornado monitoring rules to the
singletornado_frontends rule.
2018-11-06 16:33:18 -08:00
Tim Abbott 5f3b79c9e7 nagios: Fix tab-based whitespace. 2018-11-06 16:30:29 -08:00
Tim Abbott 15519ee0bc requirements: Upgrade a few dependencies. 2018-11-06 16:28:02 -08:00
Eeshan Garg 006e47198c webhooks/jira: Decode stream name on our end.
Recently, one of our users reported that a JIRA webhook was not
able to send messages to a stream with a space character in its
name. Turns out that JIRA does something weird with webhook URLs,
such that escaped space characters (%20) are escaped again, so
that when the request gets to Zulip, the double escaped %20 is
evaluated as the literal characters `%20`, and not as a space.

We fix this by unescaping the stream name on our end before
sending the message forward!
2018-11-06 15:57:04 -08:00
Tim Abbott 6ef64fc752 narrow: Throw a clear error for invalid id: narrows.
This fixes an issue where searching for `id:` (with no operand) would
cause the server to 500, rather than returning a user-facing error.
2018-11-06 15:53:02 -08:00
Tim Abbott e8290ccd6d add_new_user: Fix mention of running "python manage.py".
This won't work on Python 3.
2018-11-06 13:28:18 -08:00
Tim Abbott 8b661f2f03 slack import: Correctly detect the commenting user.
Fixes #10772.
2018-11-06 13:14:23 -08:00
Tim Abbott 86b38b1de7 tornado: Ensure TORNADO_PROCESSES is an integer.
When reading something from get_config, the default return type is a
string, which mean that various comparisons didn't work correctly.
2018-11-06 13:08:28 -08:00
Tim Abbott 1bf385e35f import: Avoid sending a content-type of None to S3.
The previous logic was incorrect, in that if `content_type` was set to
None (which happens with Slack/HipChat export, among other things),
then we wouldn't run the `guess_type` logic to auto-detect the
Content-Type to send to S3.
2018-11-06 13:03:14 -08:00
Tim Abbott 81a4c846f4 hipchat: Set s3_path for exported emoji.
This fixes an issue where the import process would fail when importing
to a server using the S3 backend.
2018-11-06 13:02:04 -08:00
Yashashvi Dave 8c72adc106 subscribers: Hide subscribers of non-subscribed stream for guest user in UI.
Guest users can't access subscribers of any(public or private)
non-subscribed streams. Therefore, hide subscribers list
of all non-subscribed streams from guest users in UI.

Fixes #10749 (the previous parts were fixed already).
2018-11-06 12:57:19 -08:00
Eeshan Garg c6dd801525 docs: Fix a couple of broken links.
For the PHP link, I just found an equivalent link that detailed
some well-known PHP security vulnerabilities.

As for the now defunct RequestBin, another instance is hosted
by another domain, so we can recommend using that for now.
2018-11-06 12:54:36 -08:00
Tim Abbott 539e84e9a1 hipchat import: Stop setting last_modified=None.
The last_modified field is intended to support setting the
orig-last-modified field in the S3 backend when importing, basically
to keep track of this bit of pre-export data for debugging.  In the
event that it isn't available, the correct thing to do is not write
out an invalid `last_modified` field; we should just not write it out
at all.
2018-11-06 12:50:36 -08:00
Tim Abbott a4f5d825b7 Revert "upgrade-zulip-from-git: Support specifying tag or commit ID for refname."
This reverts commit 5ea7feee7e.

We had to revert this because it didn't work with non-master branch names.
2018-11-06 12:40:45 -08:00
sameerchoubey 996caddb11 webhooks/taiga: Stop using emojis in notifications.
This fixes the fact that these emoji were sometimes not displaying
properly (because of changes in the emoji names used in the codebase),
while also making this integration more standard (since it was the
only one with such an aggressive use of emoji).
2018-11-06 12:06:49 -08:00
Roger Souza f445a71272 hello: Fix responsiveness bug when page is at 800px. 2018-11-05 10:20:34 -08:00
Steve Howell a092bee6b3 import: Reduce memory usage for UserMessage ids.
The UserMessage table can be huge, so creating a
bunch of entries in `ID_MAP` can overflow memory.

We don't have any tables that depend on `UserMessage`,
and we don't send the 'id' fields from `zerver_usermessage`
to the database, so re-mapping them was just busy-work.
2018-11-05 10:18:01 -08:00
Tim Abbott 2636db64e4 Update translation data from transifex. 2018-11-02 17:35:21 -07:00
Tim Abbott e55a3b6cd1 version: Bump after 1.9.0-rc3 release. 2018-11-02 17:30:13 -07:00
Tim Abbott 27f6e9359d Release Zulip Server 1.9.0-rc3. 2018-11-02 17:19:38 -07:00
ruchit2801 01f634cea0 docs: Update virtualbox link to not pin a version.
Older versions of virtual box were giving installation error in new
MacOS Mojave.  While originally we considered changing the docs to
point to the current version 5.2.20, it seems better to just not pin a
version.

Significantly tweaked by tabbott.
2018-11-02 17:16:44 -07:00
Tim Abbott 40ff41e135 tornado: Fix populate_db failing to call send_event properly.
This isn't the right long-term fix; theoretically, send_event
shouldn't be doing anything with populate_db, but that's for later.
2018-11-02 17:07:21 -07:00
Jack Zhang 5902a573be push_notifications: Play 'default' sound for iOS notifications.
Fixes zulip/zulip-mobile#2651.

This was tested on an iPhone 7 running iOS 12.
2018-11-02 17:01:50 -07:00
Tim Abbott 0cac7e1cd3 tornado: Extract functions for Tornado queue names.
This moves all control for what queue to use for which realm in our
Tornado system to just the sharding.py file; no actual sharding is
done yet.
2018-11-02 17:00:10 -07:00
Tim Abbott 152c44b6d2 tornado: Extract function for specifying Tornado URI.
Since TORNADO_PROCESSES is 1 in all default configurations, this
doesn't have any user-facing effect.
2018-11-02 17:00:09 -07:00
Tim Abbott ec065e92ee tornado: Store port on SockJS connection object.
This will make it available for use inside our websockets code.
2018-11-02 16:55:33 -07:00
Tim Abbott 5e7aa27c29 puppet: Add supervisord support for multiple tornado processes. 2018-11-02 16:55:33 -07:00
Tim Abbott ea1ec68899 events: Pass a realm object into send_event.
This is a preparator refactor for supporting hosting different Tornado
processes on different servers; to look up which Tornado server we
should be sending the event to, we'll need the realm object.
2018-11-02 16:47:39 -07:00
Tim Abbott 75e48459b5 tornado: Support using a port-aware file for dumping event queues.
This should make it possible for there to safely be multiple Tornado
processes running on different ports on the same system.

It may also fix a rare race bug in development, where previously, it
was possible for the Tornados processes for Casper and the main
development server to interfere; I haven't investigated whether this
was a real bug or not, but now those two services will use independent
Tornado files.

We still need to add something to direct traffic between the different
Tornado processes.
2018-11-02 16:47:39 -07:00
Tim Abbott a5acbd51c3 settings: Add new zulip.conf setting for number of Tornado processes.
This will eventually be used to support Tornado sharding; for now,
it's just used to contain the code intended to support that feature.
2018-11-02 16:47:26 -07:00
Tim Abbott ca8e3a278a settings: Extract get_config function.
This is a close analogue to the existing get_secret.
2018-11-02 16:06:46 -07:00
Tim Abbott 0976e3bd39 purge_queue: Use more robust Tornado patterns.
We're about to make the list of possible Tornado queue names broader.
2018-11-02 16:06:46 -07:00
Tim Abbott 9339c191da tornado: Fix missing mypy annotation. 2018-11-02 14:43:55 -07:00
Tim Abbott cf24a20185 tornado: Refactor logic for persistent queue filenames.
Now, these are computed using a function, which will make it easier to
edit these paths to depend on which Tornado process it is in coming
commits.
2018-11-02 14:19:10 -07:00
Tim Abbott 98f28fa6ce tornado: Remove unused send_notification() function.
This hasn't been used in a long time, probably since
3fddc11cc2.
2018-11-02 14:14:39 -07:00
Tim Abbott e221a01638 Revert "Change if(realm.domain == mit.edu) to use Realm.is_zephyr_..."
This reverts commit 3645bb9225.

This change was incorrect, because the `is_zephyr_mirror_realm`
property on Realm is a property and thus isn't available in the
migration codebase.

Since this migration is only run for very old servers, this should
have no impact.
2018-11-02 13:16:45 -07:00
Tim Abbott c5b1406c54 requirements: Update disposable-email-domains. 2018-11-02 12:23:09 -07:00
Yashashvi Dave 02a5849d4c statistics: Guest user can't access realm statistics.
Don't allow guest user to access realm statistics from
UI or at API level.

Fixes part of #10749.
2018-11-02 11:43:09 -07:00