Commit Graph

9299 Commits

Author SHA1 Message Date
Wyatt Hoodes 9c68a97472 import/export: Use separate analytics.json for analytics data.
This helps keep the realm.json small and easy to process; previously,
almost the entire size of that file was the analytics data.

We implement this by refactoring the analytics Config objects into a
separate subroutine that writes to a separate file, plus the
corresponding import code.

Manual testing was performed by exporting the 'analytics' realm, and
importing back to a newly created 'test' realm.  The 'test' realm was
then exported and the json files were inspected.  The data appeared
consistent with no abnormalities.

Fixes: #11220.
2019-02-04 10:59:24 -08:00
Tim Abbott f34a5d6828 signals: Document the weird unused import for signal registration. 2019-02-02 23:04:02 -08:00
Anders Kaseorg 3127fb4dbd zerver/tests: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:43:03 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Anders Kaseorg 3968fe9fdf tornado: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:33:13 -08:00
Anders Kaseorg 4e0a79d146 test_messages: Don’t mock an unused import.
Since 8a1794caa3, we don't do any
caching in that system.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:29:53 -08:00
Anders Kaseorg 023f8a0def test_outgoing_webhook_interfaces: Don’t mock an unused import.
We eliminated use of this function in outgoing_webhook.py in
bdc95b5d72.

Tweaked by tabbott to also eliminate code only used for that mock.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:28:11 -08:00
Anders Kaseorg 56a675d5ec export: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:25:27 -08:00
Anders Kaseorg e12c433745 bugdown: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:25:22 -08:00
Anders Kaseorg 4e21cc0152 views: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:23:43 -08:00
Anders Kaseorg 2a2e744b27 management: Remove unused imports in management commands.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:12:36 -08:00
Anders Kaseorg a0041afaf9 python: Stop importing Length from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:09:10 -08:00
Anders Kaseorg b80a095196 python: Stop importing PushNotificationBouncerException from wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:09:01 -08:00
Anders Kaseorg 1a3c195f7e python: Stop importing TestCase from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:08:59 -08:00
Anders Kaseorg e8b32a4b67 python: Stop importing get_display_recipient from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:08:57 -08:00
Anders Kaseorg 39ac378220 webhooks: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:05:20 -08:00
Anders Kaseorg 4bd28f7ae6 migrations: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:01:04 -08:00
Tim Abbott 96f096f38e analytics: Add section for remote Zulip servers. 2019-02-02 16:55:12 -08:00
Tim Abbott a6d3bbfc63 zilencer: Add client-size rate limiting of analytics upload.
This should help both by avoiding high memory usage causing OOM kills
on the client, as well as timeouts causing an exception email to be
sent.
2019-02-02 11:51:22 -08:00
Eeshan Garg 954a98b49f webhooks/jira: Ignore issuelink_created events.
Notifications for this event aren't very useful to begin with,
plus exceptions from this event are cluttering up our webhook
error logs.
2019-02-02 14:53:15 -03:30
Hemanth V. Alluri 73d26c8b28 streams: Render and store the stream description from the backend.
This commit does the following three things:
    1. Update stream model to accomodate rendered description.
    2. Render and save the stream rendered description on update.
    3. Render and save stream descriptions on creation.

Further, the stream's rendered description is also sent whenever the
stream's description is being sent.

This is preparatory work for eliminating the use of the
non-authoritative marked.js markdown parser for stream descriptions.
2019-02-01 22:24:18 -08:00
Tim Abbott 022c8beaf5 analytics: Add APIs for submitting analytics to another server.
This adds a new API for sending basic analytics data (number of users,
number of messages sent) from a Zulip server to the Zulip Cloud
central analytics database, which will make it possible for servers to
elect to have their usage numbers counted in published stats on the
size of the Zulip ecosystem.
2019-02-01 22:03:52 -08:00
Rishi Gupta 8df0d95559 billing: Remove deprecated RealmAuditLog.requires_billing_update. 2019-02-01 18:49:43 -08:00
Rishi Gupta 85f7ac8172 analytics: Remove Anomaly model. 2019-02-01 18:48:18 -08:00
Tim Abbott 1f4de8328d actions: Add a comment explaining historical messages in detail. 2019-02-01 18:41:03 -08:00
kunal-mohta ac55a5222c messages: Add support for quickly deleting all messages in a topic.
This is primarily a feature for onboarding, where an organization
administrator might send a bunch of random test messages as part of
joining, but then want a pristine organization when their users later
join.

But it can theoretically be used for other use cases (e.g. for
moderation or removing threads that are problematic in some way).

Tweaked by tabbott to handle corner cases with
is_history_public_to_subscribers.

Fixes #10912.
2019-02-01 17:09:44 -08:00
sameerchoubey 65489b0391 webhooks/stripe: Handle invoice events explicitly.
Closes #10909
2019-02-01 21:38:20 -03:30
Eeshan Garg bffc473ec8 webhooks/zapier: Support stream messages for the Zapier Zulip app. 2019-02-01 21:19:26 -03:30
Eeshan Garg b1ce237624 webhooks/zapier: Support authentication for the Zapier Zulip app. 2019-02-01 21:19:26 -03:30
Eeshan Garg f9ee393e4f webhooks/common: Allow suppressing exceptions for missing headers.
This is useful in cases where an event type HTTP header isn't
crucial for inferring the payload's type.
2019-02-01 21:19:26 -03:30
Shubham Dhama 5779320874 invites: Use stream_id instead of stream names in invitation.
This replaces the current usage of stream names with stream ids.

This commit also removes the `traditional` attribute from the invite
form as now we are sending stream_ids as an argument; this was the
only place in the codebase we used traditional=true, and it's great to
have it removed.
2019-02-01 15:47:20 -08:00
Shoumorup e8ba08367b requirements: Upgrade pyflakes.
Pyflakes has been upgraded from 2.0.0 to 2.1.0 and
a few new linter errors have been fixed.

Fixes #11397.
2019-02-01 07:26:13 -08:00
Tim Abbott 1bcd7dfbad remote_server: Return the result dictionary on success. 2019-01-31 17:39:41 -08:00
Tim Abbott 2d11e163dd push_notifications: Move "push" part of URLs to callers.
This will make it possible for us to use this library for endpoints
not directly related to push notifications.
2019-01-31 15:22:00 -08:00
Rishi Gupta 4fab08172b billing: Remove deprecated billing fields from Realm. 2019-01-31 15:12:43 -08:00
Rishi Gupta 8f7e38c559 billing: Remove deprecated billing tables and fields from corporate/models. 2019-01-31 15:12:43 -08:00
Rishi Gupta 55179f913c billing: Remove references to Customer.has_billing_relationship. 2019-01-31 15:12:43 -08:00
Tim Abbott 88fae0b6a9 remote_server: Extract remote_server.py library.
This moves the network request code for connecting to the push
notification bouncer service into its own module.
2019-01-31 15:08:46 -08:00
Eeshan Garg f4221f2655 internal_prep_message: Don't call ensure_stream for stream IDs.
Ever since we implemented support for stream IDs in Addressee,
Addressee.stream_name() can now return None. This commit ensures
that _internal_prep_message only calls ensure_stream when
Addressee.stream_name() is not None.
2019-01-31 12:55:10 -08:00
Eeshan Garg 0420ecadeb addressee: Use stream ID for sender's default sending stream. 2019-01-31 12:43:38 -08:00
Eeshan Garg 998437c123 check_message: Support sending stream messages by ID.
This commit also contains the following auxiliary changes:
* Adds a custom exception, StreamWithIDDoesNotExist for when
  a stream with a given ID does not exist because the error
  message returned by StreamDoesNotExist only makes with stream
  names, not IDs.
* Adds a new helper, get_stream_by_id_in_realm, which is similar
  to get_user_profile_by_id_in_realm (introduced in #10391).
* Adds a helper, validate_stream_id_with_pm_notification, which
  returns the Stream object associated with a given ID and also
  handles PM notifications to the bot owner if the message was
  sent by a bot and if the stream does not exist or has no
  subscribers.
* Modifies the message sent by send_pm_if_empty_stream to
  accommodate stream IDs.

Note that all of the above changes are required before check_message
can be modified to support stream IDs.
2019-01-31 12:43:38 -08:00
Eeshan Garg b8221555d2 addressee: Add support for stream IDs to Addressee.legacy_build(). 2019-01-31 12:43:38 -08:00
Tim Abbott 9b25f8789f hipchat: Fix handling of user IDs in Stride import.
We've had this code oscillated a few times; the original comparison
was added as part of Stride import but broke HipChat import.
c34a8f2e69 fixed HipChat import but
regressed Stride.

This change fixes this for both HipChat + Stride.
2019-01-31 12:40:05 -08:00
Pragati Agrawal e1772b3b8f tools: Upgrade Pycodestyle and fix new linter errors.
Here, we are upgrading pycodestyle version from 2.4.0 to 2.5.0.

Fixes: #11396.
2019-01-31 12:21:41 -08:00
Vishnu Ks 0d0007742f requirements: Upgrade pika from 0.12.0 to 0.13.0.
The important changes to pika for us are based on this PR of ours:
https://github.com/pika/pika/pull/1129

Fixes #11394.
2019-01-31 10:04:07 -08:00
Harshit Bansal 837483b026 tests: Don't fail silently on 500 on logins. 2019-01-29 16:01:30 -08:00
Harshit Bansal 6777b94d41 lib: Extract `validate_user_custom_profile_field()`. 2019-01-29 16:01:30 -08:00
Harshit Bansal 9bac192241 dev_ldap: Add custom profile data. 2019-01-29 16:01:30 -08:00
Harshit Bansal 88c1d868d2 dev_ldap: Fix password in mode 'c' to be equal to username. 2019-01-29 16:01:30 -08:00
Harshit Bansal dccfadc1a5 minor: Fix some typing errors. 2019-01-29 16:01:30 -08:00