Commit Graph

9425 Commits

Author SHA1 Message Date
Rishi Gupta e40731574a integrations: Fix missing logo for trello.
Fixes regression from 6f8481c.
2019-02-19 15:41:18 -08:00
synicalsyntax 2a5fe09d16 integrations: Update Opsgenie logo.
Converts logo from PNG to SVG format.
2019-02-19 15:07:24 -08:00
synicalsyntax 1595162496 integrations: Update Jira logo.
Converts logo from PNG to SVG format.
2019-02-19 15:07:24 -08:00
synicalsyntax 5bf89b946d integrations: Update Bitbucket logo.
Converts logo from PNG to SVG format.
2019-02-19 15:07:24 -08:00
synicalsyntax 6f8481cd33 integrations: Update Trello logo.
Converts logo from PNG to SVG format.
2019-02-19 15:07:24 -08:00
Raymond Akornor 4dc7f5354d tests: Optimize test-backend performance.
This optimizes test-backend by skipping webhook
tests when run in default mode.

Tweaked by tabbott to extend the documentation and update the CI
commands.
2019-02-19 14:50:41 -08:00
Eeshan Garg c78c3f423c webhooks/github: Ignore project_card events.
The payloads for this event are missing some important details
about the Project's changes, such as the name of the project,
the card's column name, etc. Without such details, the resultant
notifications would not be useful at all!
2019-02-19 14:25:38 -08:00
Eeshan Garg ec81410b03 webhooks/github: Ignore repository_vulnerability_alert event.
This event isn't incredibly common/useful and errors for this
event were cluttering up our webhook logs.
2019-02-19 18:01:05 -03:30
sahil839 7157edf4af settings: Add support for uploading logo for night mode.
This adds a new field named realm_night_logo which is used for
displaying the organization logo when the user is in night mode.

Fixes #11176.
2019-02-18 15:15:57 -08:00
Eeshan Garg 0500639d01 api_tests: Test zulip.Client.add_reaction. 2019-02-18 10:24:13 -08:00
Tim Abbott 7a49611a94 settings_invites: Display links for multi-use invitations.
This makes it possible to figure out which is which if you have
several active at the same time.
2019-02-15 14:28:07 -08:00
Vishnu Ks 763eca6ca9 invites: Add UI for revoking multiuse invites. 2019-02-15 14:13:31 -08:00
Vishnu Ks 410e2574d1 settings: Remove unused setting ACCOUNT_ACTIVATION_DAYS.
ACCOUNT_ACTIVATION_DAYS doesn't seems to be used anywhere.
INVITATION_LINK_VALIDITY_DAYS seems to do it's job currently.

(It was only ever used in very early Zulip commits).
2019-02-15 13:56:10 -08:00
Tim Abbott 0c0aec3cc9 export: Fix finding manage.py to export usermessages.
We were using a hardcoded relative path, which doesn't work if you're
not running this from the root of the Zulip checkout.

As part of fixing this, we need to make `LOCAL_UPLOADS_DIR` an
absolute path.

Fixes #11581.
2019-02-15 11:32:36 -08:00
Anders Kaseorg ce01a4c5a3 backup: Add Zulip, OS, and PostgreSQL version to the tarball.
==> zulip-backup/os-version <==
    Ubuntu trusty

    ==> zulip-backup/postgres-version <==
    90324

    ==> zulip-backup/zulip-version <==
    2.0.0-rc1+git
    2.0.0-rc1-50-gbf169b8-dirty

Documentation added by tabbott.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-14 15:11:57 -08:00
Greg Price 0213aa0b16 push notif: Don't forget to clear "active" flag on sending to bouncer.
Since da8f4bc0e back in August, this control flow has caused
`flags.active_mobile_push_notification` to be cleared if we don't send
these `remove` messages at all, and if we send them directly to GCM...
but not if we send them via the Zulip notification bouncer.

As a result, on a server configured to send `remove` notification-messages
via the bouncer, we accumulate "active" messages and never clear them.

If the user then does `mark_all_as_read`, we end up sending a `remove`
for each of those messages again, and all in one giant burst.  We've
seen puzzling bursts of hundreds of removals pass through the bouncer
since turning on removals on chat.zulip.org; it's likely many of them
are caused by this bug.

This issue was made more acute with f4478aad5, which unconditionally
enabled removals.

Test added by tabbott.
2019-02-14 14:52:53 -08:00
Vishnu Ks 949d098e99 management: Add tooling for transfering local uploads to s3.
This solves a common migration problem for folks who cut corners when
first setting up Zulip.

Fixes #11294.
2019-02-14 14:31:46 -08:00
Vishnu Ks 6d138bd3e5 tests: Fix broken avatar_disk_path for original size input.
string.replace returns the new string and does
not change the original string.
2019-02-14 14:18:03 -08:00
Rishi Gupta e183c316dd help: Rename help/change-your-avatar to help/set-your-avatar. 2019-02-13 17:50:39 -08:00
Abdelhadi Dyouri 4ac2db56f8 export: Correctly treat emoji author field as optional.
While we likely will eventually want to make every custom emoji have
an author, that's not the data model today.

Fixes #11518.
2019-02-13 16:12:06 -08:00
Greg Price 630481cb7a push notif: Switch from GCM to FCM endpoint.
This is the only server-side change required for the FCM migration!

Optionally, at some point in the future we might choose to migrate
to the new ("v1") API which FCM also offers.  Nothing revolutionary
but there are some nice things about it:
  https://firebase.google.com/docs/cloud-messaging/migrate-v1
2019-02-13 13:57:57 -08:00
Greg Price 84e0b68b16 push notif: Rename `gcm` to less confusing `gcm_client`.
This opens up space in the namespace for, say, the library
module itself.
2019-02-13 13:57:57 -08:00
Tim Abbott 7099d01641 email mirror: Fix missing variable for logging. 2019-02-13 13:16:55 -08:00
Greg Price f4478aad54 push notif: Unconditionally remove notifications on message read.
The client-side fix to make these not a problem was in release
16.2.96, of 2018-08-22.  We've been sending them from the
development community server chat.zulip.org since 2018-11-29.
We started forcing clients to upgrade with commit fb7bfbe9a,
deployed 2018-12-05 to zulipchat.com.

(The mobile app unconditionally makes a request to a route on
zulipchat.com to check for this kind of forced upgrade, so that
applies to mobile users of any Zulip server.)

So at this point it's long past safe for us to unconditionally
send these.  Hardwire the old `SEND_REMOVE_PUSH_NOTIFICATIONS`
setting to True, and simplify it out.
2019-02-13 13:13:45 -08:00
savish 1b334b906a tests: Use subTest in test_admin_user_can_change_profile_data.
Fixes #11070.
2019-02-13 10:50:39 -08:00
savish aa605468b6 tests: Use subTest in test_update_realm_properties. 2019-02-13 10:49:13 -08:00
savish 9c292fbeab tests: Use subTest in test_bugdown_fixtures. 2019-02-13 10:49:13 -08:00
Rishi Gupta 378d14af7e import: Fix tests related to plan_type.
Broken in 4d08461.
2019-02-12 18:11:31 -08:00
Tim Abbott 12d5e870c5 tests: Fix import test failure.
Broken in 4d08461ab1.
2019-02-12 17:46:55 -08:00
Tim Abbott 4d08461ab1 import: Set plan_type to SELF_HOSTED on import.
We've for a while had logic to set plan_type to LIMITED when importing
into Zulip Cloud; we need corresponding logic to set it to SELF_HOSTED
when importing into a self-hosted server.

Fixes #11541.
2019-02-12 16:01:02 -08:00
Vishnu Ks 868a763cec auth2: Don't use session for passing multiuse invite key.
For Google auth, the multiuse invite key should be stored in the
csrf_state sent to google along with other values like is_signup,
mobile_flow_otp.

For social auth, the multiuse invite key should be passed as params to
the social-auth backend. The passing of the key is handled by
social_auth pipeline and made available to us when the auth is
completed.
2019-02-12 15:51:11 -08:00
Eeshan Garg 179b747769 streams: Refactor multi-option helpers into separate functions.
For internal stream messages, most of the time, we have access to
a Stream object. For the few corner cases where we don't, it is a
much cleaner approach to have a separate function that accepts a
stream name than having one multi-option helper that accepts both
names and objects.
2019-02-12 11:10:26 -08:00
Eeshan Garg 1d718adce4 zerver/views: Rename integrations.py to documentation.py.
The name "integrations" is misleading because we now also use
this module for our REST API docs.
2019-02-11 21:25:14 -08:00
Eeshan Garg b0c761dfb1 zapier_app: Include bot details in response to auth requests.
The bot's details (such as ID and name) are used to format a
connection label in Zapier's UI. This allows users to distinguish
between different Zapier-to-Zulip connections set up with different
bots.
2019-02-11 21:24:52 -08:00
Tim Abbott d6140b684f notifications: Don't send error emails on bouncer 500s.
Since the individual server administrator can't do anything about
these, this should not trigger an email notification.
2019-02-11 21:19:28 -08:00
Anders Kaseorg e0a51948d9 script: Add ready-to-run tooling for doing backups.
Based on an initial version by Tim Abbott (#11204).

Fixes #552.
2019-02-11 17:30:37 -08:00
Rishi Gupta 028874bab3 open graph: Remove extraneous spaces from descriptions.
Our html collects extra spaces in a couple of places. The most prominent is
paragraphs that look like the following in the .md file:
* some text
  continued

The html will have two spaces before "continued".
2019-02-11 12:05:19 -08:00
Rishi Gupta 0119b383a6 help: Update Accounts basics section to conform to new tabbed style. 2019-02-11 12:05:19 -08:00
Rishi Gupta d3125f59e1 open graph: Omit .code-section navigation from open graph. 2019-02-11 12:05:19 -08:00
Rishi Gupta e1f02dc6f2 open graph: Include multiple paragraphs in description tags. 2019-02-11 12:05:19 -08:00
Rishi Gupta 83236dc283 help: Add tab styling to untabbed instructions as well.
This changes the border-radius to 6px for the tabbed display, which is not
in line with the current Zulip style for border-radius (4px). However 6px
really looks a lot better for this (possibly because it's a bigger box than
most of our other boxes?)
2019-02-11 12:05:19 -08:00
anhhuynh bd548eb7f3 emails: Remove newlines from translated strings in email templates.
Fixes #11390.
2019-02-10 12:28:17 -08:00
Greg Price a6c2c16666 push notif tests: Clean up how we set up device token fixtures.
I was hoping this would make things faster... it does, but sadly only
by about 70ms, 5% of this file's test runtime.

It sure does make this file rather less action-at-a-distance, though,
as well as fixing some duplication.
2019-02-08 15:18:12 -08:00
Greg Price ccc1f3cd85 push notif tests: Stop importing module as bizarre name `apn`. 2019-02-08 15:18:12 -08:00
Greg Price c372ceb7a2 push notif tests: Use mock.patch instead of `apn.gcm = ...`. 2019-02-08 15:18:12 -08:00
Greg Price e289c5835d push notif tests: Use mock.patch instead of `apn.gcm = None`.
This is what it's for -- and it cleans up after itself, too.
2019-02-08 15:18:12 -08:00
Greg Price ff12072f18 push notif: Skip four layers of setup for some pure unit tests.
This saves about 50ms, for just four test cases.
2019-02-08 15:18:12 -08:00
Greg Price 1e11e929ec push notif: Guess a GCM `priority` on behalf of old servers. 2019-02-08 15:18:12 -08:00
Greg Price a293aeee23 push notif: Explicitly set GCM priority `normal` for remove.
If we make a practice on the Zulip server of always explicitly setting
the desired priority, then when an old server doesn't set the priority
we can reasonably have the bouncer make a guess.
2019-02-08 15:18:12 -08:00
Greg Price ffabebd7f3 push notif: Set GCM priority `high` for real notifications.
This is the payoff of this branch!  Fixes zulip/zulip-mobile#3185.
2019-02-08 15:18:12 -08:00
Greg Price 699bf262ca push notif: Test GCM options parsing more comprehensively.
The payoff from making these into real unit tests.
2019-02-08 15:18:12 -08:00
Greg Price 02ba302009 push notif: Simplify tests for parsing GCM options.
Huzzah for unit tests!
2019-02-08 15:18:12 -08:00
Greg Price f1cb9be79c push notif: Consolidate and simplify GCM tests. 2019-02-08 15:18:12 -08:00
Greg Price 6da9aeee7d push notif: Expand and clarify docs on GCM options. 2019-02-08 09:44:00 -08:00
Greg Price 674b254b65 push notif: Accept GCM `priority` option.
That is, this allows a Zulip server to now set the `priority`; but if
it doesn't, we use upstream's default value, which has the same effect
as we've always previously had by not setting it at all.

But when this is deployed to the push notifications bouncer server, it
does allow another server to set priority when pushing notifications
through the bouncer.
2019-02-08 09:42:59 -08:00
Greg Price 575f320009 push notif: Factor out a helper parse_gcm_options.
Looks a bit silly as is, but this will give us a good home for
logic to start parsing more than zero allowed options.
2019-02-08 09:41:07 -08:00
Greg Price 9ebad5d505 push notif: Reorg `send_android_...` slightly for clarity.
In particular, get the `remote`/`DeviceTokenClass` bit out from
in between `json_request` and the various bits of code preparing
for it.
2019-02-08 09:40:45 -08:00
Greg Price 49fd2e65de push notif: Add GCM options to bouncer API; empty for now.
The first use case for this will be setting `priority`,
coming up shortly.
2019-02-08 09:40:43 -08:00
Eeshan Garg 3470e541c8 internal_send_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with re-fetching the Stream object by name.
2019-02-08 08:59:03 -08:00
Eeshan Garg c240008edb internal_prep_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with getting a Stream object back.
2019-02-08 08:53:16 -08:00
Tim Abbott 4826393ec4 hipchat: Fix help documentation for slim_mode. 2019-02-07 16:17:01 -08:00
Vishnu Ks 28769e040f invites: Add backend for multiuse admin invites. 2019-02-07 15:41:00 -08:00
Eeshan Garg 4da28970fa addressee: Rename for_stream to for_stream_name.
The name for_stream_name is more appropriate here.  The name
for_stream is more suitable for a function that takes in a Stream
object, which we're about to add.
2019-02-07 13:41:24 -08:00
Tim Abbott 970c7d1ddf notifications: Fix hash-named duplicate sound files in options.
Our hash-naming of production assets interacted badly with the "look
at files in a directory" algorithm used to determine what sound
options exist for the "notification sound" feature.  For lack of a
better solution, we fix this by excluding files with an extra `.` in
their name.
2019-02-07 13:36:41 -08:00
Eeshan Garg 361d2d0688 webhooks/stripe: Add fixture and test for invoiceitem.created. 2019-02-05 18:40:38 -03:30
Tim Abbott 11ea8ae9fe tests: Fix test broken by recent url preview change. 2019-02-05 13:45:00 -08:00
Tim Abbott a4b294da98 url preview: Remove useless logging.error in open graph code path.
As detailed in the comment, someone pasting a broken URL isn't a
situation that a server administrator needs to be notified about.
2019-02-05 13:25:47 -08:00
Eeshan Garg 20769f0fe9 zapier_app: Add support for private/huddle messages. 2019-02-05 17:25:58 -03:30
Rishi Gupta 36472413e4 hotspots: Add hotspot for gear menu. 2019-02-05 12:19:21 -08:00
Harshit Bansal ae3881b472 tests: Extract `SocialAuthBase` class.
Extracts out common tests so that future social-auth backends can
be tested without duplicating tests. I have been careful to not
change any testing logic.
2019-02-05 11:39:31 -08:00
Harshit Bansal 1a5e07e0f9 ldap: Add ability to automatically sync custom profile fields. 2019-02-05 10:25:50 -08:00
YashRE42 22e3955262 search: Add stop words to page_params.
Add all the stop words to page_params, reading from the
`zulip_english.stop` database, with caching to avoid loading the file
on every page load.

Part of #10592.
2019-02-05 10:23:28 -08:00
Tim Abbott 47c85fa02e email: Set email based on realm email_address_visibility.
This causes changing the email_address_visibility field to actually
modify what user_profile.email values are generated for users, both on
user creation and afterwards as email addresses are edited.

The overall feature isn't yet complete, but this brings us pretty close.
2019-02-04 18:45:10 -08:00
Rishi Gupta 71958fc1d8 Revert "webhooks/stripe: Add support for remaining payout events."
These ended up more noisy than helpful.

This reverts commit d4cd789213.
2019-02-04 12:34:32 -08:00
Rishi Gupta 942e419b85 Revert "webhooks/stripe: Handle invoice events explicitly."
The original commit made a number of well-meaning but suboptimal product
changes to the invoice events, such as threading them under the invoice id
rather than the customer id. It also seems to be causing 500s for
invoiceitem events, though I'm not sure why.

This reverts commit 65489b0391.
2019-02-04 12:18:47 -08:00
Rohitt Vashishtha 19672241e6 markdown: Disable definition/reference links in marked.
We had disabled reference style links in bugdown, however,
we hadn't disabled them in marked. This commit rectifies
that and adds test cases for the same.

Fixes #11350.
2019-02-04 11:16:37 -08:00
Wyatt Hoodes 9c68a97472 import/export: Use separate analytics.json for analytics data.
This helps keep the realm.json small and easy to process; previously,
almost the entire size of that file was the analytics data.

We implement this by refactoring the analytics Config objects into a
separate subroutine that writes to a separate file, plus the
corresponding import code.

Manual testing was performed by exporting the 'analytics' realm, and
importing back to a newly created 'test' realm.  The 'test' realm was
then exported and the json files were inspected.  The data appeared
consistent with no abnormalities.

Fixes: #11220.
2019-02-04 10:59:24 -08:00
Tim Abbott f34a5d6828 signals: Document the weird unused import for signal registration. 2019-02-02 23:04:02 -08:00
Anders Kaseorg 3127fb4dbd zerver/tests: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:43:03 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Anders Kaseorg 3968fe9fdf tornado: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:33:13 -08:00
Anders Kaseorg 4e0a79d146 test_messages: Don’t mock an unused import.
Since 8a1794caa3, we don't do any
caching in that system.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:29:53 -08:00
Anders Kaseorg 023f8a0def test_outgoing_webhook_interfaces: Don’t mock an unused import.
We eliminated use of this function in outgoing_webhook.py in
bdc95b5d72.

Tweaked by tabbott to also eliminate code only used for that mock.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:28:11 -08:00
Anders Kaseorg 56a675d5ec export: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:25:27 -08:00
Anders Kaseorg e12c433745 bugdown: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:25:22 -08:00
Anders Kaseorg 4e21cc0152 views: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:23:43 -08:00
Anders Kaseorg 2a2e744b27 management: Remove unused imports in management commands.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:12:36 -08:00
Anders Kaseorg a0041afaf9 python: Stop importing Length from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:09:10 -08:00
Anders Kaseorg b80a095196 python: Stop importing PushNotificationBouncerException from wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:09:01 -08:00
Anders Kaseorg 1a3c195f7e python: Stop importing TestCase from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:08:59 -08:00
Anders Kaseorg e8b32a4b67 python: Stop importing get_display_recipient from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:08:57 -08:00
Anders Kaseorg 39ac378220 webhooks: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:05:20 -08:00
Anders Kaseorg 4bd28f7ae6 migrations: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:01:04 -08:00
Tim Abbott 96f096f38e analytics: Add section for remote Zulip servers. 2019-02-02 16:55:12 -08:00
Tim Abbott a6d3bbfc63 zilencer: Add client-size rate limiting of analytics upload.
This should help both by avoiding high memory usage causing OOM kills
on the client, as well as timeouts causing an exception email to be
sent.
2019-02-02 11:51:22 -08:00
Eeshan Garg 954a98b49f webhooks/jira: Ignore issuelink_created events.
Notifications for this event aren't very useful to begin with,
plus exceptions from this event are cluttering up our webhook
error logs.
2019-02-02 14:53:15 -03:30
Hemanth V. Alluri 73d26c8b28 streams: Render and store the stream description from the backend.
This commit does the following three things:
    1. Update stream model to accomodate rendered description.
    2. Render and save the stream rendered description on update.
    3. Render and save stream descriptions on creation.

Further, the stream's rendered description is also sent whenever the
stream's description is being sent.

This is preparatory work for eliminating the use of the
non-authoritative marked.js markdown parser for stream descriptions.
2019-02-01 22:24:18 -08:00
Tim Abbott 022c8beaf5 analytics: Add APIs for submitting analytics to another server.
This adds a new API for sending basic analytics data (number of users,
number of messages sent) from a Zulip server to the Zulip Cloud
central analytics database, which will make it possible for servers to
elect to have their usage numbers counted in published stats on the
size of the Zulip ecosystem.
2019-02-01 22:03:52 -08:00
Rishi Gupta 8df0d95559 billing: Remove deprecated RealmAuditLog.requires_billing_update. 2019-02-01 18:49:43 -08:00
Rishi Gupta 85f7ac8172 analytics: Remove Anomaly model. 2019-02-01 18:48:18 -08:00