Commit Graph

4100 Commits

Author SHA1 Message Date
Raymond Akornor 4dc7f5354d tests: Optimize test-backend performance.
This optimizes test-backend by skipping webhook
tests when run in default mode.

Tweaked by tabbott to extend the documentation and update the CI
commands.
2019-02-19 14:50:41 -08:00
sahil839 7157edf4af settings: Add support for uploading logo for night mode.
This adds a new field named realm_night_logo which is used for
displaying the organization logo when the user is in night mode.

Fixes #11176.
2019-02-18 15:15:57 -08:00
Eeshan Garg 0500639d01 api_tests: Test zulip.Client.add_reaction. 2019-02-18 10:24:13 -08:00
Tim Abbott 7a49611a94 settings_invites: Display links for multi-use invitations.
This makes it possible to figure out which is which if you have
several active at the same time.
2019-02-15 14:28:07 -08:00
Vishnu Ks 763eca6ca9 invites: Add UI for revoking multiuse invites. 2019-02-15 14:13:31 -08:00
Vishnu Ks 410e2574d1 settings: Remove unused setting ACCOUNT_ACTIVATION_DAYS.
ACCOUNT_ACTIVATION_DAYS doesn't seems to be used anywhere.
INVITATION_LINK_VALIDITY_DAYS seems to do it's job currently.

(It was only ever used in very early Zulip commits).
2019-02-15 13:56:10 -08:00
Tim Abbott 0c0aec3cc9 export: Fix finding manage.py to export usermessages.
We were using a hardcoded relative path, which doesn't work if you're
not running this from the root of the Zulip checkout.

As part of fixing this, we need to make `LOCAL_UPLOADS_DIR` an
absolute path.

Fixes #11581.
2019-02-15 11:32:36 -08:00
Greg Price 0213aa0b16 push notif: Don't forget to clear "active" flag on sending to bouncer.
Since da8f4bc0e back in August, this control flow has caused
`flags.active_mobile_push_notification` to be cleared if we don't send
these `remove` messages at all, and if we send them directly to GCM...
but not if we send them via the Zulip notification bouncer.

As a result, on a server configured to send `remove` notification-messages
via the bouncer, we accumulate "active" messages and never clear them.

If the user then does `mark_all_as_read`, we end up sending a `remove`
for each of those messages again, and all in one giant burst.  We've
seen puzzling bursts of hundreds of removals pass through the bouncer
since turning on removals on chat.zulip.org; it's likely many of them
are caused by this bug.

This issue was made more acute with f4478aad5, which unconditionally
enabled removals.

Test added by tabbott.
2019-02-14 14:52:53 -08:00
Vishnu Ks 949d098e99 management: Add tooling for transfering local uploads to s3.
This solves a common migration problem for folks who cut corners when
first setting up Zulip.

Fixes #11294.
2019-02-14 14:31:46 -08:00
Vishnu Ks 6d138bd3e5 tests: Fix broken avatar_disk_path for original size input.
string.replace returns the new string and does
not change the original string.
2019-02-14 14:18:03 -08:00
Abdelhadi Dyouri 4ac2db56f8 export: Correctly treat emoji author field as optional.
While we likely will eventually want to make every custom emoji have
an author, that's not the data model today.

Fixes #11518.
2019-02-13 16:12:06 -08:00
Greg Price 630481cb7a push notif: Switch from GCM to FCM endpoint.
This is the only server-side change required for the FCM migration!

Optionally, at some point in the future we might choose to migrate
to the new ("v1") API which FCM also offers.  Nothing revolutionary
but there are some nice things about it:
  https://firebase.google.com/docs/cloud-messaging/migrate-v1
2019-02-13 13:57:57 -08:00
Greg Price 84e0b68b16 push notif: Rename `gcm` to less confusing `gcm_client`.
This opens up space in the namespace for, say, the library
module itself.
2019-02-13 13:57:57 -08:00
Tim Abbott 7099d01641 email mirror: Fix missing variable for logging. 2019-02-13 13:16:55 -08:00
Greg Price f4478aad54 push notif: Unconditionally remove notifications on message read.
The client-side fix to make these not a problem was in release
16.2.96, of 2018-08-22.  We've been sending them from the
development community server chat.zulip.org since 2018-11-29.
We started forcing clients to upgrade with commit fb7bfbe9a,
deployed 2018-12-05 to zulipchat.com.

(The mobile app unconditionally makes a request to a route on
zulipchat.com to check for this kind of forced upgrade, so that
applies to mobile users of any Zulip server.)

So at this point it's long past safe for us to unconditionally
send these.  Hardwire the old `SEND_REMOVE_PUSH_NOTIFICATIONS`
setting to True, and simplify it out.
2019-02-13 13:13:45 -08:00
Tim Abbott 4d08461ab1 import: Set plan_type to SELF_HOSTED on import.
We've for a while had logic to set plan_type to LIMITED when importing
into Zulip Cloud; we need corresponding logic to set it to SELF_HOSTED
when importing into a self-hosted server.

Fixes #11541.
2019-02-12 16:01:02 -08:00
Eeshan Garg 179b747769 streams: Refactor multi-option helpers into separate functions.
For internal stream messages, most of the time, we have access to
a Stream object. For the few corner cases where we don't, it is a
much cleaner approach to have a separate function that accepts a
stream name than having one multi-option helper that accepts both
names and objects.
2019-02-12 11:10:26 -08:00
Tim Abbott d6140b684f notifications: Don't send error emails on bouncer 500s.
Since the individual server administrator can't do anything about
these, this should not trigger an email notification.
2019-02-11 21:19:28 -08:00
Rishi Gupta 83236dc283 help: Add tab styling to untabbed instructions as well.
This changes the border-radius to 6px for the tabbed display, which is not
in line with the current Zulip style for border-radius (4px). However 6px
really looks a lot better for this (possibly because it's a bigger box than
most of our other boxes?)
2019-02-11 12:05:19 -08:00
Greg Price 1e11e929ec push notif: Guess a GCM `priority` on behalf of old servers. 2019-02-08 15:18:12 -08:00
Greg Price a293aeee23 push notif: Explicitly set GCM priority `normal` for remove.
If we make a practice on the Zulip server of always explicitly setting
the desired priority, then when an old server doesn't set the priority
we can reasonably have the bouncer make a guess.
2019-02-08 15:18:12 -08:00
Greg Price ffabebd7f3 push notif: Set GCM priority `high` for real notifications.
This is the payoff of this branch!  Fixes zulip/zulip-mobile#3185.
2019-02-08 15:18:12 -08:00
Greg Price 6da9aeee7d push notif: Expand and clarify docs on GCM options. 2019-02-08 09:44:00 -08:00
Greg Price 674b254b65 push notif: Accept GCM `priority` option.
That is, this allows a Zulip server to now set the `priority`; but if
it doesn't, we use upstream's default value, which has the same effect
as we've always previously had by not setting it at all.

But when this is deployed to the push notifications bouncer server, it
does allow another server to set priority when pushing notifications
through the bouncer.
2019-02-08 09:42:59 -08:00
Greg Price 575f320009 push notif: Factor out a helper parse_gcm_options.
Looks a bit silly as is, but this will give us a good home for
logic to start parsing more than zero allowed options.
2019-02-08 09:41:07 -08:00
Greg Price 9ebad5d505 push notif: Reorg `send_android_...` slightly for clarity.
In particular, get the `remote`/`DeviceTokenClass` bit out from
in between `json_request` and the various bits of code preparing
for it.
2019-02-08 09:40:45 -08:00
Greg Price 49fd2e65de push notif: Add GCM options to bouncer API; empty for now.
The first use case for this will be setting `priority`,
coming up shortly.
2019-02-08 09:40:43 -08:00
Eeshan Garg 3470e541c8 internal_send_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with re-fetching the Stream object by name.
2019-02-08 08:59:03 -08:00
Eeshan Garg c240008edb internal_prep_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with getting a Stream object back.
2019-02-08 08:53:16 -08:00
Vishnu Ks 28769e040f invites: Add backend for multiuse admin invites. 2019-02-07 15:41:00 -08:00
Eeshan Garg 4da28970fa addressee: Rename for_stream to for_stream_name.
The name for_stream_name is more appropriate here.  The name
for_stream is more suitable for a function that takes in a Stream
object, which we're about to add.
2019-02-07 13:41:24 -08:00
Tim Abbott 970c7d1ddf notifications: Fix hash-named duplicate sound files in options.
Our hash-naming of production assets interacted badly with the "look
at files in a directory" algorithm used to determine what sound
options exist for the "notification sound" feature.  For lack of a
better solution, we fix this by excluding files with an extra `.` in
their name.
2019-02-07 13:36:41 -08:00
Tim Abbott a4b294da98 url preview: Remove useless logging.error in open graph code path.
As detailed in the comment, someone pasting a broken URL isn't a
situation that a server administrator needs to be notified about.
2019-02-05 13:25:47 -08:00
Eeshan Garg 20769f0fe9 zapier_app: Add support for private/huddle messages. 2019-02-05 17:25:58 -03:30
Rishi Gupta 36472413e4 hotspots: Add hotspot for gear menu. 2019-02-05 12:19:21 -08:00
YashRE42 22e3955262 search: Add stop words to page_params.
Add all the stop words to page_params, reading from the
`zulip_english.stop` database, with caching to avoid loading the file
on every page load.

Part of #10592.
2019-02-05 10:23:28 -08:00
Tim Abbott 47c85fa02e email: Set email based on realm email_address_visibility.
This causes changing the email_address_visibility field to actually
modify what user_profile.email values are generated for users, both on
user creation and afterwards as email addresses are edited.

The overall feature isn't yet complete, but this brings us pretty close.
2019-02-04 18:45:10 -08:00
Wyatt Hoodes 9c68a97472 import/export: Use separate analytics.json for analytics data.
This helps keep the realm.json small and easy to process; previously,
almost the entire size of that file was the analytics data.

We implement this by refactoring the analytics Config objects into a
separate subroutine that writes to a separate file, plus the
corresponding import code.

Manual testing was performed by exporting the 'analytics' realm, and
importing back to a newly created 'test' realm.  The 'test' realm was
then exported and the json files were inspected.  The data appeared
consistent with no abnormalities.

Fixes: #11220.
2019-02-04 10:59:24 -08:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Anders Kaseorg 4e0a79d146 test_messages: Don’t mock an unused import.
Since 8a1794caa3, we don't do any
caching in that system.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:29:53 -08:00
Anders Kaseorg 023f8a0def test_outgoing_webhook_interfaces: Don’t mock an unused import.
We eliminated use of this function in outgoing_webhook.py in
bdc95b5d72.

Tweaked by tabbott to also eliminate code only used for that mock.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:28:11 -08:00
Anders Kaseorg 56a675d5ec export: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:25:27 -08:00
Anders Kaseorg e12c433745 bugdown: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:25:22 -08:00
Anders Kaseorg a0041afaf9 python: Stop importing Length from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:09:10 -08:00
Anders Kaseorg e8b32a4b67 python: Stop importing get_display_recipient from the wrong file.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:08:57 -08:00
Tim Abbott a6d3bbfc63 zilencer: Add client-size rate limiting of analytics upload.
This should help both by avoiding high memory usage causing OOM kills
on the client, as well as timeouts causing an exception email to be
sent.
2019-02-02 11:51:22 -08:00
Hemanth V. Alluri 73d26c8b28 streams: Render and store the stream description from the backend.
This commit does the following three things:
    1. Update stream model to accomodate rendered description.
    2. Render and save the stream rendered description on update.
    3. Render and save stream descriptions on creation.

Further, the stream's rendered description is also sent whenever the
stream's description is being sent.

This is preparatory work for eliminating the use of the
non-authoritative marked.js markdown parser for stream descriptions.
2019-02-01 22:24:18 -08:00
Tim Abbott 022c8beaf5 analytics: Add APIs for submitting analytics to another server.
This adds a new API for sending basic analytics data (number of users,
number of messages sent) from a Zulip server to the Zulip Cloud
central analytics database, which will make it possible for servers to
elect to have their usage numbers counted in published stats on the
size of the Zulip ecosystem.
2019-02-01 22:03:52 -08:00
Rishi Gupta 85f7ac8172 analytics: Remove Anomaly model. 2019-02-01 18:48:18 -08:00
Tim Abbott 1f4de8328d actions: Add a comment explaining historical messages in detail. 2019-02-01 18:41:03 -08:00