Commit Graph

1414 Commits

Author SHA1 Message Date
Eeshan Garg f4221f2655 internal_prep_message: Don't call ensure_stream for stream IDs.
Ever since we implemented support for stream IDs in Addressee,
Addressee.stream_name() can now return None. This commit ensures
that _internal_prep_message only calls ensure_stream when
Addressee.stream_name() is not None.
2019-01-31 12:55:10 -08:00
Eeshan Garg 998437c123 check_message: Support sending stream messages by ID.
This commit also contains the following auxiliary changes:
* Adds a custom exception, StreamWithIDDoesNotExist for when
  a stream with a given ID does not exist because the error
  message returned by StreamDoesNotExist only makes with stream
  names, not IDs.
* Adds a new helper, get_stream_by_id_in_realm, which is similar
  to get_user_profile_by_id_in_realm (introduced in #10391).
* Adds a helper, validate_stream_id_with_pm_notification, which
  returns the Stream object associated with a given ID and also
  handles PM notifications to the bot owner if the message was
  sent by a bot and if the stream does not exist or has no
  subscribers.
* Modifies the message sent by send_pm_if_empty_stream to
  accommodate stream IDs.

Note that all of the above changes are required before check_message
can be modified to support stream IDs.
2019-01-31 12:43:38 -08:00
Rishi Gupta 7c11fe819a billing: Update license ledger when users are added and removed. 2019-01-28 16:04:11 -08:00
Eeshan Garg 681f07f98f check_message: Refactor stream validation into separate function. 2019-01-28 11:48:17 -08:00
kunal-mohta b89769420a stream_color: Extend subscribe API to support color attribute.
This fixes an annoying bug where clicking to subscribe to a stream
would change the color shown in the "manage streams" UI immediately
after you click.

Fixes #11072.
2019-01-25 16:44:59 -08:00
Steve Howell 0ef5d1f9c8 user status: Add "status_text" support in the backend.
You can now pass in an info field with a value
like "out to lunch" to the /users/me/status,
and the server will include that in its outbound
events.

The semantics here are that both "away" and
"status_text" have to have defined values in order
to cause changes.  You can omit the keys or
pass in None when values don't change.

The way you clear info is to pass the empty
string.

We also change page_params to have a dictionary
called "user_status" instead of a set of user
ids.  This requires a few small changes on the
frontend.  (We will add "status_text" support in
subsequent commits; the changes here just keep
the "away" feature working correctly.)
2019-01-24 09:46:35 -08:00
Steve Howell ac861f2b7d user status: Restructure code.
We now have single function that handle both away
and not-away.

This refactoring sets us up to piggyback "info" more
easily onto status updates.

The only thing that changes here is that we don't
delete database rows any more when users revoke
their away status.  Instead we just set the status
to NORMAL.
2019-01-24 09:46:35 -08:00
kunal-mohta c07f85250d messages: Extend do_delete_message to do_delete_messages.
do_delete_message has been renamed to do_delete_messages and all
occurrences of the function replaced with the appropriate new version.
2019-01-23 16:49:12 -08:00
kunal-mohta 67f2533ba3 messages: Rename do_delete_messages to do_delete_messages_by_sender. 2019-01-23 16:49:12 -08:00
Rohitt Vashishtha 988af1c803 markdown: Use silent mentions in Notification Bot messages.
Messages converted:
- new stream
- rename stream
- new user signup
2019-01-16 16:01:06 -08:00
Hemanth V. Alluri 716bcad393 custom profile fields: Extract check_remove_user_custom_profile_value.
This moves the logic for deleting the user's custom profile field
value in the remove_user_custom_profile_data view function to a method
named check_remove_user_custom_profile_value in actions.py, so that we
can reuse it in the next commit.
2019-01-15 15:28:16 -08:00
Aditya Bansal 4e4c33fee2 reminders: Make reminders setup work only for Streams.
We make this change because setting up reminders in PM's didn't
play really well with our current infrastructure. Basically the
reminder messages from the bot can't appear in the same narrow as
that of a PM between two people and therefore we disable it.
Though we make an exception here where a person wants to set up
reminder for himself.
2019-01-14 12:23:48 -08:00
Eeshan Garg 020f2d0db5 send_message_backend: Add support for recipient user IDs.
Note that create_mirrored_message_users has not been updated to
use user IDs.

Tweaked by tabbott to add a unit test for the huddle case as well.
2019-01-08 16:17:20 -08:00
Eeshan Garg 060de48800 mypy: Accept integer user IDs in handle_deferred_message. 2019-01-08 19:08:49 -03:30
Eeshan Garg b3e8e36a8d messages: Check for empty recipient lists in Addressee.
The check for empty recipient lists (the "message_to" argument)
does not belong in check_message. As we implement support for
sending messages by user IDs (see #9474), we will be extending
much of the existing code in Addressee.legacy_build that validates
recipient lists. Therefore, Addressee.legacy_build is a much more
apt area to check for empty recipient lists.

Also, Addressee.for_private and Addressee.for_user_ids also need
to do their own validation, since not everything goes through
Addressee.legacy_build. It is okay to simply throw a 500 in these
cases because we expect that callers will be doing their own
validation for calls that don't go through Addressee.legacy_build().

This commit is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
Eeshan Garg 2f634f8c06 backend: Support user IDs for sending typing notifications.
This is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
Eeshan Garg 8f1dba6aad extract_recipients: Support user IDs.
This is a part of our efforts surrounding #9474.
2019-01-08 19:08:49 -03:30
sahil839 4dcccf32f8 zerver: Add feature for notification at rename of a stream.
Feature of sending notification to the stream using notification bot
is added. user_profile is also passed to do_rename_stream for using
the name of user who renamed the stream in notification.
Notification is sent to the stream using
internal_send_stream_message in do_rename_stream.

Fixes #11034.
2019-01-07 19:13:01 -08:00
Marco Burstein 9ddadd39f4 compose: Add support for using Zoom as the video chat provider.
This adds Zoom call properties to the `Realm` model, creates endpoints
for creating calls, adds a frontend and tests.

Fixes #10979.
2019-01-07 10:00:02 -08:00
Shubham Dhama 42c262b807 invite: Replace `invite_as_admin` usage with `invite_as`.
Since we have already added the `invite_as` field to models, we can now
replace usage of `invite_as_admin` properly with its equivalent `invite_as
== PreregistrationUser.INVITE_AS['REALM_ADMIN']`.

Hence, also removed now redundant `invite_as`.
2019-01-05 14:46:38 -08:00
Tim Abbott 2ba24bdd09 settings: Fix flushing of API key cache.
The logic for flushing the API key has been broken every since we
added the cache, since we were incorrectly flushing the new API key,
not the old API key, from the cache after regeneration.
2019-01-04 17:26:10 -08:00
Steve Howell 423db23c36 status: Add endpoint to toggle "away" status. 2019-01-02 09:16:30 -08:00
Shubham Dhama 93e3543e5e user: Extend `do_create_user` for guest users. 2018-12-30 10:16:17 -08:00
Tim Abbott a52eeb364b reactions: Avoid 500 when trying to double-add a reaction.
This IntegrityError has been happening occasionally in production due
to races, likely due to some sort of mobile app double-post bug.
Handle this by avoiding a 500, and returning the same 400 we would do
if there hadn't been a race.
2018-12-29 14:10:48 -08:00
Rishi Gupta 955dfc6397 billing: Remove references to Realm.has_seat_based_plan. 2018-12-24 10:27:56 -08:00
Joshua Pan ad1df0ebeb settings: Add support for customizing the top-left logo.
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.

Tweaked significantly by tabbott to rebase, fix styling, etc.

Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.

Fixes #7995.
2018-12-18 12:44:52 -08:00
Tim Abbott b2fc017671 i18n: Use the recipient's language when sending outgoing emails.
It appears that our i18n logic was only using the recipient's language
for logged-in emails, so even properly tagged for translation and
translated emails for functions like "Find my team" and "password
reset" were being always sent in English.

With great work by Vishnu Ks on the tests and the to_emails code path.
2018-12-17 09:49:36 -08:00
Tim Abbott d9912cb23c management: Fix buggy create_user management command.
The create_user management command incorrectly was double-calling
notify_new_user.
2018-12-16 22:22:52 -08:00
Rishi Gupta 8a95526ced billing: Always transition to Realm.LIMITED via do_change_plan_type.
Fixes a bug in import_realm where secondary attributes like message
visibility weren't being set, and also makes bugs like this less likely in
the future.

Also, putting the plan_type change at the end of import_realm, so that
future restrictions to LIMITED realms don't affect the import process.
2018-12-13 13:26:24 -08:00
Rishi Gupta b245c661da billing: Change do_change_plan_type to take a realm instead of a user.
More often than not, changes in plan type are not directly due to user
action.
2018-12-13 13:26:24 -08:00
Vishnu Ks 8a1794caa3 message: Store the value of first_visible_message_id in Realm table.
This eliminates a bunch of potentially buggy caching code, with no
material negative side effects.
2018-12-12 15:11:17 -08:00
Marco Burstein ba46dc83c6 notifications: Add a setting for changing the notification sound.
Also, add a new notification sound, "ding". It comes from
https://freesound.org, where the original Zulip notification sound comes
from as well. In the future, new sounds can be added by adding audio
files to the `static/audio/notification_sounds` directory.

Tweaked significantly by tabbott:
* Avoided removing static/audio/zulip.ogg, because that file is
  checked for by old versions of the desktop app.
* Added a views check for the sound being valid + tests.
* Added additional tests.
* Restructured the test_events test to be cleaner.
* Removed check_bool_or_string.
* Increased max length of notification_sound.
* Provide available_notification_sounds in events data set if global
  notifications settings are requested.

Fixes #8051.
2018-12-09 21:25:30 -08:00
Joshua Pan 441dca0eaf email: Support delivery email in email change system. 2018-12-06 17:43:26 -08:00
Tim Abbott 7e6f30caab actions: Extract send_user_email_update_event.
This will make it easy for us to reuse this event-sending logic for
the email_address_visibility change code path.
2018-12-06 16:26:46 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Raymond Akornor 92dc3637df send_email: Add support for multiple recipients.
This adds a function that sends provided email to all administrators
of a realm, but in a single email. As a result, send_email now takes
arguments to_user_ids and to_emails instead of to_user_id and
to_email.

We adjust other APIs to match, but note that send_future_email does
not yet support the multiple recipients model for good reasons.

Tweaked by tabbott to modify `manage.py deliver_email` to handle
backwards-compatibily for any ScheduledEmail objects already in the
database.

Fixes #10896.
2018-12-03 15:12:11 -08:00
Tim Abbott d96624490e bulk_add_subscriptions: Fix confusing access to user_profile.realm.
Previously, we frequently accessed user_profile.realm from outside the
loops that interact with UserProfile objects.  This variable reuse
outside the loop could be confusing and should be a style/lint
violation.

While in this case, the behavior was correct (in that all users in the
loops were within the same realm), extracting a separate `realm`
variable significantly clarifies what's going on here.
2018-12-03 10:38:10 -08:00
Tim Abbott d0f71881f4 docs: Add detailed documentation on the process for sending messages.
This has long been something missing from our suite of documentation.
2018-11-29 16:25:35 -08:00
Hemanth V. Alluri 6b1d724f5c zerver: Add bugdown rendering for text custom profile fields.
This is the first step of letting users use Zulip markdown in their
SHORT_TEXT and LONG_TEXT custom profile fields, so that they can
include emphasis, links, etc.

This doesn't include any frontend logic yet, however.
2018-11-29 09:47:32 -08:00
Eeshan Garg 91b90460ee lib/actions: Add helper recipient_for_user_ids().
This function is equivalent to recipient_for_emails, but fetches
user_profiles by IDs, not by emails.

This commit is a part of our efforts surrounding #9474, but is
more primarily geared towards adding support for sending typing
notifications by user IDs.
2018-11-27 11:59:11 -08:00
Eeshan Garg 68a06abb79 actions: Stop using Addressee's helpers in recipient_for_emails.
recipient_for_emails is used by our typing notifications code.
user_profiles_from_unvalidated_emails is used by our typing
notifications code *and* for sending messages.
user_profiles_from_unvalidated_emails is a part of a larger
framework used by Addressee to validate recipient emails when sending
messages and will eventually need to be removed as we move forward
with #9474. So it makes sense to just inline this function within
recipient_for_emails so that we don't break our typing notifications
code in the future.

This commit is a part of our efforts surrounding #9474.
2018-11-27 11:59:11 -08:00
Tim Abbott 6a9e6503b7 python: Remove unnecessary NonBinaryStr TypeVar.
This was a potentially useful value (later superceded by use of Text)
when first introduced, but now that we're on Python 3, it should just
be str.
2018-11-27 11:44:09 -08:00
Raymond Akornor d00b889402 auth: Add an organization reactivation flow with admin confirmation.
This adds a web flow and management command for reactivating a Zulip
organization, with confirmation from one of the organization
administrators.

Further work is needed to make the emails nicer (ideally, we'd send
one email with all the admins on the `To` line, but the `send_email`
library doesn't support that).

Fixes #10783.

With significant tweaks to the email text by tabbott.
2018-11-27 10:58:34 -08:00
Marco Burstein 6f569719c9 integrations: Change the truncation marker for long messages.
Change the truncation marker from `...` to `\n[message truncated]`
when receiving messages from the API or through e-mail. Also, update
tests to account for the new change.

Fix #10871.
2018-11-26 11:09:39 -08:00
Tim Abbott 209dd5db67 actions: Add a function for changing realm subdomains.
This is initial work, which will help us establish habits of using a
well-tested approach for renaming a Zulip organization (since as part
of https://github.com/zulip/zulip-mobile/issues/3142, we'll likely
need to make this function do more).
2018-11-15 14:39:14 -08:00
Steve Howell 0a171bf9bf Fix "prev_subject" with LEGACY_PREV_TOPIC.
I added `LEGACY_` prefix to the var, because
otherwise you'd have really confusing code
where you change the key from `PREV_TOPIC`
to "prev_topic".
2018-11-12 15:47:11 -08:00
Steve Howell af1acf9239 Rename constant to MAX_TOPIC_NAME_LENGTH. 2018-11-07 10:03:53 -08:00
Steve Howell 2cf46f0122 Extract save_message_for_edit_use_case().
This is mostly extracted to help clean out
all "subject" references from actions.py.
2018-11-07 10:03:53 -08:00
Steve Howell 0cf4cddc5b Extract update_messages_for_topic_edit().
This is somewhat hairy logic, so it's nice
to extract it and not worry about variable leaks.

Also, this moves some legacy "subject" references out
of actions.py.
2018-11-07 10:03:53 -08:00
Steve Howell 32232377f7 Rename bugdown.subject_links -> topic_links. 2018-11-07 10:03:53 -08:00
Steve Howell 0e854288ee Add some string constants to topic.py.
The goal here is to make it easy to
forbid "subject" in actions.py.
2018-11-07 10:03:53 -08:00
Steve Howell 7252861785 Extract filter_by_topic_name_via_message. 2018-11-07 10:03:53 -08:00
Steve Howell 2fd0cfe708 Use topic_name() helper in more places. 2018-11-07 10:03:53 -08:00
Steve Howell 55afadc286 Introduce Message.set_topic_name(). 2018-11-07 10:03:53 -08:00
Steve Howell dc8f893494 Extract filter_by_exact_message_topic(). 2018-11-07 10:03:53 -08:00
Steve Howell 56ecd227ff Add topic-related helpers for ScheduledMessage. 2018-11-07 10:03:53 -08:00
Steve Howell 50e3f85557 Extract topic.py library.
We start by including functions that do custom
queries for topic history.

The goal of this library is partly to quarantine
the legacy "subject" column on Message.
2018-11-07 10:03:53 -08:00
Tim Abbott ea1ec68899 events: Pass a realm object into send_event.
This is a preparator refactor for supporting hosting different Tornado
processes on different servers; to look up which Tornado server we
should be sending the event to, we'll need the realm object.
2018-11-02 16:47:39 -07:00
Pragati Agrawal d5df0377cc settings_users: Support guest user in admin-user-table.
This supports guest user in the user-info-form-modal as well as in the
role section of the admin-user-table.

With some fixes by Tim Abbott and Shubham Dhama.
2018-10-29 12:33:35 -07:00
Vishnu Ks cc8dd0e971 billing: Set message_visibility_limit to 10000 for Limited. 2018-10-26 17:10:38 -07:00
Steve Howell 551fc7f165 bots: Prevent bots from having duplicate full names.
Bots are not allowed to use the same name as
other users in the realm (either bot or human).

This is kind of a big commit, but I wanted to
combine the post/patch (aka add/edit) checks
into one commit, since it's a change in policy
that affects both codepaths.

A lot of the noise is in tests.  We had good
coverage on the previous code, including some places
like event testing where we were expediently
not bothering to use different names for
different bots in some longer tests.  And then
of course I test some new scenarios that are relevant
with the new policy.

There are two new functions:

    check_bot_name_available:
        very simple Django query

    check_change_bot_full_name:
        this diverges from the 3-line
        check_change_full_name, where the latter
        is still used for the "humans" use case

And then we just call those in appropriate places.

Note that there is still a loophole here
where you can get two bots with the same
name if you reactivate a bot named Fred
that was inactive when the second bot named
Fred was created.  Also, we don't attempt
to fix historical data.  So this commit
shouldn't be considered any kind of lockdown,
it's just meant to help people from
inadvertently creating two bots of the same
name where they don't intend to.  For more
context, we are continuing to allow two
human users in the same realm to have the
same full name, and our code should generally
be tolerant of that possibility.  (A good
example is our new mention syntax, which disambiguates
same-named people using ids.)

It's also worth noting that our web app client
doesn't try to scrub full_name from its payload in
situations where the user has actually only modified other
fields in the "Edit bot" UI.  Starting here
we just handle this on the server, since it's
easy to fix there, and even if we fixed it in the web
app, there's no guarantee that other clients won't be
just as brute force.  It wasn't exactly broken before,
but we'd needlessly write rows to audit tables.

Fixes #10509
2018-10-24 16:59:57 -07:00
Rishi Gupta 458169928c billing: Rename Zulip Premium to Zulip Standard. 2018-10-24 10:42:16 -07:00
Rishi Gupta fbd884f934 billing: Rename MAX_INVITES_PREMIUM.
Largely to have parallel structure with INVITES_DEFAULT_REALM_DAILY_MAX.
2018-10-23 14:47:00 -07:00
Rishi Gupta a7fe6b1c34 billing: Fix max_invites for PREMIUM_FREE.
We could migrate all the current PREMIUM_FREE organizations to have more
invites, but this setting mainly affects orgs right as they are starting, so
it's probably fine.
2018-10-23 14:47:00 -07:00
Vishnu Ks 57e0a24020 management: Scrub delivery email as well while running scrub_realm. 2018-10-16 11:16:46 -07:00
Steve Howell 493aae2958 imports: Make loading UserMessage faster and more robust.
We use UserMessageLite to avoid Django overhead, and we
do updates in chunks of 10000.  (The export may be broken
into several files already, but a reasonable chunking at
import time is good defense against running out of memory.)
2018-10-13 16:43:28 -07:00
Vishnu Ks 962d72b58b retention: move_messages_to_archive should accept multiple message ids.
This will speed up the scrub realm management command. Calling the
function with a single message_id in a loop was extremely inefficient.
2018-10-11 15:31:12 -07:00
Vishnu Ks 6972de21be management: Add command to scrub a realm of personal data. 2018-10-11 15:30:26 -07:00
Vishnu Ks 5bdadc8061 upload: Create function to delete avatar image. 2018-10-11 15:30:26 -07:00
Shubham Padia 6bfa29b8e6 notifications: Fix soft-deactivated users don't get push notifications.
Fixes the urgent part of #10397.

It was discovered that soft-deactivated users don't get mobile push
notifications for messages on private streams that they have configured
to send push notifications.

Reason: `handle_push_notification` calls `access_message`, and that
logic assumes that a user who is a recipient of a message has an
associated UserMessage row. Those UserMessage rows are created
lazily for soft-deactivated users, so they might not exist (yet)
until the user comes back.

Solution: Ensure that userMessage row is created for
stream_push_user_ids and stream_email_user_ids in create_user_messages.
2018-09-21 12:06:18 -07:00
Rishi Gupta 31ed4492ce billing: Add backend for downgrading. 2018-08-31 17:49:34 -07:00
Vishnu Ks 7b307fa58f billing: Set max_invites to 3000 for Premium organizations.
This is meant to be effectively infinite for legitimate use.
2018-08-28 16:03:34 -07:00
Tim Abbott c313a939f7 custom profile fields: Cleanup event generation logic.
In my opinion, this makes the code somewhat more readable.
2018-08-22 22:45:08 -07:00
Yashashvi Dave d5153bd136 events: Convert custom user field value to json object on update event.
In user type custom field, field value is list of user ids. We weren't
converting list to json object in update event payload. This throws
error in frontend, cause we store stringify representation of custom
field value. Therefore, after update event is recieved field-value-
type gets updated to array from string which throws json parsing error.
2018-08-22 22:45:08 -07:00
Shubham Dhama e784c95d97 guests: Prevent guests from sending to unsubscribed public streams.
This matches the overall security model of these users only having
access to streams they are subscribed to.
2018-08-22 17:53:42 -07:00
Steve Howell fe6680c316 refactor: Flatten code in check_invite_limit().
Using early-exit here allows us to more easily
comment why there are certain exemptions to
this logic.

We also only require callers to pass in realm,
not the whole user object.
2018-08-22 16:52:30 -07:00
Tim Abbott 2e6aaf3215 actions: Use better query for active_mobile_push_notification. 2018-08-21 15:28:05 -07:00
Vishnu Ks 9bb338be11 models: Add plan_type to Realm. 2018-08-21 12:39:06 -07:00
Steve Howell a6bc3886e6 refactor: Extract send_peer_remove_event().
This prevents leaking some variables into an already
cluttered function.

We also add test coverage for what's now an
early-exit condition in the new function--we exempt
public MIT streams from these events.
2018-08-21 11:23:40 -07:00
Steve Howell 79fb36c599 refactor: Extract maybe_add_event() function.
This change was partially driven by a quirk in Python
where peephole optimizations make `continue` lines
appear not to be covered.

I also think it's generally a good idiom to extract
functions for loop bodies when they don't actually
accumulate values or maintain other state.  With this
commit we now prevent potential bugs for vars like
`is_stream` leaking between loop iterations.
2018-08-21 11:23:40 -07:00
Steve Howell e99c0929f0 tests: Test race handling for creating mirror users.
We simulate a race condition by mocking create_user
to actually create a user, but then raise an
IntegrityError (as if another process had actually
created the user, not our test).

I also changed the real code to use explicitly
named parameters.
2018-08-21 11:23:40 -07:00
Steve Howell 0e56fecbd9 peformance: Avoid broad StreamCount queries.
Our get_streams_traffic function used to query
all streams in the StreamCount table if you
passed in `None` for `streams`.

Now we require that you pass in a list of
stream_ids.

I don't know how much work this will save
the database, since probably the bulk of
the work is aggregating.  If we need to fine
tune DB performance, we could possibly add
`realm` as an argument and add it to the filter.

What we'll immediately get, for large multi-realm
installations, is less data over the wire and
less work for the ORM.
2018-08-17 08:14:42 -07:00
Steve Howell 040dafbfc5 refactor: Streamline subscribed_to_stream().
The prior code uses an awkward idiom that
pre-dates the `exists()` function, and it
had an unreachable line of code.

The new version should be faster, since we
don't create a throwaway heavy Django object
or send needless data over the wire.
2018-08-17 08:14:42 -07:00
Steve Howell c41377aaab Remove _default_stream_permision_check.
This functions appears to be redundant to
`access_stream_by_name`.  The only
meaningful line of code in the function that we're
removing, the code that raises an error,
appears to be unreachable, despite reasonably
extensive tests.

The only thing the function was restricting
was that the case where the bot's owner was
unsubscribed to a private stream, which
is already locked down in
`access_stream_by_name` calls inside of
`patch_bot_backend`.

This commit increases test coverage
by removing unreachable code.

It's possible this function had
some theoretical value before we
introduced the `require_non_guest_human_user`
decorator to the `patch_bot_backend`
view, since in theory the bot itself
could have subscribed to a stream that
the owner didn't subscribe to.  Even
then it's not clear that allowing the
bot to set that as a default stream
would have been harmful, since they
can already access it.
2018-08-17 08:14:42 -07:00
Steve Howell 93e8798ac7 Extract get_last_message_id().
We want our methodology for extracting the last message
id to be consistent, particularly in terms of how we
handle edge cases.  (I'll concede that the
`bulk_remove_subscriptions` codepath never hits that
corner case in practice, but it's harmless to handle
the theoretical case.)

It may also be nice to have this function show up
clearly in profiling.

This also adds some direct testing to the function.

It's not clear to me why we don't use `latest('id')`
in the implementation, but that's outside the scope
of this commit.
2018-08-17 08:14:42 -07:00
Steve Howell 9a7a93c80b refactor: Extract validate_sender_can_write_to_stream().
This de-clutters check_message a bit and also makes
it easy to audit our rules for who can write to a
stream.

Also, this works around a bug with Python where its
optimizations for the `pass` instruction make them
not appear to run and show up as uncovered in
coverage reports.
2018-08-14 10:34:58 -07:00
Steve Howell 3112c6596c tests: Change message-type error to AssertionError.
We validate the user input upstream of this code.
2018-08-13 10:37:35 -07:00
Steve Howell 297f086b6a tests: Add coverage for service bot events.
We test the "skipping" logic a couple different ways.
2018-08-13 10:37:35 -07:00
Steve Howell 413a0174f4 Extract a zephyr.py library.
Right now it only has one function, but the function
we removed never really belonged in actions.py, and
now we have better test coverage on actions.py, which
is an important module to get to 100%.
2018-08-11 14:51:26 -07:00
Steve Howell 5bc33213c9 Refactor generate_topic_history_from_db_rows.
Sorting the rows first simplifies the loop logic here.

This has good test coverage--you'll get a failing
test if you comment out the sort.
2018-08-11 14:51:26 -07:00
Tim Abbott da8f4bc0e9 push notifications: Add support for removing GCM push notifications.
This uses the recently introduced active_mobile_push_notification
flag; messages that have had a mobile push notification sent will have
a removal push notification sent as soon as they are marked as read.

Note that this feature is behind a setting,
SEND_REMOVE_PUSH_NOTIFICATIONS, since the notification format is not
supported by the mobile apps yet, and we want to give a grace period
before we start sending notifications that appear as (null) to
clients.  But the tracking logic to maintain the set of message IDs
with an active push notification runs unconditionally.

This is designed with at-least-once semantics; so mobile clients need
to handle the possibility that they receive duplicat requests to
remove a push notification.

We reuse the existing missedmessage_mobile_notifications queue
processor for the work, to avoid materially impacting the latency of
marking messages as read.

Fixes #7459, though we'll need to open a follow-up issue for
using these data on iOS.
2018-08-10 13:58:39 -07:00
Tim Abbott cc5c8fc022 do_update_pointer: Improve docs for old mobile app code path. 2018-08-10 13:58:39 -07:00
Tim Abbott 8ba726d47d do_update_pointer: Fix missing where= declaration.
Fixes a regression introduced in 23246ff816.

However, we'll be shortly removing this feature, since it's legacy
support for an app that no longer is supported.
2018-08-10 13:58:39 -07:00
Shubham Padia 9dcac3479e actions.py: Set is_private flag in do_send_messages. 2018-08-09 16:08:04 -07:00
Shubham Padia a524d425ad actions.py: Block client interaction with flags in the NON_API_FLAGS.
Raise error if flag is present in NON_API_FLAGS or is not present in
UserMessage.flags.
2018-08-09 16:08:03 -07:00
Steve Howell 2e42bdd7c3 Remove "/stats" command for now.
The "/stats" command doesn't actually do anything
interesting yet, and it also writes to the message
feed instead of replying directly to the user.

The history of this command was that it was
written during a PyCon sprint.  It was mainly intended
as an example for subsequent slash commands.  The
ones we built after "/stats" have sort of outgrown
"/stats" and don't follow the original structure
for "/stats".  (The "/day", "/ping", and "/settings"
commands were built shortly after.)j

We probably want to ressurect "/stats" fairly soon,
after figuring out some useful stats and refining
the UI.

As you can see from this commit, resurrecting the
code here shouldn't be too difficult, but it
may actually be pretty rare that we just translate
slash commands into fleshed out messages.
2018-08-08 16:49:27 -07:00
Yago González 6a192ac84c utils: Move random API key generator as generate_api_key.
random_api_key, the function we use to generate random tokens for API
keys, has been moved to zerver/lib/utils.py because it's used in more
parts of the codebase (apart from user creation), and having it in
zerver/lib/create_user.py was prone to cyclic dependencies.

The function has also been renamed to generate_api_key to have an
imperative name, that makes clearer what it does.
2018-08-08 16:45:25 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Yago González 5566646c45 user_groups: Handle renaming to existing names.
Renaming a user group to a name shared by other group wasn't a scenario
handled by the backend, and the server errored whenever this was
attempted.

Now a json_error is returned, letting the user know that a user group
with that name already exists.
2018-08-08 11:03:47 -07:00
Yashashvi Dave 290388e5e0 stream settings: Fix bug in UI when last user unsubscribe private stream.
When last user(only in case of admin) unsubscribe from private stream,
stream page doesn't get updated. Cause we delete the private stream
as soon as last user unsubscribe from stream.
So `sub` get undefined in frontend, cause that stream is deleted
before unsubscribe-user-from-stream event is received.

Fix this by changing order of events sent to frontend. Event
`subscription: remove` should be sent before `stream: delete` event
from backend.
2018-08-07 13:30:53 -07:00
Tim Abbott e7c7211c30 mypy: Fix type of messages in do_update_message_flags.
Ever since we moved the stream/everything cases to separate functions,
the messages argument has actually been required.
2018-08-01 17:37:16 -07:00
Tim Abbott 23246ff816 do_update_pointer: Switch to using where_unread().
This produces a more efficient database query (just because postgres
doesn't use the right index by default the other way).
2018-08-01 16:51:56 -07:00
Tim Abbott c775be8ea4 do_mark_stream_messages_as_read: Accept a Client object.
We also fix an incorrect Optional in the type annotations.
2018-08-01 16:49:57 -07:00
Tim Abbott 6e55342e21 bulk_remove_subscriptions: Pass client object in.
We need the client object to pass on to do_mark_stream_as_read.
2018-08-01 16:48:31 -07:00
Tim Abbott c60f197fde do_update_message_flags: Accept a Client object.
This is important for upcoming logging changes.
2018-08-01 16:40:58 -07:00
Tim Abbott 0e44010976 do_mark_all_as_read: Accept a client object.
This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott 5f0519dfb4 do_update_pointer: Pass client object from callers.
We also fix an unused import.

This is needed for upcoming logging changes.
2018-08-01 16:40:15 -07:00
Tim Abbott 58ee3fa8c4 page_params: Include avatars and similar data in cross-realm bots.
This ensures that the format of this data structures matches that for
in-realm bots in the main users data structure (including avatars,
etc.).

Fixes #10138.
2018-08-01 15:09:11 -07:00
Vishnu Ks 6b3706494c notifications: Pass realm_creation argument to enqueue_welcome_emails. 2018-08-01 11:29:34 -07:00
Tim Abbott 1b2a26ca83 events: Fix missing empty custom profile data dict for new users.
We were getting event-handling exceptions in JS in production if a new
user was created and then went and set a custom profile field, because
there was no `.profile_data` on their user object.  We were able to
trace the issue down to the fact that our events didn't include that
field when creating a new user.
2018-07-31 11:08:11 -07:00
Roman Godov c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Rishi Gupta e37b5dce79 stream settings: Fix brand new streams showing 0 weekly traffic.
A stream created in the last few hours likely won't be in StreamCount
(since that gets updated once a day), and hence won't be in the
recent_traffic dict.

However, get_average_weekly_stream_traffic should be None in this case,
not 0.
2018-07-23 16:43:15 -07:00
Rishi Gupta 0573c17819 stream settings: Set weekly traffic to None instead of -1 for new streams.
The only place this gets used seems to be subscription.handlebars, where a
value of None (or null) is probably less broken-seeming than -1 anyway.
2018-07-23 16:43:15 -07:00
Rishi Gupta b75aa8e35a stream settings: Fix age calculation in is_old_stream.
Match the calculation in get_average_weekly_stream_traffic.
Regression introduced in dc7cfd3.

Fixes #10002
2018-07-23 16:43:15 -07:00
Vishnu Ks 2b28042ddf models: Rename remaining audit log event types to past tense.
This makes the realm audit event type log entries more consistent.
2018-07-22 20:00:28 -07:00
Tim Abbott bf1ad714da actions.py: Refactor generate_topic_history_from_db_rows for clarity.
This refactors the generate_topic_history_from_db_rows function to not
depend upon the assumption of rows passed as parameter to be sorted in
reverse order of max_message_id field.

Additionally, we add sorting and some tests that verify correct
handling of these cases.
2018-07-14 10:11:08 +05:30
Aditya Bansal 63ec8b08b8 archives: Add endpoint to fetch topic history of web public streams.
In this commit we add a new endpoint so as to have a way of fetching
topic history for a given stream id without having to be logged in.
This can only happen if the said stream is web public otherwise we
just return an empty topics list. This endpoint is quite analogous
to get_topics_backend which is used by our main web app.

In this commit we also do a bit of duplication regarding the query
responsible for fetching all the topics from DB. Basically this
query is exactly the same as what we have in the
get_topic_history_for_stream function in actions.py. Basically
duplicating now is the right thing to do because this query is
really gonna change when we add another criteria for filtering
messages which is:
Only topics for messages which were sent during the period the
corresponding stream was web public should be returned.
Now when we will do this, the query will change and thus it won't
really be a code duplication!
2018-07-14 09:51:37 +05:30
Aastha Gupta dfde4fac85 invitations: Send 'invites_changed' event for invitations events.
Fixes #7665

In case of invitation events, 'invites_changed' event without
any real payload is sent to all the realm admins and the user.
The event is handled by reloading the list to view recent changes.

Commit tweaked by shubhamdhama:
* Send an `invite_changed` event when an user accept an invite.
    Also, added the test for the same.
* No need to delete the invite list in frontend, current logic
    handles the case when the invite data is changed properly.
* Extracted the common logic for sending an event into
    `notify_invites_changed`.
2018-07-13 18:00:06 +05:30
Sarah d1cda29fb4 API: Add stream email notification setting plumbing.
This is all the plumbing that makes it possible to enable the
stream_email_notifications setting via the Zulip API.  The flag still
doesn't do anything yet, but this is a nice checkpoint along the way
to implementing this feature.
2018-07-12 13:39:14 +05:30
Tim Abbott aa5ca4e549 actions: Remove unnecessary import of EmailMessage.
Usually we don't bother with unused imports, but this helps with
reasoning about where we use the Django mail APIs in Zulip.
2018-07-12 12:32:27 +05:30
Joshua Pan 533eccd655 models: Create delivery_email field in userprofile.
This commit creates a new field called delivery_email. For now, it is
exactly the same as email upon user profile creation and should stay
that way even when email is changed, and is used only for sending
outgoing email from Zulip.

The purpose of this field is to support an upcoming option where the
existing `email` field in Zulip becomes effectively the user's
"display email" address, as part of making it possible for users
actual email addresses (that can receive email, stored in the
delivery_email field) to not be available to other non-administrator
users in the organization.

Because the `email` field is used in numerous places in display code,
in the API, and in database queries, the shortest path to implementing
this "private email" feature is to keep "email" as-is in those parts
of the codebase, and just set the existing "email" ("display email")
model field to be something generated like
"username@zulip.example.com" for display purposes.

Eventually, we'll want to do further refactoring, either in the form
of having both `display_email` and `delivery_email` as fields, or
renaming "email" to "username".
2018-07-12 12:30:20 +05:30
Joshua Pan 4b3fb746ea Remove incorrect setting of user_profile.email.
user_profile.save() is never called, so this line of
code is actually useless.
2018-07-12 12:23:46 +05:30
Vishnu Ks a0da184d50 models: Add SUBSCRIPTION_ACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks abb218ebab models: Add SUBSCRIPTION_DEACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks ff4c1ca2c1 models: Add SUBSCRIPTION_CREATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 20fae065f6 models: Add BOT_OWNER_CHANGED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 9670e319cc models: Add REALM_REACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks d81cf2268a models: Add REALM_DEACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks d5b6f032d2 models: Add USER_API_KEY_CHANGED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 0d69fc482b models: Add USER_TOS_VERSION_CHANGED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 83ff688bb6 models: Add USER_EMAIL_CHANGED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 75ed3f92e2 models: Add USER_FULL_NAME_CHANGED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks c8aac0fbb9 models: Add USER_CHANGE_AVATAR_SOURCE event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 4c73221108 models: Add USER_CHANGE_PASSWORD event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 201b99a6f8 models: Add USER_REACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks d0b89cbb44 models: Add USER_DEACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks ce3fffdbb2 models: Add USER_ACTIVATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Vishnu Ks 2c8effe9fe models: Add USER_CREATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Aditya Bansal 0c5dadf320 actions.py: Extract topic history list generation as a function.
We extract out the logic for generating a list of all historical
topics for a given stream as a separate function.  This avoids code
duplication when we add the similar code path for grabbing all topics
for web public streams.
2018-07-10 10:22:07 +05:30
Shubham Dhama b9a8c4df8c bugdown: Move BugdownRenderingException to exceptions.py. 2018-07-09 15:45:45 +05:30
Rishi Gupta b5753d0ddc billing: Add initial support for seat based plans.
The main remaining todo for correctly populating
RealmAuditLog.requires_billing_update is supporting the de-seating (and
corresponding re-seating) that happens after being offline for two weeks.
2018-07-09 14:33:08 +05:30
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Eeshan Garg e1e4d523aa Replace remaining user-facing occurences of the term "invite-only".
Fixes #9611.
2018-06-27 12:34:29 -07:00
Rishi Gupta dc7cfd3eb7 stream settings: Fix stream_age calculation in weekly traffic calculation. 2018-06-23 12:48:15 -07:00
Rishi Gupta 3054b175c9 stream setting: Fix mypy error in get_average_weekly_stream_traffic.
Not sure why mypy wasn't catching this.
2018-06-23 12:48:15 -07:00
Rishi Gupta 15b236a21f stream settings: Show non-zero stream traffic if traffic is greater than 0.
An estimated traffic of 0 suggests a stream is dead, and has pretty
different semantics from any non-zero value. So we should round up any
number between 0 and 1 to 1.
2018-06-23 12:48:15 -07:00
Rishi Gupta 5eccabc3c5 stream settings: Set average_weekly_traffic to -1 if stream is new.
We don't ever use this value, but it's confusing to have the incorrect
calculation in the code.

Ideally we would set this to "None", but I don't know the code well enough
to be confident nothing would break.
2018-06-23 12:48:15 -07:00
Rishi Gupta f63bcb99d7 stream settings: Refactor out STREAM_TRAFFIC_CALCULATION_MIN_AGE_DAYS. 2018-06-23 12:48:15 -07:00
Rishi Gupta ccf5f1ac89 stream settings: Fix weekly stream traffic computation.
Previously, if stream_age were 13, this number would be off by almost a
factor of 2.
2018-06-23 12:48:15 -07:00
Yashashvi Dave 317a2fff2a bots settings: Improve API for editing bot owners to refer to users by ID.
Fixes #9502
2018-06-23 12:47:50 -07:00
Vishnu Ks 1b179ca530 signup: Prevent users from signing up with email containing +. 2018-06-23 12:03:30 -07:00
Vishnu Ks a88e44e254 registration: Use tokenized noreply address in email change. 2018-06-23 12:03:30 -07:00
Vishnu Ks e34fcf982f registration: Use tokenized noreply address in user invite. 2018-06-23 12:03:30 -07:00
Yashashvi Dave 8909cb1d15 custom fields: Allow list of users in user type of custom fields.
Allow user to add more than one user-value in user type of custom
fields.

Tweaked by tabbott to improve the models.py code and type annotations.
2018-06-16 09:37:49 -07:00
Yashashvi Dave f7f039e772 custom fields: Add endpoint to delete value of custom profile data.
Add delete endpoint at `users/me/profile_data` to delete value of
custom profile data.

Fixes #9669
2018-06-07 08:35:04 -07:00
Shubham Dhama 01555e8772 streams: Handle guest user ids for stream settings changes' events. 2018-06-04 11:35:37 -07:00
Tim Abbott ec90322e5d bots: Fix behavior for bots mentioned in PMs.
We've for a long time had the behavior that a bot mentioned in a
stream message receives the notification, regardless of whether the
bot was actually subscribed to the stream.

Apparently, this behavior also triggered if you mentioned a bot in a
private message (i.e. the bot would be delievered the private message
and would probably respond unhelpfully in a new group private message
thread with the PMs original recipients plus the bot).

The fix for this bug is simple: To exclude this feature for private
messages.
2018-06-03 22:29:15 -07:00
Shubham Padia 8883107a7f streams: Allow cross-realm bots to post to announcement_only streams.
This fixes an issue where if you make #announce (the default
announcement stream) announce-only, then creating a new stream will
throw an exception (because notification-bot can't send there).

Fixes #9636.
2018-06-02 10:36:29 -07:00
Tim Abbott 18702b0d71 actions: Document validate_user_access_to_subscribers_helper.
This function is important, carefully optimized, and hard to read, so
it deserves a long docstring.
2018-06-02 08:46:58 -07:00
Shubham Dhama 8e2337509d streams: Hide public streams & subscribers from guest users.
The main change here is making can_access_public_streams
2018-06-02 08:39:39 -07:00
Tim Abbott 0d84eb0a8b mypy: Clean zerver/views/users.py for strict-optional. 2018-06-01 08:48:17 -07:00
Steve Howell 554883e039 submessages: Add submessage_id to submessage event.
It's useful to have the submessage_id so that widgets
can in theory sort all the events (and not have to rely
on them arriving in sequence).
2018-05-30 14:47:36 -07:00
Shubham Padia 214ce1ccca streams: Add create and edit ui for is_announcement_only.
The user can now specify the value while creating a stream.
An admin can later change it via `Change stream permissions`
modal. Add is_announcement_only to subscription type text.
2018-05-30 14:22:15 -07:00
Shubham Padia bb8ad15fa9 streams: Set is_announcement_only when creating streams.
Adds is_announcement_only to create_stream_if_needed and
add_subscriptions_backend.
2018-05-30 14:17:33 -07:00
Steve Howell bfc13ecea4 submessages: Just send content for submessage events.
For some reason in my original version I was sending both
content and data to the client for submessage events,
where data === JSON.parse(content).  There's no reason
to not just let the client parse it, since the client
already does it for data that comes on the original
message, and since we might eventually have non-JSON
payloads.

The server still continues to validate that the payload
is JSON, and the client will blueslip if the server
regressses and sends bad JSON for some reason.
2018-05-30 13:53:11 -07:00
Robert Hönig 647c63050f botserver: Add outgoing webhook tokens to botserverrc.
The tokens will be used to authorize the server when sending
messages to the Botserver.
2018-05-30 10:00:19 -04:00
Eeshan Garg 0a1b429d04 settings: Remove PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS.
Now that we have a UI for this feature, we don't need this setting
anymore.
2018-05-24 17:18:14 -07:00
Steve Howell 69517f5ac5 Support zform-based widget content in the server.
API users, particularly bots, can now send a field
called "widget_content" that will be turned into
a submessage for the web app to look at.  (Other
clients can still rely on "content" to be there,
although it's up to the bot author to make the
experience good for those clients as well.)

Right now widget_content will be a JSON string that
encodes a "zform" widget with "choices."  Our first
example will be a trivia bot, where users will see
something like this:

    Which fruit is orange in color?

        [A] orange
        [B] blackberry
        [C] strawberry

The letters will be turned into buttons on the webapp
and have canned replies.

This commit has a few parts:
    - receive widget_content in the request (simply
        validating that it's a string)
    - parse the JSON in check_message and deeply
        validate its structure
    - turn it into a submessage in widget.py
2018-05-24 09:30:22 -07:00
Vishnu Ks 8369e2b15e registration: Allow users to import settings from other realm.
This should significantly improve the user experience for creating
additional accounts on zulipchat.com.

Currently, disabled in production pending some work on visual styling.
2018-05-23 10:30:00 -07:00
Aditya Bansal d343f25cc6 create_realm_internal_bots: Refactor to extract main op as a function.
We extract the entire operations of the management command to a
function create_if_missing_realm_internal_bots in the
zerver/lib/onboarding.py. The logic for determining if there are any realm
internal bots which have not been created is extracted to a function
missing_any_realm_internal_bots in actions.py.
2018-05-23 11:53:22 +05:30
Tim Abbott e7fa77f120 validate_email_for_realm: Clarify errors for deactivated users.
If a user's account has been deactivated, we want to provide a special
error message that makes clear what's going on.

Future work is to provide some administrative controls on whether a
user should be able to re-activate their account.
2018-05-20 19:32:53 -07:00
Tim Abbott 9360af37d4 typing: Remove unnecessary conditional for recipient types.
It wasn't actually possible for `recipient_for_emails` to return a
STREAM regardless, and this makes things read a little clearer.
2018-05-20 18:35:48 -07:00
Tim Abbott ecdc7fb296 typing: Fix unnecessary else clause for recipient validation.
The other cases all return anyway.
2018-05-20 18:27:25 -07:00
Joshua Pan bdba539480 Rename get_fixed_content_for_widget to do_widget_pre_save_actions. 2018-05-20 14:42:09 -07:00
Vishnu Ks 372e9740ac events: Add date_joined to user_dict. 2018-05-17 07:49:35 -07:00
Steve Howell 1f0ffdc415 Add widget.py to enable widgets on the server side.
The only slash command implemented in this initial
version is an extremely crippled version of a
"/stats" slash command that reports that you are
running 1 server.
2018-05-16 15:13:33 -07:00
Steve Howell de47eeb6f1 Add /submessage endpoint. 2018-05-16 15:13:33 -07:00
= e731aeda44 zerver: Remove dead code from do_change_password. 2018-05-15 13:39:53 -07:00
= 9d07faaf0c zerver: Remove dead code from do_create_realm. 2018-05-15 13:39:53 -07:00
= 344d6544da zerver: Remove dead code for accessing subscribers.
These haven't been used in years, and clutter the codebase.
2018-05-15 13:39:39 -07:00
Baron Chandler f59adfa67c actions: Change do_change_is_admin Exception to an Assertion Error.
- do_change_is_admin now raises AssertionError when a non-admin
  permission is given.
- adds test to test_users to ensure admin asserts on invalid
  permission values.
2018-05-15 11:27:01 -07:00
Tim Abbott 726017f682 actions: Remove dead do_change_bot_type function.
This is a tiny fraction of a feature we don't support, so we should
just kill it.
2018-05-13 17:58:49 -07:00
Tim Abbott 8b09118009 actions: Rename pick_color_helper to pick_color.
Now that there's only one function here, it's weird to have the
unnecessary _helper suffix.
2018-05-13 17:52:36 -07:00
Tim Abbott 234b5fa21b actions: Remove dead pick_color function.
We've been using pick_color_helper only for a long time.
2018-05-13 17:52:04 -07:00
Eeshan Garg 5c0d4660c1 models: Add history_public_to_subscribers to Stream.to_dict().
This commit also updates all the relevant parts where this attribute
could be useful, e.g. payloads for user subscriptions.
2018-05-13 09:15:07 -07:00
Shubham Padia 295fcb8536 models: Add `is_announcement_only` to stream_dict in `actions.py`.
Adds `is_announcement_only` to `stream_dict`s in order to access
the property in the frontend.
2018-05-13 09:11:51 -07:00
Shubham Padia 897ed17f0c api: Allow realm_admins to make a stream announcement_only. 2018-05-13 09:11:51 -07:00
Shubham Padia bb8577ba94 stream: Only realm admins can post to an announcement_only streams.
If a non-admin tries to post to an announcement_only stream, error
message will be shown.
2018-05-13 09:11:51 -07:00
Aditya Bansal 97b7075dd2 zerver/lib/actions.py: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Raymond Akornor f67efd5291 zerver: Migrate away from six. 2018-05-09 18:38:25 +00:00
Tim Abbott fba45bb9d3 archive: Add functions for accessing web-public streams.
These will be helpful for variable upcoming projects to support the
web-public streams feature.
2018-05-08 12:12:15 -07:00
Eeshan Garg 19fa73891e actions: Refactor rate-limiting logic out of send_pm_if_empty_stream.
It makes sense to refactor out the last_reminder logic out of
send_pm_if_empty_stream and have a generic function that can send
rate-limited PM notifications to a bot owner and can be used by
methods other than send_pm_if_empty_stream.
2018-05-05 15:46:27 -07:00
Tim Abbott 956bd74905 attachments: Send events for attachment updates.
We send add events on upload, update events when sending a message
referencing it, and delete updates on removal.

This should make it possible to do real-time sync for the attachments
UI.

Based in part on work by Aastha Gupta.
2018-05-04 16:22:27 -07:00
Tim Abbott 60185dddfd bots: Fix buggy database queries in loop.
A typo in my reading of 6cc2e8bbff meant
that we were incorrectly doing database queries for each Service
object, just to get the user_profile.id, which we already had.
2018-05-04 15:23:13 -07:00
Tim Abbott 59d3fefc07 bots: Improve performance of fetching core bot data.
This eliminates the need to call user_ids_to_users inside the
get_service_dicts_for_bots code path, saving a database query.

This completes my refactor to fix backend performance issues in this
code path.  Previously, our messy layering of queries that resulted in
Zulip doing work even if none of the bots actually had Services or
config_data.
2018-05-04 13:44:43 -07:00
Tim Abbott 0e81353ce0 bots: Merged get_services_for_bots into its caller.
This function is now only a few lines of simple code called in one
place, and is better eliminated.
2018-05-04 13:41:41 -07:00
Tim Abbott 6cc2e8bbff bots: Remove unnecessary select_related on Service objects.
Given how we're using these, there's no need for the full UserProfile
object to be fetched from the server.
2018-05-04 13:41:29 -07:00
Tim Abbott 186152bfc0 bots: Pass a list of user IDs into get_services_for_bots.
We weren't using the full profile objects.
2018-05-04 13:41:15 -07:00
Tim Abbott 4eb3c72c74 bots: Move get_services_for_bots into actions.py.
It's better to just have this bundle of code all in one place; also,
after some cleanup, we'll be inlining it into
get_service_dicts_for_bots.
2018-05-04 13:36:25 -07:00
Tim Abbott 894a952f6f bots: Compute embedded bots list using bot_dicts. 2018-05-04 13:34:54 -07:00
Tim Abbott 13f1f6a388 bots: Pass the full bot_dicts objects into get_service_dicts_for_bots.
This is preparatory refactoring for not needing to fetch the
bot_profile objects from the database.
2018-05-04 13:33:32 -07:00
Tim Abbott 447f8db8cb get_bot_configs: Adjust API to accept a list of bot user IDs.
This is preparatory refactoring for removing user_profile objects from
the get_service_dicts_for_bots code path.
2018-05-04 13:31:28 -07:00
Vishnu Ks 733da0ac07 settings: Fix 500 when trying to change email to disposable email.
Fixes #9240
2018-05-04 10:52:38 -07:00
Eeshan Garg d28d08e7da streams: Add get_default_value_for_history_public_to_subscribers().
This commit adds a function that makes it easier to get a default
value for Stream.history_public_to_subscribers when one isn't
explicitly provided.
2018-05-03 10:24:27 -07:00
Aditya Bansal 1f358954be web_public_streams: Add is_web_public to Stream table.
Also add function do_change_stream_web_public in lib/actions.py
to help in changing a streams web public status.
2018-05-02 15:23:33 -07:00
Angelika Serwa f4f64243dd custom_profile_fields: Support changing the sort order of the fields.
Tweaked by tabbott for variable naming and the URL.

Closes #8879.
2018-04-30 18:17:41 -07:00
Eeshan Garg 057ff9c91e models: Add Stream.history_public_to_subscribers.
This commit adds a new field history_public_to_subscribers to the
Stream model, which serves a similar function to the old
settings.PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS; we still use that
setting as the default value for new streams to avoid breaking
backwards-compatibility for those users before we are ready with an
actual UI for users to choose directly.

This also comes with a migration to set the value of the new field for
existing streams with an algorithm matching that used at runtime.

With significant changes by Tim Abbott.

This is an initial part of our efforts on #9232.
2018-04-28 22:54:04 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Tim Abbott 127ac0df54 auth: Remove unnecessary case from validate_email_for_realm.
The removed code path was only needed due to buggy setup code in the
test_cross_realm_scenarios test.  We address that with a less buggy
workaround, and which lets us remove unnecessary complexity from this
important validation function.

Thanks for Umair Waheed for some preliminary work on this.

Fixes #7561.
2018-04-28 11:03:03 -07:00
YJDave 6bef44a9fa org setting: Add time limits for message deleting.
Add realm setting to set time limit for message deleitng.
Set default value of message_content_delete_limit_seconds
to 600 seconds(10 min).

Thanks to Shubham Dhama for rebasing and reworking this.  Some final
edits also done by Tim Abbott.

Fixes #7344.
2018-04-27 19:22:28 -07:00
Vishnu Ks f140b0e870 emails: Add translation tags to invitation. 2018-04-27 11:59:36 -07:00
Umair Khan cf2f6b38dd profile: Add choice field.
Fixes part of #8878
2018-04-26 00:35:51 -07:00
Tim Abbott 49f58583a4 models: Introduce can_access_public_streams field.
This is a simple computed field.  It's intended to more clearly
capture the meaning of this restriction for the users in zephyr mirror
realms, and eventually support guest user accounts in normal Zulip
realms.
2018-04-20 12:06:57 -07:00
Tim Abbott dc6d7d0d12 actions: Use presence_disabled for presence zephyr_mirror code path.
This is part of the effort to remove the use of is_zephyr_mirror_realm
across the code path for situations that might be relevant for other
users.  It helps keep the code readable.
2018-04-20 12:06:57 -07:00
Roman Godov d99758129e subs: Notify organization admins when private streams are renamed.
This commit sends the event for renaming of a private stream to
organization admins of the realm, in addition to the obvious list of
subscribers of the private stream.

Normally, admins can manage a private stream (e.g. unsubscribing a
user).  But when the admin tried to unsubscribes a user from a
previously renamed stream, we previously were throwing a JS error, as
the webapp hadn't been notified about the new stream name.

Fixes #9034.
2018-04-19 22:00:18 -07:00
Umair Khan 0420b89468 profile: Add hint attribute to custom profile fields.
This is the model for #8876.
2018-04-19 11:32:35 -07:00
novokrest 4d2082ab14 actions.py: Obtain bot profiles by using users.user_ids_to_users().
Remove models.get_user_profiles_by_ids() and
obtain user's bots profiles in actions.get_service_dicts_for_bots() by
users.user_ids_to_users() instead of models.get_user_profiles_by_ids().
Fixes #8939
2018-04-04 16:24:55 -07:00
novokrest a613f3c0a1 get_owned_bot_dicts: Optimize admin's bot dicts loading from database.
Implement few optimizations for reading admin's bot dicts from database
for a constants number of requests:
- add models.get_user_profiles_by_ids() for reading bots profiles
  by single query from database
- add models.get_services_for_bots() for reading services for bots
  by single query from database
- add bot_config.get_bot_configs() for reading config data for bots
  by single query from database

Fixes #8838
2018-04-03 09:10:32 -07:00
novokrest ad3ea775df actions.py: Rename get_service_dicts_for_bots() to get_service_dicts_for_bot() 2018-04-02 11:18:50 -07:00
Shubham Padia 10115491a3 real-time sync: Server sends recipent data with delete_message event.
Fixes #8853.
In certain cases, the browser is not able to look up the message.
Include the recipient data for the message in the delete_message event,
so look up of those attributes by the browser isn't required.
2018-04-01 22:19:08 -07:00
Umair Khan f6fb88549f profile: Send operation in the event.
This allows us to show a useful message in the handler when the
event is received.
2018-03-29 13:09:11 -07:00
Tim Abbott 656f882a44 bots: Eliminate NEW_USER_BOT.
This bot was basically a duplicate of NOTIFICATION_BOT for some
specific corner cases, and didn't add much value.  It's better to just
eliminate it, which also removes some ugly corner cases around what
happens if the user account doesn't exist.
2018-03-29 12:01:21 -07:00
Eeshan Garg 538746fc65 webhooks: Stop raising an exception if stream does not exist.
webhook-errors.log file is cluttered with Stream.DoesNotExist
errors, which hides the errors that we actually need to see. So,
since check_message already sends the bot_owner a PM if the webhook
bot tries to send a message to a non-existent stream, we can ignore
such exceptions.
2018-03-24 13:50:17 -07:00
Tarun Kumar 86eaa553a6 user-groups: Add backend enforcing for new modification settings.
Add function in user-groups.py for getting member ids
for a group.
Update view to enforce checks for modifying user-groups.
Only admins and user group members can modify user-groups.
2018-03-23 14:44:47 -07:00
neiljp (Neil Pilgrim) 97836c4bac mypy: Remove Optional & Mutable from do_schedule_message.
Removes various mypy errors.
2018-03-23 11:32:00 -07:00
Sarah f5c2fb8438 org settings: Create backend api for allow_community_topic_editing.
Adds the code for updating the allow_community_topic_editing
setting.
2018-03-22 16:02:27 -07:00
Jack Weatherilt d857f26cd3 refactoring: Remove unused assignment on ensure_stream.
There were two instances of `ensure_stream` being called and assigned to
a variable with the variable not being used elsewhere. pyflakes picked
up on this (where it didn't in the previous version likely due to tuple
unpacking), so the the variable assignment has been replaced with a call
to `ensure_stream`.
2018-03-21 16:47:52 -07:00
Jack Weatherilt 3396cfc2ef refactoring: Replaced occurences of create_stream_if_needed.
Issue #2088 asked for a wrapper to be created for
`create_stream_if_needed` (called `ensure_stream`) for the 25 times that
`create_stream_if_needed` is called and ignores whether the stream was
created. This commit replaces relevant occurences of
`create_stream_if_needed` with `ensure_stream`, including imports.

The changes weren't significant enough to add any tests or do any
additional manual testing.

The refactoring intended to make the API easier to use in most cases.
The majority of uses of `create_stream_if_needed` ignored the second
parameter.

Fixes: #2088.
2018-03-21 16:47:36 -07:00
YJDave 11c995b70f custom profile data: Send event to active user on update.
On update of custom profile fields, send an event to all
active users of realm.
2018-03-21 16:08:12 -07:00
Harshit Bansal a49655e0d4 emoji: Migrate realm emoji to be addressed by `id` rather than `name`.
This commit migrates realm emoji to be addressed by their `id` rather
than their name. This fixes a long standing issue which was causing
an error on uploading an emoji with same name as a deactivated realm
emoji.

Fixes: #6977.
2018-03-20 22:24:44 -07:00
Tim Abbott 51018e12f5 message_edit: Remove obsolete block for diff highlighting.
We disabled the original "colorized HTML edit-history" feature way
back in 2013 in c51056ff8e.

That original feature involved showing what had been edited inline in
message bodies, so one could easily see what had been changed.

That old feature has since been replaced with the "view edit history"
menu option, and we're unlikely to ever want the old feature back.
So, we can just remove its code.  There's a few supporting variables
that were created to help implement this; we can clean those up and
simplify the `update_message` code now that this feature is fully
removed.
2018-03-20 15:22:53 -07:00
Greg Price 22071a44a7 antispam: Add a sitewide ratelimit on invites by new realms.
This applies only on a server open for anyone to create a realm.
Moreover, if the server admins have granted any given realm a
max_invites greater than the default, that realm is exempt too.
2018-03-16 18:00:11 -07:00
Tim Abbott c147d2e140 notifications: Fix missed-message emails for presence-idle users.
This fixes an unpleasant regression in
f5edeb01ae, where we stopped correctly
filtering users who have an open browser session that's idle.  These
users are tagged as "UserPresence.IDLE" with an current timestamp in
the database, and should be treated as idle for presence purposes.

As a result, if you had an open Zulip browser session, you incorrectly
wouldn't get missed-message emails for PMs and mentions before this fix.
2018-03-16 16:30:23 -07:00
YJDave 0281079a39 stream settings: Send all private streams subs to realm admins on load.
Tweaked by tabbott to simplify the conditionals in actions.py.

Fixes #8695.
2018-03-16 12:26:56 -07:00
YJDave 72a440a86d stream settings: Fix error in real time sync in subs add/remove event.
Currently, when other private stream subscriber add realm admin to
stream, new copy private stream is created in realm admin's streams.
Which resulted in error, cause there are two similar stream element
in stream settings.

If new subscriber is added to private stream, we first send them
stream `create` event, cause private stream are not visible until
user don't get subscribed at least once. But realm admins can now
always access private stream, so when realm admin is subscribed to
stream, realm admin get stream `create` event even if stream already
exist in on realm admin client side.

Fix this by extracting realm admins from stream `create` event on
`add` subscription operation and sending private stream `create`
event to all realm admins on stream creation operation.

Fixes #8695
2018-03-16 12:22:06 -07:00
Eeshan Garg af56df7723 webhooks: Enable custom topics and default PM notifications.
This commit adds a generic function called check_send_webhook_message
that does the following:
* If a stream is specified in the webhook URL, it sends a stream
  message, otherwise sends a PM to the owner of the bot.
* In the case of a stream message, if a custom topic is specified
  in the webhook URL, it uses that topic as the subject of the
  stream message.

Also, note that we need not test this anywhere except for the
helloworld webhook. Since helloworld is our default example for
webhooks, it is here to stay and it made sense that tests for a
generic function such as check_send_webhook_message be tested
with an actual generic webhook!

Fixes #8607.
2018-03-16 11:34:20 -07:00
Vishnu Ks 951b88dd30 models: Make email_allowed_for_realm raise exception. 2018-03-15 14:35:24 -07:00
Tim Abbott 1a3ae8a376 stream settings: Fix inadvertently merged broken whitespace.
I'm not sure why the pre-commit linter didn't flag this for me; maybe
it did and I just missed it.
2018-03-13 15:12:45 -07:00
YJDave 2031118545 stream settings: Allow realm admins to access all private stream subs.
This will allow realm admins to access subscribers of unsubscribed
private stream.  This is a preparatory commit for letting realm admins
remove those users.
2018-03-13 14:59:09 -07:00
neiljp (Neil Pilgrim) 9e1dbde82d mypy: Final small migrations to python3.5 annotations in many files. 2018-03-12 11:23:30 -07:00
Tim Abbott ef92fcbe2b topic history: Fix fetching topic history of public streams.
Apparently, we did essentially all the work to support showing full
topic history to newly subscribed users from a data flow perspective,
but didn't actually enable this feature by having the topic history
endpoint grant access to historical topics.  This fixes that gap.

I'm not altogether happy with how the code and tests read for this
feature; the code itself has more duplication than I'd like, and the
tests do too, but it works.
2018-03-11 20:59:20 -07:00
YJDave 382ac24151 create stream: Fix real time sync bug in private stream creation.
If new private stream is created by realm admin without realm admin
subscribed to it, then it doesn't automatically add created stream to
realm admin's stream list. We have to reload the browser to get newly
created stream in stream list. Cause private stream creation event is
only sent to the subscribed users to private stream, so even if realm
admin is acting user, they don't get creation event.

We should send private stream creation event to realm admin users along
with subscribed user to stream, as realm admins can access unsubscribed
private streams.

Tweaked by tabbott to fix various typos and clean up the code.
2018-03-09 18:35:12 -08:00
Shubham Padia 32e38d36aa bots: Send add/delete event on bot ownership change.
Adds realm_bot delete event. On bot ownership change, add event is
sent to the bot_owner(if not admin) and delete event to the
previous bot owner(if not admin). For admin, update event is sent.
2018-03-08 07:54:19 -08:00
Tim Abbott c3964dff6e i18n: Fix a last few strings mentioning realms. 2018-03-07 17:15:29 -08:00
Tim Abbott 9acf30b3d3 display_settings: Fix real-time sync of default language.
We apparently weren't sending to the frontend the data needed to
update the display of the current language.
2018-03-04 19:23:58 -08:00
Yago González b475e74c47 i18n: Fix string interpolation. 2018-03-04 15:44:12 -08:00
Tim Abbott 038579b840 streams: Fix counts using only one data point.
We had the aggregation query for StreamCount done slightly wrong.
2018-03-04 14:06:29 -08:00
Robert Hönig e69af3011e Add basic test for FeedbackBot.
This sets up a new test class with a simple
test, mostly for increasing coverage. The class
should in the future be extended to properly
verify the handle_feedback() logic.
2018-03-04 13:31:33 -08:00
Archana BS c7650c0d9d streams: send data for recent streams traffic to frontend.
Significantly edited by Tim Abbott and Vishnu Ks.
2018-03-04 13:24:53 -08:00
Robert Hönig 4c7dd971d0 embedded bots: Dispatch all config data on update event.
Previously, when a user updated the config data of an
embedded bot, only the updated fields were dispatched
back to the client. Dispatching all config data fields
makes the client updating code less brittle.
2018-03-04 13:11:23 -08:00