Commit Graph

562 Commits

Author SHA1 Message Date
Tim Abbott 1fe4f795af settings: Add notification settings checkboxes for wildcard mentions.
This change makes it possible for users to control the notification
settings for wildcard mentions as a separate control from PMs and
direct @-mentions.
2019-11-20 16:58:46 -08:00
Matheus Melo c96762b7a9 settings: Add setting for who can edit user groups.
Fixes #12380.
2019-11-03 16:45:13 -08:00
Mateusz Mandera 5aded51b73 register: Pre-populate Name in social backend flow.
By adding some additional plumbing (through PreregistrationUser) of the
full_name and an additional full_name_validated option, we
pre-populate the Full Name field in the registration form when coming
through a social backend (google/github/saml/etc.) and potentially skip
the registration form (if the user would have nothing to do there other
than clicking the Confirm button) and just create the account and log
the user in.
2019-11-03 16:15:48 -08:00
Mateusz Mandera 4dc3ed36c3 auth: Add initial SAML authentication support.
There are a few outstanding issues that we expect to resolve beforce
including this in a release, but this is good checkpoint to merge.

This PR is a collaboration with Tim Abbott.

Fixes #716.
2019-10-10 15:44:34 -07:00
Rishi Gupta e10361a832 models: Replace is_guest and is_realm_admin with UserProfile.role.
This new data model will be more extensible for future work on
features like a primary administrator.
2019-10-06 16:24:37 -07:00
Rishi Gupta 4256ee61cf billing: Change RealmAuditLog.event_type from str to int.
This is a more robust long-term model for storing these data.
2019-10-06 15:55:56 -07:00
Mateusz Mandera dbe508bb91 models: Migration of Message.pub_date to date_sent, part 2.
Fixes #1727.

With the server down, apply migrations 0245 and 0246. 0246 will remove
the pub_date column, so it's essential that the previous migrations
ran correctly to copy data before running this.
2019-10-05 19:01:34 -07:00
Mateusz Mandera f198475281 models: Migration of Message.pub_date to date_sent, part 1.
1. Apply migration 0243 to add date_sent column.
2. Apply migration 0244 to copy pub_date over to date_sent. Can be done
with the server running.
3. With the server down (for consistency between memory and
database state of Django objects), verify consistency with
Message.objects.exclude(date_sent=F("pub_date")).count() == 0
2019-10-05 19:01:34 -07:00
Tim Abbott 8024b1179a bots: Fix bot email addresses with EMAIL_ADDRESS_VISIBILITY_ADMINS.
When using our EMAIL_ADDRESS_VISIBILITY_ADMINS feature, we were
apparently creating bot users with different email and delivery_email
properties, due to effectively an oversight in how the code was
written (the initial migration handled bots correctly, but not bots
created after the transition).

Following the refactor in the last commit, the fix for this is just
adding the missing conditional, a test, and a database migration to
fix any incorrectly created bots leaked previously.
2019-09-23 15:53:53 -07:00
Mateusz Mandera 6bd34afa0d models: Migration of UserMessage.id to bigint, part 2.
This requires part 1 (which can take hours to run but generally
doesn't require downtime) to be completed first.

This portion of the migration will require the server to be completely
down for a brief period; for chat.zulip.org with 250M UserMessage
rows, it took about 60s to run; that time will vary depending on
hardware details like whether the server has an SSD, but fundamentally
shouldn't be long.

Our upgrade-zulip and upgrade-zulip-from-git tools can apply this
migration correctly; nothing special needs to be done.

Fixes #13040.
2019-08-26 21:25:19 -07:00
Mateusz Mandera b008515d63 models: Migration of UserMessage.id to bigint, part 1.
As part of adding support for more than 2B UserMessage rows in a Zulip
server, we need to change UserMessage.id (a field we don't access but
is needed by Django) from an int to a bigint.  This commit is a series
of migrations which create a `bigint_id` column and populates it correctly.

This migration will take a long time to run; on chat.zulip.org (a
server with a lot of history), it took about 4 hours to complete.

How to migrate with minimal downtime:

1. Run `upgrade-zulip-from-git` through this commit.  It will install
migration 0238 and then more or less hang while applying migration
0239.  Once migration 0238 is completed, however, your server should
be able to be started back up safely while migration 0239 is running.

2. Run `/home/zulip/deployments/next/scripts/restart-server` in a
separate terminal to get Zulip running again.

3. When the `upgrade-zulip-from-git` command finishes, it will
automatically re-restart the Zulip server, leaving you in a consistent
state and ready to do part 2 of the migration.

A useful `manage.py shell` query for checking the state after this
commit is consistent is this:

    assert UserMessage.objects.exclude(bigint_id=F("id")).count() == 0

Part of #13040.
2019-08-26 21:14:15 -07:00
Anders Kaseorg 5aeb562700 migrations.0001_initial: Expand \n in long SQL string.
One occasionally finds that a 1580 character string of SQL queries
might not most readably be presented on a single line.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-21 11:00:38 -07:00
Mateusz Mandera 9b2b24c617 production: Rename the 'zulip' system bot realm to 'zulipinternal'. 2019-07-31 16:01:36 -07:00
Wyatt Hoodes b01ba48369 migration_0060: Remove unreachable return statement.
This was introduced in a past refactoring.
2019-07-29 15:23:34 -07:00
Wyatt Hoodes a75abe096b migration_0064: Fix typing for file_key. 2019-07-29 15:23:10 -07:00
Wyatt Hoodes e106f95c8b migration_0064: Remove usage of Text type. 2019-07-29 15:23:10 -07:00
Wyatt Hoodes b3323366b3 migration_0077: Migrate from Text to str.
In `force_str` we assume that python 2 strings should be
considered.  This is no longer the case, so we replace all
occurences of `Text` with `str`, and remove the unreachable
condition.

(Probably further cleanup is possible, but this code shouldn't be
modified again in any case).
2019-07-25 17:54:26 -07:00
Conner Bondurant c25dcf048d models: Enforce stricter requirements on the full_name field.
This changes the requirements for UserProfile to disallow some
additional characters, with the overall goal of being able to use
formataddr in send_mail.py.

We don't need to be particularly careful in the database migration,
because user full_names are not required to be unique.
2019-07-22 18:13:34 -07:00
David Wood 9bace3f2cd notifications: Allow only notifiable in unread count.
This commit adds a new setting to the user's notification settings that
will change the behaviour of the unread count in the title bar and
desktop application.

When enabled, the title bar will show the count of unread private messages
and mentions. When disabled, the title bar will act as before, showing
the total number of unread messages.

Fixes #1736.
2019-07-13 15:49:04 -07:00
Yashashvi Dave d7ee2aced1 models: Add `external_account` in custom profile field types.
Add new custom profile field type, External account.
External account field links user's social media
profile with account. e.g. GitHub, Twitter, etc.

Fixes part of #12302
2019-07-09 17:21:54 -07:00
Mateusz Mandera b8839302c0 retention: Fix migration making archive_transaction field not null.
DELETing from archive tables and ALTERing ArchivedMessage needs to be
split into separate transactions.

zerver_archivedattachment_messages needs to be cleared out before
zerver_archivedattachment.
2019-07-04 16:42:42 -07:00
vinitS101 04f3fce761 ldap: Fix LDAP avatar synchronization to check if avatar has changed.
When "manage.py sync_ldap_user_data" is run, user avatars are now only
updated if they have changed in LDAP.

Fixes #12381.
2019-07-02 17:52:48 -07:00
Mateusz Mandera 74ba0a7082 retention: Make archive_timestamp field not nullable. 2019-07-02 17:25:31 -07:00
Mateusz Mandera a2cce62c1c retention: Use new ArchiveTransaction model.
We add a new model, ArchiveTransaction, to tie archived objects together
in a coherent way, according to the batches in which they are archived.
This enables making a better system for restoring from archive, and it
seems just more sensible to tie the archived objects in this way, rather
the somewhat vague setting of archive_timestamp to each object using
timezone_now().
2019-06-26 12:05:59 -07:00
Anders Kaseorg 6112d020f4 migrate, create_large_indexes: Use CREATE INDEX IF NOT EXISTS.
We no longer support Postgres versions missing this syntax.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 11:01:30 -07:00
Yashashvi Dave 8e269b4651 models: Rename notification to `enable_stream_audible_notifications`.
Rename notification property `enable_stream_sounds` to
`enable_stream_audible_notifications` to match with other
notification property patterns.

Fixes part of #12304
2019-06-12 16:24:51 -07:00
Mateusz Mandera d481ee9a40 retention: Add message_retention_days field to Stream model. 2019-06-06 11:17:42 -07:00
sahil839 5a130097bf settings: Add display setting for demoting inactive streams.
This adds a setting to control Zulip's default behavior of sorting to
bottom and graying out inactive streams.  The previous logic is still
the default "automatic", but this gives users more control.  See the
models.py comment for details.

Fixes #11524.
2019-06-03 23:07:56 -07:00
Puneeth Chaganti 8c0c9ca7a4 url preview: Turn Realm.inline_url_embed_preview off by default. 2019-05-31 15:28:32 -07:00
Mateusz Mandera 29529cf2e7 retention: Add ArchivedSubMessage model. 2019-05-29 16:26:11 -07:00
Mateusz Mandera 292b4bb0d7 retention: Add ArchivedReaction model. 2019-05-29 16:26:11 -07:00
Yashashvi Dave cb85ca8601 models: Alter video_chat_provider field type to integer.
Migration rewritten by tabbott because it did not work.
2019-05-13 12:02:28 -07:00
Roman Godov a50824e031 models: Rename Subscription.in_home_view field to is_muted.
This renames Subscription.in_home_view field to is_muted, for greater
clarity as to what it does just from seeing the setting name, without
having to look it up.

Also disabled an obsolete test_migrations test.

Fixes #10042.
2019-05-12 22:08:10 -07:00
Pragati Agrawal a838de63d8 settings: Add option for fixed/fluid width.
This adds a new option for "fluid width" under `Display settings` section
of SETTINGS/DISPLAY SETTINGS tab.

Fixes: #11844.
2019-05-09 13:57:57 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
Puneeth Chaganti d474a41c03 digest: Turn off digest_emails_enabled flag for realms by default. 2019-05-08 14:39:12 -07:00
David Wood 34d810aac3 settings: Migrate to create_stream_policy structure.
This commit replaces the `create_stream_by_admins_only` setting with a
new `create_stream_policy` setting, which mirroring the structure of
the existing `invite_to_stream_policy`.

This is important preparation for migrating the waiting period feature
to be its own independent setting.

Fixes #12236.
2019-05-06 16:27:55 -07:00
Shubham Padia 4994a6c2c9 realm: Add setting to disable avatar changes.
This is useful when syncing avatars from an integrated LDAP/active
directory.

The upload avatar and delete avatar buttons are hidden if avatar
changes are disabled and the user is a non-admin.
If the user has a gravatar set, then the user will not be able to
upload an image as their avatar if avatar changes are disabled.

Part of #12132.
2019-05-03 12:52:43 -07:00
David Wood 272ed90685 settings: Create an explicit invite_to_stream_policy setting.
This commit creates a new organization setting that determines whether
a user can invite other users to streams. Previously this was linked
to the waiting period threshold, but this was both not documented and
overly limiting.

With significant tweaks by tabbott to change the database model to not
involve two threshhold fields, edit the tests, etc.

This requires follow-up work to make the create stream policy setting
work how this code implies it should.

Fixes #12042.
2019-04-29 17:11:28 -07:00
Puneeth Chaganti f3d0ccb9f6 models: Add a digest_weekday field to the Realm model.
Allow realms to specify the day of the week when the digest should be sent out.
When enqueue-ing digests, pick only the realms that chose the current weekday as
the day to send out digests.
2019-04-27 15:20:41 -07:00
Mateusz Mandera a269c4bdd4 email_mirror: Make email_token a unique column of Stream. 2019-03-17 12:55:35 -07:00
Raymond Akornor 89351cdd19 send_email: Add ScheduledEmail support for multiple recipients.
Follow up on 92dc363. This modifies the ScheduledEmail model
and send_future_email to properly support multiple recipients.

Tweaked by tabbott to add some useful explanatory comments and fix
issues with the migration.
2019-03-15 11:02:12 -07:00
Challa Venkata Raghava Reddy b69aec2dbc streams: Add first_message_id tracking first message in stream.
This field is primarily intended to support avoiding displaying the
"more topics" feature in new organizations and streams, where we might
know that all messages in the stream are already available in the
browser.

Based on original work by Roman Godov, and significantly modified by
tabbott.

The second migration involved here could be expensive on Zulip Cloud,
but is unlikely to be an issue on other servers.
2019-03-11 13:30:49 -07:00
Hemanth V. Alluri ae126c452b stream-descriptions: Create wrapper for rendering stream descriptions.
In commit de65a04 we can see that if the need ever arises to modify
how stream descriptions are rendered, we would need to make changes
at 5 different call points which can be quite cumbersome. So this
functionality has been extracted to a new method called
'render_stream_descriptions'.
2019-03-06 17:16:14 -08:00
Tim Abbott de65a04ae0 streams: Disable inline URL preview when rendering stream descriptions.
We want to use the baseline features of bugdown, but not fancy things
like inline URL previews, since the whole structure of stream
descriptions is to have a single-line thing supporting some
formatting.

The migration part of this change fixes a bug encountered by some
organizations upgrading from older versions of Zulip.
2019-02-28 17:00:40 -08:00
Anders Kaseorg 649235cfec python: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-22 16:54:36 -08:00
sahil839 7157edf4af settings: Add support for uploading logo for night mode.
This adds a new field named realm_night_logo which is used for
displaying the organization logo when the user is in night mode.

Fixes #11176.
2019-02-18 15:15:57 -08:00
Vishnu Ks 28769e040f invites: Add backend for multiuse admin invites. 2019-02-07 15:41:00 -08:00
Anders Kaseorg 4bd28f7ae6 migrations: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:01:04 -08:00
Hemanth V. Alluri 73d26c8b28 streams: Render and store the stream description from the backend.
This commit does the following three things:
    1. Update stream model to accomodate rendered description.
    2. Render and save the stream rendered description on update.
    3. Render and save stream descriptions on creation.

Further, the stream's rendered description is also sent whenever the
stream's description is being sent.

This is preparatory work for eliminating the use of the
non-authoritative marked.js markdown parser for stream descriptions.
2019-02-01 22:24:18 -08:00
Rishi Gupta 8df0d95559 billing: Remove deprecated RealmAuditLog.requires_billing_update. 2019-02-01 18:49:43 -08:00
Rishi Gupta 4fab08172b billing: Remove deprecated billing fields from Realm. 2019-01-31 15:12:43 -08:00
Pragati Agrawal 1e811b42ec org settings: Add realm level setting for missed message content in email.
This adds a setting under "Notification" section of
"Organization settings" tab, which enables Organization administrator to
control whether the missed message emails include the message content or
not.

Fixes: #11123.
2019-01-25 14:34:10 -08:00
Steve Howell 7e904c3a1a user status: Add status_text field to the database.
The status_text field will be something like "out to lunch."
2019-01-24 09:46:35 -08:00
Marco Burstein 9ddadd39f4 compose: Add support for using Zoom as the video chat provider.
This adds Zoom call properties to the `Realm` model, creates endpoints
for creating calls, adds a frontend and tests.

Fixes #10979.
2019-01-07 10:00:02 -08:00
Shubham Dhama 42c262b807 invite: Replace `invite_as_admin` usage with `invite_as`.
Since we have already added the `invite_as` field to models, we can now
replace usage of `invite_as_admin` properly with its equivalent `invite_as
== PreregistrationUser.INVITE_AS['REALM_ADMIN']`.

Hence, also removed now redundant `invite_as`.
2019-01-05 14:46:38 -08:00
Shubham Dhama c9414a3572 invite: Rename NORMAL_USER to MEMBER of Preregistration.INVITE_AS.
Renamed to keep the user-facing and the codebase terminologies coherent.
2019-01-05 14:43:06 -08:00
Steve Howell a8301ca14a status: Add UserStatus model and core library for away status. 2019-01-02 09:12:03 -08:00
Shubham Dhama 235b0db82b preregistrationuser: Add a model field for all type of invited users.
This adds a model field `invited_as` to PreregistrationUser model class
which will replace `invited_as_admin` in future. Intentions behind adding
this are that we can specify "types" of users for an invited person other
than admin or regular that is, guest user or maybe many others in the
future.
2018-12-30 10:16:17 -08:00
seresheim 49dbd85a89 auth: Add support for Azure Active Directory authentication.
This takes advantage of all of our work on making the
python-social-auth integration reusable for other authentication
backends.
2018-12-18 16:39:03 -08:00
Joshua Pan ad1df0ebeb settings: Add support for customizing the top-left logo.
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.

Tweaked significantly by tabbott to rebase, fix styling, etc.

Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.

Fixes #7995.
2018-12-18 12:44:52 -08:00
Vishnu Ks 8a1794caa3 message: Store the value of first_visible_message_id in Realm table.
This eliminates a bunch of potentially buggy caching code, with no
material negative side effects.
2018-12-12 15:11:17 -08:00
Marco Burstein ba46dc83c6 notifications: Add a setting for changing the notification sound.
Also, add a new notification sound, "ding". It comes from
https://freesound.org, where the original Zulip notification sound comes
from as well. In the future, new sounds can be added by adding audio
files to the `static/audio/notification_sounds` directory.

Tweaked significantly by tabbott:
* Avoided removing static/audio/zulip.ogg, because that file is
  checked for by old versions of the desktop app.
* Added a views check for the sound being valid + tests.
* Added additional tests.
* Restructured the test_events test to be cleaner.
* Removed check_bool_or_string.
* Increased max length of notification_sound.
* Provide available_notification_sounds in events data set if global
  notifications settings are requested.

Fixes #8051.
2018-12-09 21:25:30 -08:00
Tim Abbott 9d32857c37 realm: Add new Realm.email_address_visibility field.
This is preparatory work for settings controlling who can see user
emails; it includes the API-level support for editing it, but no code
to actually enforce the policy.
2018-12-06 15:56:30 -08:00
Hemanth V. Alluri 6b1d724f5c zerver: Add bugdown rendering for text custom profile fields.
This is the first step of letting users use Zulip markdown in their
SHORT_TEXT and LONG_TEXT custom profile fields, so that they can
include emphasis, links, etc.

This doesn't include any frontend logic yet, however.
2018-11-29 09:47:32 -08:00
Tim Abbott c679920c01 python: Fix unnecessary uses of str_utils library. 2018-11-27 11:44:09 -08:00
Rishi Gupta 6b637428a7 billing: Add Realm.seat_limit.
Currently unenforced, and will mostly need to be set by hand.
2018-11-20 18:50:43 -08:00
Tim Abbott e221a01638 Revert "Change if(realm.domain == mit.edu) to use Realm.is_zephyr_..."
This reverts commit 3645bb9225.

This change was incorrect, because the `is_zephyr_mirror_realm`
property on Realm is a property and thus isn't available in the
migration codebase.

Since this migration is only run for very old servers, this should
have no impact.
2018-11-02 13:16:45 -07:00
Tim Abbott c57c4cf703 notifications: Fix push notifications with multiple realms.
Previously, Zulip did not correctly handle the case of a mobile device
being registered with a push device token being registered for
multiple accounts on the same server (which is a common case on
zulipchat.com).  This was because our database `unique` and
`unique_together` indexes incorrectly enforced the token being unique
on a given server, rather than unique for a given user_id.

We fix this gap, and at the same time remove unnecessary (and
incorrectly racey) logic deleting and recreating the tokens in the
appropriate tables.

There's still an open mobile app bug causing repeated re-registrations
in a loop, but this should fix the fact that the relevant mobile bug
causes the server to 500.

Follow-up work that may be of value includes:
* Removing `ios_app_id`, which may not have much purpose.
* Renaming `last_updated` to `data_created`, since that's what it is now.

But none of those are critical to solving the actual bug here.

Fixes #8841.
2018-10-10 16:15:52 -07:00
Tim Abbott 510afd0c93 migrations: Disable atomic for delivery_email migration.
I'm not sure theoretically why this should be required only for some
installations, but these articles all suggest the root problem is
doing these two migrations together atomically (creating the field and
setting a value for it), so the right answer is to declare the
migration as not atomic:

https://stackoverflow.com/questions/12838111/django-db-migrations-cannot-alter-table-because-it-has-pending-trigger-events
https://confluence.atlassian.com/confkb/upgrade-failed-with-the-error-message-error-cannot-alter-table-content-because-it-has-pending-trigger-events-747606853.html
2018-09-28 14:14:20 -07:00
Tim Abbott 42f49d19b3 emoji: Set default emoji to google-blob for existing users too.
This fixes an inconsistent test failure with test_users.py (that
depended on the ordering between this migration and the creation of
test database users like hamlet).
2018-08-31 15:45:33 -07:00
Harshit Bansal e7311cdf5d emoji: Bring back the feature of changing emojisets.
This is largely inspired by requests from people not liking the
Google's new emojiset. A lot of people were requesting to revert
back to old blobs emojiset so we are re-enabling this feature
after making relevant infrastructure changes for supporting google's
old blob emojiset and re-adding support for twitter emojiset.

Fixes: #10158.
2018-08-31 11:52:24 -07:00
Pragati Agrawal ba4e4e38ae user settings: Add setting to make login notifications optional.
This adds a feature in the "Notification" section of "Settings" tab,
which lets user enable or disable login emails notification.

Tweaked by tabbott to simplify the test.

Fixes: #5795, progress towards #5854.
2018-08-28 14:02:31 -07:00
Vishnu Ks dcb7b15069 billing: Use UserProfile.is_billing_admin instead of Customer.billing_user. 2018-08-27 12:35:46 -07:00
Joshua Pan 79e2266d6e models: Create starred_message_counts boolean field. 2018-08-21 13:42:22 -07:00
Vishnu Ks 9bb338be11 models: Add plan_type to Realm. 2018-08-21 12:39:06 -07:00
Yashashvi Dave d5f3c9b3bc custom profile field: Rename field types in UI. 2018-08-21 11:37:51 -07:00
Tim Abbott 6827c42e02 migrations: Fix accounting in is_private migration.
The previous logic didn't correctly handle the case of total being 0.

Fixes #10378.
2018-08-20 16:49:08 -07:00
Yashashvi Dave 868362046b zerver/models.py: Change custom field name max length to 40 char. 2018-08-17 08:36:22 -07:00
Shubham Padia 40fdc4724b migrations: Set initial value for is_private flags.
This initializes the is_private flag to have the correct value for
historical messages.
2018-08-09 16:09:44 -07:00
Harshit Bansal 9057e543ac emoji: Disable support for letting users switch emojisets.
Due to copyright issues with potentially displaying Apple emojisets on
non-apple devices, as well as iamcal dropping support for the emojione
emojiset (see https://github.com/iamcal/emoji-data/pull/142), we are
dropping (perhaps temporarily) support for allowing users to switch
emojisets in Zulip.

This commit just hides the feature from the user but leaves most of
the infrastructure in place so that in the future if we decide to
re-enable the support we will not need to redo the infrastructure work
(some JS-side code is deleted, mostly because we'll want to re-add the
feature using the do_settings_change infrastructure anyway).

The most likely emoji set to add is the legacy "blobs" Google emoji
set, since it seems popular with some users.

Tweaked by tabbott to remove some additional JS code and update the
changelog.
2018-08-08 15:48:04 -07:00
Tim Abbott 35cb7528f9 models: Add new UserMessage flag active_mobile_push_notification.
This flag is used to track which user/message pairs correspond to an
active mobile push notification, that should potentially be cleared
when the user reads the message.

This flag should never appear on a message that is also marked as
read; eventually we may want a cron job to check for that condition.

We include a partial index on UserMessage for this flag.
2018-08-01 16:51:56 -07:00
Roman Godov 5e70577f84 models: Rename Realm.show_digest_email field.
This renames Realm.show_digest_email field to
digest_emails_enabled, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-08-01 11:05:58 -07:00
Roman Godov c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Shubham Padia bf6dc4472b models: Add is_private flag to UserMessage and add index for it.
The is_private flag is intended to be set if recipient type is
'private'(1) or 'huddle'(3), otherwise i.e if it is 'stream'(2), it
should be unset.

This commit adds a database index for the is_private flag (which we'll
need to use it). That index is used to reset the flag if it was
already set. The already set flags were due to a previous removal of
is_me_message flag for which the values were not cleared out.

For now, the is_private flag is always 0 since the really hard part of
this migration is clearing the unspecified previous state; future
commits will fully implement it actually doing something.

History: Migration rewritten significantly by tabbott to ensure it
runs in only 3 minutes on chat.zulip.org.  A key detail in making that
work was to ensure that we use the new index for the queries to find
rows to update (which currently requires the `order_by` and `limit`
clauses).
2018-07-30 15:43:55 -07:00
Roman Godov 34ae3dfd44 models: Delete unused Subscription.notifications field.
This deletes the unused Subscription.notifications field and removes
it from some testing and analytics code (which should not have been
using it in the first place).

Fixes #10042.
2018-07-26 15:54:57 -07:00
Tim Abbott e74c37a129 migrations: Convert migration 0041 to not import from zerver.
This should help avoid problems when upgrading from very old versions of Zulip.
2018-07-24 09:00:14 -07:00
Vishnu Ks 2b28042ddf models: Rename remaining audit log event types to past tense.
This makes the realm audit event type log entries more consistent.
2018-07-22 20:00:28 -07:00
Joshua Pan 533eccd655 models: Create delivery_email field in userprofile.
This commit creates a new field called delivery_email. For now, it is
exactly the same as email upon user profile creation and should stay
that way even when email is changed, and is used only for sending
outgoing email from Zulip.

The purpose of this field is to support an upcoming option where the
existing `email` field in Zulip becomes effectively the user's
"display email" address, as part of making it possible for users
actual email addresses (that can receive email, stored in the
delivery_email field) to not be available to other non-administrator
users in the organization.

Because the `email` field is used in numerous places in display code,
in the API, and in database queries, the shortest path to implementing
this "private email" feature is to keep "email" as-is in those parts
of the codebase, and just set the existing "email" ("display email")
model field to be something generated like
"username@zulip.example.com" for display purposes.

Eventually, we'll want to do further refactoring, either in the form
of having both `display_email` and `delivery_email` as fields, or
renaming "email" to "username".
2018-07-12 12:30:20 +05:30
Rishi Gupta b5753d0ddc billing: Add initial support for seat based plans.
The main remaining todo for correctly populating
RealmAuditLog.requires_billing_update is supporting the de-seating (and
corresponding re-seating) that happens after being offline for two weeks.
2018-07-09 14:33:08 +05:30
Tim Abbott 7ccefc3e5d migrations: Remove dependence on PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS.
We don't want to keep around a declaration of
PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS forever, so we should just move
this to a getattr; if the user has set it on their server, we'll use
the value; otherwise, we just use False.
2018-05-29 07:36:53 -07:00
Yashashvi Dave e82c879b85 custom fields: Add user type of custom fields.
Fixes #8878
2018-05-27 23:01:21 -07:00
Tim Abbott 6967b6519d settings: Add a development-only setting for less dense mode.
This should make it easier for us to iterate on a less-dense Zulip.

We create two classes on body, less_dense_mode and more_dense_mode, so
that it's easy as we refactor to separate the two concepts from things
like colors that are independent.
2018-05-24 12:31:37 -07:00
Tim Abbott 3853e133f4 migrations: Add missing reverse_code for migration 0167. 2018-05-21 09:42:19 -07:00
Steve Howell ff097623fa Add SubMessage table. 2018-05-16 15:13:33 -07:00
Tim Abbott f0ef335412 models: Remove unused ModelReprMixin class.
It appeared to be used as a base class in various Django migrations,
but because it didn't define any model fields, it wasn't actually.
2018-05-15 19:11:22 -07:00
Shubham Padia a5759108d3 models: Add field is_announcement_only to stream. 2018-05-13 09:06:20 -07:00
Yago González 6837fc5d56 i18n: Add missing strings for custom profile fields and fix capitalization.
The "Short/Long Text" option for custom profile fields wasn't properly
capitalized (i.e. "Text" should have been all lowercase), and also
wasn't properly tagged for translation.

For the sake of consistency, the change to proper capitalization has
also been applied to the models and any tests involving this feature.

Due to a bug in Django, it complained about the models having changed
and thus not being consistent with the migrations. That isn't actually
true (since the database stores the numeric values for each key), but
the migrations have been modified to avoid this error. This does not
affect the migrations' behaviour in any way.
2018-05-06 19:44:36 -07:00
Aditya Bansal 1f358954be web_public_streams: Add is_web_public to Stream table.
Also add function do_change_stream_web_public in lib/actions.py
to help in changing a streams web public status.
2018-05-02 15:23:33 -07:00
Angelika Serwa f4f64243dd custom_profile_fields: Support changing the sort order of the fields.
Tweaked by tabbott for variable naming and the URL.

Closes #8879.
2018-04-30 18:17:41 -07:00
Yashashvi Dave 0d7d94d0db custom fields: Add support for custom URL field type. 2018-04-30 10:53:23 -07:00
Yashashvi Dave 4033f210af custom fields: Add support for custom date field type. 2018-04-30 23:04:25 +05:30
Eeshan Garg 057ff9c91e models: Add Stream.history_public_to_subscribers.
This commit adds a new field history_public_to_subscribers to the
Stream model, which serves a similar function to the old
settings.PRIVATE_STREAM_HISTORY_FOR_SUBSCRIBERS; we still use that
setting as the default value for new streams to avoid breaking
backwards-compatibility for those users before we are ready with an
actual UI for users to choose directly.

This also comes with a migration to set the value of the new field for
existing streams with an algorithm matching that used at runtime.

With significant changes by Tim Abbott.

This is an initial part of our efforts on #9232.
2018-04-28 22:54:04 -07:00
Tim Abbott 7d6bb3dcb4 settings: Remove obsolete default_desktop_notifications setting.
This actually hasn't been hooked up to do anything in years.

While we're at it, we remove the entire "Zulip Labs" settings page.
2018-04-28 13:46:07 -07:00
Shubham Dhama 30b1ec9433 org settings: Change default allow_community_topic_editing value.
This changes the default value of allow_community_topic_editing to
`True` which was merged with `False` default value temporarily.
2018-04-28 13:00:29 -07:00
YJDave 6bef44a9fa org setting: Add time limits for message deleting.
Add realm setting to set time limit for message deleitng.
Set default value of message_content_delete_limit_seconds
to 600 seconds(10 min).

Thanks to Shubham Dhama for rebasing and reworking this.  Some final
edits also done by Tim Abbott.

Fixes #7344.
2018-04-27 19:22:28 -07:00
Umair Khan cf2f6b38dd profile: Add choice field.
Fixes part of #8878
2018-04-26 00:35:51 -07:00
Vishnu Ks c9e932a7ce settings: Add support for Hangouts as the video chat provider.
The only thing that's annoying about this feature is that you need to
be a paying G Suite customer to use it.
2018-04-23 09:39:47 -07:00
Vishnu Ks d1c143de42 models: Add attribute for video provider in realm. 2018-04-23 09:15:12 -07:00
Shubham Dhama 1191f1730a guest: Add a model field for guest users.
This adds new field `is_guest` to UserProfile model and
is meant for the new type of user i.e. "Guest Users".

(Part of #8385).
2018-04-20 16:08:29 -07:00
Umair Khan 0420b89468 profile: Add hint attribute to custom profile fields.
This is the model for #8876.
2018-04-19 11:32:35 -07:00
Shubham Dhama 0e6757af5c org settings: Change default realm description to empty string.
This fixes a traceback that users would get when editing the realm
description just after creating a new organization.
2018-04-06 15:15:47 -07:00
Tim Abbott 758d7b9146 bots: Clean up editing bots impacting non-bot users.
This fixes a bug where the endpoint for editing bot users would allow
an organization administrator to edit the full name of a bot user.

A combination of this an another recently fixed bug made it possible
for this process to set a `bot_owner` for a non-bot user; so we also
include a migration to fix that for any users that might have had our
model invariants corrupted in that way.
2018-04-02 18:58:26 -07:00
Umair Khan c30a282dd9 profile: Remove integer and float fields. 2018-04-02 09:46:21 -07:00
Shubham Dhama b7aae89029 org settings: Add realm level default time format setting. 2018-03-31 16:23:26 -07:00
Tim Abbott 4570936ac2 models: Fix default value for last_reminder.
This was causing a rather confusing test flake in
test_stream_error_pm_to_bot_owner.  What was happening was that if
this test (which used that code path) ran within 5 minutes of the
populate_db run, it would fail.
2018-03-29 11:53:12 -07:00
Tim Abbott f7c563dfcc migrations: Fix 0149_realm_emoji migration for S3 case.
The original migration implementation didn't follow the S3 API correctly.
2018-03-23 10:48:50 -07:00
Sarah ecd75ccba6 models: Add allow_community_topic_editing setting. 2018-03-22 16:02:24 -07:00
Harshit Bansal a49655e0d4 emoji: Migrate realm emoji to be addressed by `id` rather than `name`.
This commit migrates realm emoji to be addressed by their `id` rather
than their name. This fixes a long standing issue which was causing
an error on uploading an emoji with same name as a deactivated realm
emoji.

Fixes: #6977.
2018-03-20 22:24:44 -07:00
Tim Abbott b94a24ffe3 reactions: Fix buggy migration for realm emoji.
The original implementation of this migration had a highly unfortunate
bug that would result in it deleting all reactions to realm emoji on
the server; we missed this in review, so essentially all historical
realm emoji reactions on chat.zulip.org were lost :(.

We both correct the problem, and also add logging of the deleted rows
that would help should anything be deleted erroneously.
2018-03-20 21:50:47 -07:00
Greg Price dc1eeef30a antispam: Make a setting for default Realm.max_invites.
This makes this value much easier for a server admin to change than it
was when embedded directly in the code.  (Note this entire mechanism
already only applies on a server open for anyone to create a realm.)

Doing this also means getting the default out of the database.
Instead, we make the column nullable, and when it's NULL in the
database, treat that as whatever the current default is.  This better
matches anyway the likely model where there are a few realms with
specially-set values, and everything else should be treated uniformly.

The migration contains a `RenameField` step, which sounds scary
operationally -- but it really does mean just the *field*, in
the model within the Python code.  The underlying column's name
doesn't change.
2018-03-16 18:00:11 -07:00
Tim Abbott 69a7069ac4 migrations: Fix text version to Custom Emoji.
The fact that we need to do this is basically a Django bug; these
strings aren't used in the database itself.
2018-03-16 17:16:18 -07:00
Greg Price 4139e6c763 reactions: Fix migration to correctly handle corner case.
If an emoji that was deleted was the only realm emoji, or more
generally if all realm emoji were deleted, then we would just leave
the reaction unchanged, with an `emoji_code` that is now corrupt.

Instead, treat this case the same as if only this emoji was deleted
while others remain.
2018-03-15 18:53:51 -07:00
Vishnu Ks a44255eedb emails: Add backend for disallowing disposable email addresses. 2018-03-11 22:05:58 -07:00
YJDave c94b21e9ac settings: Add setting to disable message content in missed message emails.
Fixes #6938.
2018-03-09 21:16:02 -08:00
Harshit Bansal 5aa8629a10 reactions: Migrate `emoji_code` to store `id` for realm emoji.
Till now, we had been storing realm emoji's name in emoji code field
in reactions' model. This commit migrates it to store realm emoji's id.
It is a part of effort to migrate realm emojis to be referenced by their
id and not by name.
2018-03-09 13:46:27 -08:00
Shubham Dhama 777b6de689 org settings: Add setting to prevent users from adding bots.
Fixes: #7908.
2018-03-09 13:21:55 -08:00
Marco Burstein bdb86f1b5e emoji: Add support for translating emoticons.
Add `translate_emoticons` to `prop_types` and `expected_keys`.
Furthermore, create a emoji-translating Markdown inline pattern.

Also use a JavaScript version of `translate_emoticons` and then use
this function during Markdown previews and as a preprocessor. This
is only needed for previews, because usually emoticon translation
happens on the backend after sending.

Add tests for emoticon translation, a settings UI, and a /help/ page
as well.

Tweaked by tabbott to fix various test failurse as well as how this
handles whitespace, requiring emoticons to not have adjacent
characters.

Fixes #1768.
2018-03-04 15:37:24 -08:00
Vishnu Ks 497ee97ce3 models: Change description attribute of user group to textfield. 2018-03-01 11:27:26 -08:00
Aastha Gupta d124597f4b org-settings: Add setting to turn off welcome emails.
This adds an organization-level setting to provide an option to turn
off the welcome emails.

Fixes #8000.
2018-02-28 12:39:01 -08:00
Aditya Bansal f50e325075 RealmAuditLog: Fill subscription events with event_last_message_id=None.
Creating the very first organization administrator user and
subscribing them to streams before any messages were sent resulted in
RealmAuditLog entries being created with a `event_last_message_id` of
None, because that's the maximum ID in the empty set.

We correct this by fixing the incorrectly created RealmAuditLog
entries, both for new servers and also fixing old broken entries on
existing servers.

This fixes an issue where if a user setup a Zulip server with just the
organization administrator, and then forgot about it (so that the
initial user became soft-deactivated), trying to sign in 3 weeks later
would throw an exception.

This fixes the issue reported here:

https://chat.zulip.org/#narrow/stream/9-issues/subject/500.20error.20on.20login/near/511981
2018-02-22 09:45:22 -08:00
Greg Price 52f1c57c2a migrations: Fix mypy error in placeholder migration. 2018-02-05 18:03:07 -08:00
Roman Godov b875fe07eb settings: Added setting to turn on and off realm name in email subject.
Users having only account in one realm will not be distracted by realm
name in subject lines of every email.  Users who have multiple
accounts in realms can turn this setting on and receive a
corresponding realm name in email's subject.

Tweaked by tabbott to rebase and address a few small issues.

Fixes #5489.
2018-02-05 18:01:54 -08:00
Greg Price 6f128279e8 migrations: Add reverser for emoji_alt_code migration.
This is easy to do, and prevents this feature from getting a server
admin stuck in potentially a pretty uncomfortable way -- unable to
roll back a deploy.
2018-02-05 16:55:21 -08:00
Greg Price 0b3a414be1 Revert "migrations: Replace special chars in stream & user names with space."
This reverts commit acebd3a5e, as well as a subsequent fixup commit
0975bebac "quick fix: Fix migrations to be linear."

These changes need more work and thought before they're ready to
deploy on any large established Zulip server, such as zulipchat.com.
See discussion on #6534.

In place of the removed migration, leave behind a placeholder so
`manage.py migrate` doesn't get confused on installs where it was
already applied.
2018-02-05 16:55:21 -08:00
Vishnu Ks fe787c617c upload: Add a quota field to Realm. 2018-01-29 16:06:11 -08:00
Vishnu Ks 035d3c5e97 upload: Remove UserProfile field for old per-user quota. 2018-01-29 16:06:11 -08:00
Aditya Bansal dcce8e7219 scheduledmessages: Add delivery_type field to the table. 2018-01-19 11:33:11 -05:00
Aditya Bansal 6c36cf3c82 scheduledmessages: Add model for the ScheduledMessage table. 2018-01-10 09:18:02 -05:00
Robert Hönig 29a343cb44 Rename BotUserConfigData to BotConfigData.
This is in compliance with the naming of
BotStorageData.
2018-01-06 14:13:18 -05:00
Vishnu Ks f2a7d44c29 models: Add message_visibility_limit attribute to Realm. 2018-01-04 08:52:40 -05:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Aastha Gupta daf86eb664 settings: Add "text" option to emoji_set model.
We no longer have a special UI setting and model
field ("emoji_alt_code") for saying users want text-only
emojis.  We now instead make "text" be a fifth choice
for "emojiset".

Fixes #7406
2018-01-02 14:55:01 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
Rishi Gupta 869b4d41ef models: Add ScheduledEmail.realm.
The two extra queries in the test are due to the assert in
send_future_email.
2017-12-19 17:46:36 -08:00
Steve Howell 0975bebacc quick fix: Fix migrations to be linear.
I merged a "good" but stale build, and the migrations
were out of order.  This fixes them.
2017-12-12 11:31:07 -06:00
Rhea Parekh acebd3a5e9 migrations: Replace special chars in stream & user names with space.
Also handle potential duplicate stream names created.
Fixes #6534
2017-12-12 10:04:31 -06:00
Rishi Gupta dbe3576706 registration: Enforce realm is None only if realm_creation.
Commit d4ee3023 and its parent have the history behind this code.

Since d4ee3023^, all new PreregistrationUser objects, except those for
realm creation, have a non-None `realm`.  Since d4ee3023, any legacy
PreregistrationUsers, with a `realm` of None despite not being for
realm creation, are treated as expired.  Now, we ignore them
completely, and remove any that exist from the database.

The user-visible effect is to change the error message for
registration (or invitation) links created before d4ee3023^ to be
"link does not exist", rather than "link expired".

This change will at most affect users upgrading straight from 1.7 or
earlier to 1.8 (rather than from 1.7.1), but I think that's not much
of a concern (such installations are probably long-running
installations, without many live registration or invitation links).

[greg: tweaked commit message]
2017-12-11 18:36:14 -08:00
Rishi Gupta 7d1c88f0fb invitations: Limit realms to 100 invites per day.
To guard against using zulip invites as a vector for spam. Stopgap measure
until we figure out something better.
2017-11-29 22:18:05 -08:00
Sarah ba70b542d3 notification settings: Add stream email notifications setting.
Adds a field to UserProfile for enabling emails about stream
messages and a field to Subscriptions about email
notification preferences.
2017-11-28 17:51:18 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
Vishnu Ks 08aca174fe models: Add signup_notifications_stream attribute to Realm. 2017-11-21 17:39:50 -08:00
derAnfaenger 6049cd27b1 migrations: Fix conflict. 2017-11-17 08:24:56 -08:00
derAnfaenger 395f1e9270 embedded bots: Add database config storage.
Storage limititations are only set on the value of
a config entry, since this is the only user-accessible
part of the schema. Keys are statically set by each
embedded bot.
2017-11-16 23:06:38 -08:00
Brock Whittaker f9f0f356be settings: Add backend storage and interface for night mode.
This allows the night mode setting to be stored in the backend.
2017-11-15 16:36:06 -08:00
Tim Abbott 38e97089fc python: Sort imports in database migrations. 2017-11-15 15:44:00 -08:00
Vishnu Ks 2af249dd5d api: Add description attribute to default stream group. 2017-11-14 14:41:42 -08:00
Brock Whittaker ca2f93db31 settings: Restyle "emoji style" section.
This restyles and rewords some of the emoji style section to look
better and fit it more with the current style guide.

Tweaked by tabbott to modify the historical migration rather than
adding a new one.  This is OK because the emojiset choices text change
doesn't touch the database; it's just a Django Python code thing.

Also removed translation tags, since we don't need them for a set of
brand names.
2017-11-10 17:04:27 -08:00
Umair Khan 39ca38837e user-groups: Add description to groups. 2017-11-09 16:35:52 -08:00
YJDave de67bf81fd org settings: Allow users to delete their message.
Fixes #7049.
2017-11-09 10:20:34 -08:00
rht 5ee40bf718 Remove usage of six.moves.binary_type. 2017-11-09 10:00:00 -08:00
rht 864e16f30d zerver/migrations: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht dc37e3f72c zerver/migrations: Remove u prefix from strings. 2017-11-02 11:01:47 -07:00
Umair Khan 636046aec9 user-groups: Add basic backend for UserGroup model.
This adds the data model and bugdown support for the new UserGroup
mention feature.

Before it'll be fully operational, we'll still need:
* A backend API for making these.
* A UI for interacting with that API.
* Typeahead on the frontend.
* CSS to make them look pretty and see who's in them.
2017-10-31 15:16:14 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Vishnu Ks 8c68a167fe backend: Allow Administrators to invite new users as admins.
Tweaked by tabbott to have the field before the invitation is
completed be called invite_as_admins, not invited_as_admins, for
readability.

Fixes #6834.
2017-10-27 11:19:38 -07:00
rht c2a5b5f160 zerver/migrations: Use python 3 syntax for typing. 2017-10-26 15:24:56 -07:00
Vishnu Ks 4f00170735 models: Create DefaultStreamGroup. 2017-10-26 11:11:51 -07:00
Steve Howell 0164cfdf5c migrations: Index MutedTopic on stream/topic.
This index lets us quickly find all users who may be muting
a topic, which useful when we send messages out.
2017-10-23 17:26:15 -07:00
Tim Abbott 88245d2cd6 migrations: Fix deactivated realm corner cases with 0108.
Previously the default-string-id migration would not correctly handle
ignoring deactivated realms.
2017-10-20 09:22:35 -07:00
Tim Abbott 91520fca22 realm_emoji migration: Handle relative URLs.
This is necessary for the upgrade path in a case a server was using a
path to a file uploaded in Zulip for their custom emoji.
2017-10-19 09:00:43 -07:00
derAnfaenger 51c5fec0e4 embedded bots: Add BotUserStateData model. 2017-10-18 09:15:53 -07:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
rht b2ad8fd747 py3: Remove all `from __future__ import unicode_literals`.
This was mostly used in migrations, so it's a pretty safe change.
2017-10-17 23:07:42 -07:00
Steve Howell a6ad9a6d7c Add is_zephyr to the Stream model.
Add this field to the Stream model will prevent us from having
to look at realm data for several types of stream operations, which
can be prone to either doing extra database lookups or making
our cached data bloated.

Going forward, we'll set stream.is_zephyr to True whenever the
realm's string id is "zephyr".
2017-10-11 16:15:56 -07:00
rht 4975122338 zerver/migrations: Remove from __future__ import *. 2017-09-28 10:16:18 -07:00
Rishi Gupta 9cec8f08d1 hotspots: Turn on hotspots for new users.
We need a migration to clear the tutorial_status for existing users,
so that we don't show hotspots to anyone who signed up for Zulip in
the month or so since we deleted the old tutorial.
2017-09-25 15:47:34 -07:00
Tim Abbott 632180274a mypy: Add imports needed for new migration. 2017-09-25 06:45:56 -07:00
Tim Abbott 69ea571fcf zerver: Ensure single-realm servers have '' string_id. 2017-09-25 06:14:00 -07:00
Vishnu Ks 3cc9feb676 models: Create MultiuseInvite model. 2017-09-22 07:51:58 -07:00
Aditya Bansal ddf2075228 Make migration 0032 use an old version of user_avatar_path.
This fixes upgrading from very old Zulip servers (e.g. 1.4.3) all the
way to current.

Fixes: #6516.
2017-09-16 03:27:02 -07:00
Tim Abbott e8f835d852 migrations: Fix migration 0041 failures for long attachment filenames.
We should have done this a long time ago, but better late than never.
Basically, this migration would crash in the event that there were any
attachments with particularly long names.  The fix is the next
migration, 0042; we just inline it here to avoid that crash.
2017-09-14 07:00:07 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Steve Howell 4dfe6bb320 Add migration to fix unread messages. 2017-09-07 07:06:03 -07:00
Steve Howell 0721115c64 model: Remove user_profile.muted_topics.
(We now track muted topics in the MutedTopic model.
2017-09-02 09:19:51 -07:00
Steve Howell 4ac6bc46c7 Add MutedTopic model.
This commit completely switches us over to using a
dedicated model called MutedTopic to track which topics
a user has muted.

This includes the necessary migrations to create the
table and populate it from legacy data in UserProfile.

A subsequent commit will actually remove the old field
in UserProfile.
2017-09-02 09:19:51 -07:00
Tim Abbott f1648af607 migrations: Update UserMessage model for is_me_message removal.
And while we're at it, document the related migration we need to do.
2017-08-27 10:11:43 -07:00
Tim Abbott d43f5ceeec database: Add database index for wildcard mentions. 2017-08-16 13:28:04 -07:00
Tim Abbott 8bb812c8a9 database: Add database index for alert words. 2017-08-16 12:39:01 -07:00
Tim Abbott 9081f2cf44 reactions: Store the emoji codepoint in the database.
This is the first part of a larger migration to convert Zulip's
reactions storage to something based on the codepoint, not the emoji
name that the user typed in, so that we don't need to worry about
changes in the names we're using breaking the emoji storage.
2017-08-15 09:29:27 -07:00
Umair Khan bb0eb76bf3 github: Don't ask for password in registration. 2017-08-09 13:44:57 -07:00
neiljp (Neil Pilgrim) b782db48e1 mypy: Remove superfluous older 'type: ignore' annotations. 2017-08-08 11:27:51 -07:00
Tim Abbott 53e4d8562b lint: Add a lint check for bare `type: ignore`s. 2017-07-27 16:31:55 -07:00
Steve Howell b588bc47fc Create zerver_usermessage_unread_message_id index.
This creates a partial index on UserMessage that lets us
find unread messages for particular user id.
2017-07-26 22:28:19 -07:00
Harshit Bansal f2c04576bd models: Fix the URL validation code in `RealmFilter` model. 2017-07-24 17:31:08 -07:00
Tim Abbott ed381763f0 mypy: Fix incorrect mypy annotation in migration. 2017-07-18 12:56:23 -07:00
Tim Abbott 9e81fc39e0 migrations: Use bulk_create in migration 0093.
This fixes a performance issue that caused this migration to run for a
really long time.

It still takes about 1 minute to run with the 75K Subscription objects
we have on chat.zulip.org, but that's within the realm of acceptable.
2017-07-18 11:56:06 -07:00
Tim Abbott a36cf3b492 migrations: Fix missing select_related in RealmAuditLog migration.
This migration accidentally did a bunch of database queries fetching
data in a loop.
2017-07-18 11:55:49 -07:00
Umair Khan a610cec8fc migrations: Fix migration numbering. 2017-07-17 23:09:44 -07:00
Aditya Bansal f2d6194ae1 actions: Start logging subscription activities in RealmAuditLog. 2017-07-17 17:23:41 -07:00
Rishi Gupta aa845e7f60 models: Replace ScheduledJob with ScheduledEmail.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
2017-07-17 16:05:38 -07:00
Durga Akhil Mundroy 146dfa6f0b org-permissions: Add allow_edit_history organiztion setting.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization.  It controls access through 2
key mechanisms:

* For long-ago edited messages, get_messages removes the edit history
  content from messages it sends to clients.

* For newly edited messages, clients are responsible for checking the
  setting and not saving the edit history data.  Since the webapp was
  the only client displaying it before this change, this just required
  some changes in message_events.js.

Significantly modified by tabbott to fix some logic bugs and add a
test.
2017-07-16 10:10:06 -07:00
Vaida Plankyte 28ea174ed9 backend: Implement high contrast mode display setting. 2017-07-14 14:53:24 -07:00
Aditya Bansal ec6fdd92d4 models: Add long_term_idle/last_active_message_id to UserProfile.
In this commit we are adding two new fields to the UserProfile
table. These fields are the:
long_term_idle: For storing a bool value representing status of user
being online in long time where 'long' will have a specific
definition.
last_active_message_id: For storing the message id which was last
updated into the UserMessage table for a particular user.
2017-07-10 12:31:50 -04:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
James Rowan 368bd66d8b emails: Refactor send_email functions to take both a sender name and address.
This will allow for customized senders for emails, e.g. 'Zulip Digest' for
digest emails and 'Zulip Missed Messages' for missed message emails.

Also:
* Converts the sender name to always be "Zulip", if the from_email used to
  be settings.NOREPLY_EMAIL_ADDRESS or settings.ZULIP_ADMINISTRATOR.

* Changes the default value of settings.NOREPLY_EMAIL_ADDRESS in the
  prod_setting_template to no longer have a display name. The only use of
  that display name was in the email pathway.
2017-07-04 14:25:01 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
Umair Khan a21f2d7715 migrations: Transition to new squashed migration. 2017-06-23 08:40:57 -04:00
Umair Khan cec78e23bc migrations: Remove squashed migration files. 2017-06-23 08:40:57 -04:00
Umair Khan 6fbddf578a migrations: Squash migrations from 0001 to 0028. 2017-06-23 08:40:57 -04:00
Sampriti Panda c36c3ae20b bots: Change bot_type to DEFAULT_BOT for bots with None bot_type
Fixes #5440
2017-06-20 11:34:40 -04:00
Harshit Bansal e85c11a461 models: Add `deactivated` field to RealmEmoji model.
Instead of deleting the emojis we will just mark them as `deactivated`.
This will prevent their further use but at the same time they will be
displayed properly in the reactions and messages in which they are used.
This field will be used as a flag to indicate whether the realm emoji
has been deleted or not.

Includes a database migration.
2017-06-15 02:59:57 -07:00
Umair Khan aeb68a6643 zerver: Add on_delete arg in foreign keys.
on_delete will be a required arg for ForeignKey in Django 2.0. Set it
to models.CASCADE on models and in existing migrations if you want to
maintain the current default behavior.
See https://docs.djangoproject.com/en/1.11/ref/models/fields/#django.db.models.ForeignKey.on_delete
2017-06-13 15:13:45 -07:00
Steve Howell c96d58734f migrations: Index mentioned messages.
This index is basically free, since it only consumes space for
messages that mention the user, but results in a massive performance
improvement when querying messages that mention a user.
2017-06-13 13:58:58 -07:00
Steve Howell 52f7d1a3e2 migrations: Index starred messages.
This index is essentially free, since it only consumes resources for
starred messages, and massively improves the performance of the
"starred messages" narrow.
2017-06-13 13:58:58 -07:00
Rick Chern 70d68f7e71 Refactoring: Replace get_user_profile_by_email() in lib/upload.py 2017-05-23 12:37:49 -07:00
Cory Lynch b1bfe9f42e Add migration to force lowercase existing realm emoji.
Since realm emoji are now required to be lowercase,
an appropriate migration was added to retroactively
fix any emoji that might have contained uppercase
letters.

Also, the validator on the model was changed to
reject uppercase letters.
2017-05-11 19:10:21 -07:00
Tim Abbott 223624be25 settings: Add support for longer, markdown-powered realm descriptions.
This makes it possible to create much prettier login pages.

Further work on styling may be necessary.
2017-05-11 13:59:46 -07:00
Tim Abbott 0e05f3f4ee emails: Remove pre-email-migration scheduled jobs.
This fixes an issue introduced when we migrated the format of all of
our emails, which caused any old ScheduledJob rows to be corrupted.
2017-05-10 09:45:40 -07:00
Aditya Bansal ce4c4f4e0f pep8: Add compliance with rule E261 to migrations/0001_initial.py. 2017-05-07 23:21:50 -07:00
rahuldeve 413c5f93bb Outgoing Webhook System: Add Service model. 2017-05-02 08:20:31 -07:00
K.Kanakhin f13d6a18eb realm-emoji: Add realm emoji uploading instead url providing.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.

Fixes #1134
2017-05-01 14:50:20 -07:00
Tim Abbott 427629ca4c emoji: Fix strings in migration 0076.
It's arguably a bug that Django puts the value strings into the
migration object, but this was causing test failures.
2017-04-24 22:50:19 -07:00
Harshit Bansal fb3fda031d models: Add `emojiset` field to `UserProfile` model.
Includes a database migration.
2017-04-24 22:30:06 -07:00
Tim Abbott b92385ea8e attachment: Make path_id field unique.
This will help avoid future bugs similar to that fixed in migration
0072.
2017-04-18 21:27:28 -07:00
Tim Abbott 0f855b84cc attachment: Add migration to fix duplicate attachment objects.
The comment in the migration explains this change in detail.
2017-04-18 21:27:27 -07:00
Umair Khan cf3b6c6ca9 profile: Support custom profile data.
Implements backend for #1760.
2017-04-18 15:20:59 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Rishi Gupta b396be4c98 models: Add index on RealmAuditLog.event_time.
Useful for the upcoming check_realmauditlog_by_user_query, if nothing else.

But I suspect it will indeed get use; looking for events around or within a
certain time is pretty natural for an audit log.

The main argument against I would say is that this should actually be a
joint index with something else. I'm not sure what that something else
should be, so just optimizing for what I think
check_realmauditlog_by_user_query will need for now.
2017-04-14 11:41:07 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Amy Liu 6f061beb46 hotspots: Add backend support for tutorial hotspots.
This commit adds the backend support for a new style of tutorial which
allows for highlighting of multiple areas of the page with hotspots that
disappear when clicked by the user.
2017-03-29 11:34:32 -07:00
Tim Abbott 141469399b RealmAuditLog: Add an extra_data field.
This will be useful for logging any additional details that we might
want to display.
2017-03-27 13:22:58 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
K.Kanakhin fe3213798d retention: Add models to store expired messages.
This addresses part of #106.
2017-03-25 18:12:06 -07:00
Ayush Jain bddcfb1c96 Add realm-level settings to control inline image and url preview.
This gives users more control in case they don't want previews,
especially for the "previews of linked websites" feature.

Fixes: #2640.
2017-03-21 15:46:17 -07:00
adnrs96 8ae35211b5 migration: Sync sizes for existing Attachment objects with actual files.
Tweaked by tabbott to add a nop reverse migration.

Fixes #3883.
2017-03-21 00:53:22 -07:00
Sarah Stringer aa880b0419 Add organization description field to realm settings.
This adds an organization description field to the Realm model, as well as
an input field to the organization settings template. Added three tests.
Set the max length of the field to 100 characters.

Fixes #3962.
2017-03-19 14:05:01 -07:00
Umair Khan 645d7c295c Timezone should default to empty string.
Related to #1506
2017-03-17 21:57:16 -07:00
Umair Khan 242d3ffaf4 Add timezone field in UserProfile.
Implements backend of #1506.
2017-03-15 11:18:24 -07:00
Rishi Gupta 3645bb9225 Change if(realm.domain == mit.edu) to use Realm.is_zephyr_mirror_realm. 2017-03-13 14:17:14 -07:00
adnrs96 556dde8e3e avatar migration: Handle missing avatars for S3 gracefully.
In this commit we start to handle missing avatars gracefully in case
migration was being run on the AWS.
2017-03-08 22:24:07 -08:00
Tim Abbott 7c9ee2f677 avatar migration: Remove now-unnecessary debugging print. 2017-03-07 23:17:11 -08:00
Tim Abbott a32f666f5c avatar migration: Handle missing avatars gracefully.
This makes the outcome if a user didn't have an avatar due to a past
email change reasonable; the user will just be bumped back to
gravatar, fixing their invalid state.
2017-03-07 23:10:15 -08:00
Tim Abbott 0afd4d5740 avatar: Add error handling for avatar migraton. 2017-03-07 23:10:15 -08:00
adnrs96 ae31c6cc0d avatar: Move avatar data from email based to user id based.
This commit introduces a migration for moving avatars from email based
to user id based storage.

This is in responce to change in behaviour of user_avatar_path to
return path comprising of realm id and a hash based on user id. Also
we fix test_helpers accordingly.

Fixes #3776.
2017-03-07 22:44:26 -08:00
Tim Abbott 5488de11e6 migrations: Renumber 0059_userprofile_quota to fix conflict. 2017-03-04 18:38:27 -08:00
Philip Skomorokhov 866a7b06b2 upload: Limit total size of files uploaded by a user to 1GB.
Fixes #3884.
2017-03-04 18:08:30 -08:00
Raghav Jajodia cd2d798498 admin: Added realm option to prevent users from changing their email.
A realm option to prevent users from changing their email address is added.
Fixes #3777.
2017-03-04 17:32:48 -08:00
Tim Abbott d25bfb88d3 lint: Ban importing zerver.models in database migrations.
This doesn't work correctly, but in a subtle way.
2017-03-04 17:32:48 -08:00