Commit Graph

427 Commits

Author SHA1 Message Date
Vishnu Ks fe787c617c upload: Add a quota field to Realm. 2018-01-29 16:06:11 -08:00
Vishnu Ks 035d3c5e97 upload: Remove UserProfile field for old per-user quota. 2018-01-29 16:06:11 -08:00
Aditya Bansal dcce8e7219 scheduledmessages: Add delivery_type field to the table. 2018-01-19 11:33:11 -05:00
Aditya Bansal 6c36cf3c82 scheduledmessages: Add model for the ScheduledMessage table. 2018-01-10 09:18:02 -05:00
Robert Hönig 29a343cb44 Rename BotUserConfigData to BotConfigData.
This is in compliance with the naming of
BotStorageData.
2018-01-06 14:13:18 -05:00
Vishnu Ks f2a7d44c29 models: Add message_visibility_limit attribute to Realm. 2018-01-04 08:52:40 -05:00
Alena Volkova 45f0c76c44 settings: Limit the creation of generic bots.
This commit adds a setting to limit creation of generic bots
to admins for realms that want that restriction.  (Generic
bots, apart from being considered spammy on some realms,
have less locked down permissions than webhook bots).

Fixes #7066.
2018-01-02 18:12:22 -05:00
Aastha Gupta daf86eb664 settings: Add "text" option to emoji_set model.
We no longer have a special UI setting and model
field ("emoji_alt_code") for saying users want text-only
emojis.  We now instead make "text" be a fifth choice
for "emojiset".

Fixes #7406
2018-01-02 14:55:01 -05:00
YJDave fa44d2ea69 settings: Remove autoscroll_forever setting.
Fixes #6845
2018-01-02 10:35:49 -05:00
Rishi Gupta 869b4d41ef models: Add ScheduledEmail.realm.
The two extra queries in the test are due to the assert in
send_future_email.
2017-12-19 17:46:36 -08:00
Steve Howell 0975bebacc quick fix: Fix migrations to be linear.
I merged a "good" but stale build, and the migrations
were out of order.  This fixes them.
2017-12-12 11:31:07 -06:00
Rhea Parekh acebd3a5e9 migrations: Replace special chars in stream & user names with space.
Also handle potential duplicate stream names created.
Fixes #6534
2017-12-12 10:04:31 -06:00
Rishi Gupta dbe3576706 registration: Enforce realm is None only if realm_creation.
Commit d4ee3023 and its parent have the history behind this code.

Since d4ee3023^, all new PreregistrationUser objects, except those for
realm creation, have a non-None `realm`.  Since d4ee3023, any legacy
PreregistrationUsers, with a `realm` of None despite not being for
realm creation, are treated as expired.  Now, we ignore them
completely, and remove any that exist from the database.

The user-visible effect is to change the error message for
registration (or invitation) links created before d4ee3023^ to be
"link does not exist", rather than "link expired".

This change will at most affect users upgrading straight from 1.7 or
earlier to 1.8 (rather than from 1.7.1), but I think that's not much
of a concern (such installations are probably long-running
installations, without many live registration or invitation links).

[greg: tweaked commit message]
2017-12-11 18:36:14 -08:00
Rishi Gupta 7d1c88f0fb invitations: Limit realms to 100 invites per day.
To guard against using zulip invites as a vector for spam. Stopgap measure
until we figure out something better.
2017-11-29 22:18:05 -08:00
Sarah ba70b542d3 notification settings: Add stream email notifications setting.
Adds a field to UserProfile for enabling emails about stream
messages and a field to Subscriptions about email
notification preferences.
2017-11-28 17:51:18 -08:00
Tim Abbott 586e18b237 auth: Allow accounts with the same email in different realms.
[Modified by greg to (1) keep `USERNAME_FIELD = 'email'`,
(2) silence the corresponding system check, and (3) ban
reusing a system bot's email address, just like we do in
realm creation.]
2017-11-28 16:23:10 -08:00
derAnfaenger c8a5ae753c embedded bots: Consistently use 'storage' instead of 'state.' 2017-11-27 21:05:34 -08:00
Vishnu Ks 08aca174fe models: Add signup_notifications_stream attribute to Realm. 2017-11-21 17:39:50 -08:00
derAnfaenger 6049cd27b1 migrations: Fix conflict. 2017-11-17 08:24:56 -08:00
derAnfaenger 395f1e9270 embedded bots: Add database config storage.
Storage limititations are only set on the value of
a config entry, since this is the only user-accessible
part of the schema. Keys are statically set by each
embedded bot.
2017-11-16 23:06:38 -08:00
Brock Whittaker f9f0f356be settings: Add backend storage and interface for night mode.
This allows the night mode setting to be stored in the backend.
2017-11-15 16:36:06 -08:00
Tim Abbott 38e97089fc python: Sort imports in database migrations. 2017-11-15 15:44:00 -08:00
Vishnu Ks 2af249dd5d api: Add description attribute to default stream group. 2017-11-14 14:41:42 -08:00
Brock Whittaker ca2f93db31 settings: Restyle "emoji style" section.
This restyles and rewords some of the emoji style section to look
better and fit it more with the current style guide.

Tweaked by tabbott to modify the historical migration rather than
adding a new one.  This is OK because the emojiset choices text change
doesn't touch the database; it's just a Django Python code thing.

Also removed translation tags, since we don't need them for a set of
brand names.
2017-11-10 17:04:27 -08:00
Umair Khan 39ca38837e user-groups: Add description to groups. 2017-11-09 16:35:52 -08:00
YJDave de67bf81fd org settings: Allow users to delete their message.
Fixes #7049.
2017-11-09 10:20:34 -08:00
rht 5ee40bf718 Remove usage of six.moves.binary_type. 2017-11-09 10:00:00 -08:00
rht 864e16f30d zerver/migrations: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht dc37e3f72c zerver/migrations: Remove u prefix from strings. 2017-11-02 11:01:47 -07:00
Umair Khan 636046aec9 user-groups: Add basic backend for UserGroup model.
This adds the data model and bugdown support for the new UserGroup
mention feature.

Before it'll be fully operational, we'll still need:
* A backend API for making these.
* A UI for interacting with that API.
* Typeahead on the frontend.
* CSS to make them look pretty and see who's in them.
2017-10-31 15:16:14 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Vishnu Ks 8c68a167fe backend: Allow Administrators to invite new users as admins.
Tweaked by tabbott to have the field before the invitation is
completed be called invite_as_admins, not invited_as_admins, for
readability.

Fixes #6834.
2017-10-27 11:19:38 -07:00
rht c2a5b5f160 zerver/migrations: Use python 3 syntax for typing. 2017-10-26 15:24:56 -07:00
Vishnu Ks 4f00170735 models: Create DefaultStreamGroup. 2017-10-26 11:11:51 -07:00
Steve Howell 0164cfdf5c migrations: Index MutedTopic on stream/topic.
This index lets us quickly find all users who may be muting
a topic, which useful when we send messages out.
2017-10-23 17:26:15 -07:00
Tim Abbott 88245d2cd6 migrations: Fix deactivated realm corner cases with 0108.
Previously the default-string-id migration would not correctly handle
ignoring deactivated realms.
2017-10-20 09:22:35 -07:00
Tim Abbott 91520fca22 realm_emoji migration: Handle relative URLs.
This is necessary for the upgrade path in a case a server was using a
path to a file uploaded in Zulip for their custom emoji.
2017-10-19 09:00:43 -07:00
derAnfaenger 51c5fec0e4 embedded bots: Add BotUserStateData model. 2017-10-18 09:15:53 -07:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
rht b2ad8fd747 py3: Remove all `from __future__ import unicode_literals`.
This was mostly used in migrations, so it's a pretty safe change.
2017-10-17 23:07:42 -07:00
Steve Howell a6ad9a6d7c Add is_zephyr to the Stream model.
Add this field to the Stream model will prevent us from having
to look at realm data for several types of stream operations, which
can be prone to either doing extra database lookups or making
our cached data bloated.

Going forward, we'll set stream.is_zephyr to True whenever the
realm's string id is "zephyr".
2017-10-11 16:15:56 -07:00
rht 4975122338 zerver/migrations: Remove from __future__ import *. 2017-09-28 10:16:18 -07:00
Rishi Gupta 9cec8f08d1 hotspots: Turn on hotspots for new users.
We need a migration to clear the tutorial_status for existing users,
so that we don't show hotspots to anyone who signed up for Zulip in
the month or so since we deleted the old tutorial.
2017-09-25 15:47:34 -07:00
Tim Abbott 632180274a mypy: Add imports needed for new migration. 2017-09-25 06:45:56 -07:00
Tim Abbott 69ea571fcf zerver: Ensure single-realm servers have '' string_id. 2017-09-25 06:14:00 -07:00
Vishnu Ks 3cc9feb676 models: Create MultiuseInvite model. 2017-09-22 07:51:58 -07:00
Aditya Bansal ddf2075228 Make migration 0032 use an old version of user_avatar_path.
This fixes upgrading from very old Zulip servers (e.g. 1.4.3) all the
way to current.

Fixes: #6516.
2017-09-16 03:27:02 -07:00
Tim Abbott e8f835d852 migrations: Fix migration 0041 failures for long attachment filenames.
We should have done this a long time ago, but better late than never.
Basically, this migration would crash in the event that there were any
attachments with particularly long names.  The fix is the next
migration, 0042; we just inline it here to avoid that crash.
2017-09-14 07:00:07 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Steve Howell 4dfe6bb320 Add migration to fix unread messages. 2017-09-07 07:06:03 -07:00
Steve Howell 0721115c64 model: Remove user_profile.muted_topics.
(We now track muted topics in the MutedTopic model.
2017-09-02 09:19:51 -07:00
Steve Howell 4ac6bc46c7 Add MutedTopic model.
This commit completely switches us over to using a
dedicated model called MutedTopic to track which topics
a user has muted.

This includes the necessary migrations to create the
table and populate it from legacy data in UserProfile.

A subsequent commit will actually remove the old field
in UserProfile.
2017-09-02 09:19:51 -07:00
Tim Abbott f1648af607 migrations: Update UserMessage model for is_me_message removal.
And while we're at it, document the related migration we need to do.
2017-08-27 10:11:43 -07:00
Tim Abbott d43f5ceeec database: Add database index for wildcard mentions. 2017-08-16 13:28:04 -07:00
Tim Abbott 8bb812c8a9 database: Add database index for alert words. 2017-08-16 12:39:01 -07:00
Tim Abbott 9081f2cf44 reactions: Store the emoji codepoint in the database.
This is the first part of a larger migration to convert Zulip's
reactions storage to something based on the codepoint, not the emoji
name that the user typed in, so that we don't need to worry about
changes in the names we're using breaking the emoji storage.
2017-08-15 09:29:27 -07:00
Umair Khan bb0eb76bf3 github: Don't ask for password in registration. 2017-08-09 13:44:57 -07:00
neiljp (Neil Pilgrim) b782db48e1 mypy: Remove superfluous older 'type: ignore' annotations. 2017-08-08 11:27:51 -07:00
Tim Abbott 53e4d8562b lint: Add a lint check for bare `type: ignore`s. 2017-07-27 16:31:55 -07:00
Steve Howell b588bc47fc Create zerver_usermessage_unread_message_id index.
This creates a partial index on UserMessage that lets us
find unread messages for particular user id.
2017-07-26 22:28:19 -07:00
Harshit Bansal f2c04576bd models: Fix the URL validation code in `RealmFilter` model. 2017-07-24 17:31:08 -07:00
Tim Abbott ed381763f0 mypy: Fix incorrect mypy annotation in migration. 2017-07-18 12:56:23 -07:00
Tim Abbott 9e81fc39e0 migrations: Use bulk_create in migration 0093.
This fixes a performance issue that caused this migration to run for a
really long time.

It still takes about 1 minute to run with the 75K Subscription objects
we have on chat.zulip.org, but that's within the realm of acceptable.
2017-07-18 11:56:06 -07:00
Tim Abbott a36cf3b492 migrations: Fix missing select_related in RealmAuditLog migration.
This migration accidentally did a bunch of database queries fetching
data in a loop.
2017-07-18 11:55:49 -07:00
Umair Khan a610cec8fc migrations: Fix migration numbering. 2017-07-17 23:09:44 -07:00
Aditya Bansal f2d6194ae1 actions: Start logging subscription activities in RealmAuditLog. 2017-07-17 17:23:41 -07:00
Rishi Gupta aa845e7f60 models: Replace ScheduledJob with ScheduledEmail.
ScheduledJob was written for much more generality than it ended up being
used for. Currently it is used by send_future_email, and nothing
else. Tailoring the model to emails in particular will make it easier to do
things like selectively clear emails when people unsubscribe from particular
email types, or seamlessly handle using the same email on multiple realms.
2017-07-17 16:05:38 -07:00
Durga Akhil Mundroy 146dfa6f0b org-permissions: Add allow_edit_history organiztion setting.
This new setting controls whether or not users are allowed to see the
edit history in a Zulip organization.  It controls access through 2
key mechanisms:

* For long-ago edited messages, get_messages removes the edit history
  content from messages it sends to clients.

* For newly edited messages, clients are responsible for checking the
  setting and not saving the edit history data.  Since the webapp was
  the only client displaying it before this change, this just required
  some changes in message_events.js.

Significantly modified by tabbott to fix some logic bugs and add a
test.
2017-07-16 10:10:06 -07:00
Vaida Plankyte 28ea174ed9 backend: Implement high contrast mode display setting. 2017-07-14 14:53:24 -07:00
Aditya Bansal ec6fdd92d4 models: Add long_term_idle/last_active_message_id to UserProfile.
In this commit we are adding two new fields to the UserProfile
table. These fields are the:
long_term_idle: For storing a bool value representing status of user
being online in long time where 'long' will have a specific
definition.
last_active_message_id: For storing the message id which was last
updated into the UserMessage table for a particular user.
2017-07-10 12:31:50 -04:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
James Rowan 368bd66d8b emails: Refactor send_email functions to take both a sender name and address.
This will allow for customized senders for emails, e.g. 'Zulip Digest' for
digest emails and 'Zulip Missed Messages' for missed message emails.

Also:
* Converts the sender name to always be "Zulip", if the from_email used to
  be settings.NOREPLY_EMAIL_ADDRESS or settings.ZULIP_ADMINISTRATOR.

* Changes the default value of settings.NOREPLY_EMAIL_ADDRESS in the
  prod_setting_template to no longer have a display name. The only use of
  that display name was in the email pathway.
2017-07-04 14:25:01 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
Umair Khan a21f2d7715 migrations: Transition to new squashed migration. 2017-06-23 08:40:57 -04:00
Umair Khan cec78e23bc migrations: Remove squashed migration files. 2017-06-23 08:40:57 -04:00
Umair Khan 6fbddf578a migrations: Squash migrations from 0001 to 0028. 2017-06-23 08:40:57 -04:00
Sampriti Panda c36c3ae20b bots: Change bot_type to DEFAULT_BOT for bots with None bot_type
Fixes #5440
2017-06-20 11:34:40 -04:00
Harshit Bansal e85c11a461 models: Add `deactivated` field to RealmEmoji model.
Instead of deleting the emojis we will just mark them as `deactivated`.
This will prevent their further use but at the same time they will be
displayed properly in the reactions and messages in which they are used.
This field will be used as a flag to indicate whether the realm emoji
has been deleted or not.

Includes a database migration.
2017-06-15 02:59:57 -07:00
Umair Khan aeb68a6643 zerver: Add on_delete arg in foreign keys.
on_delete will be a required arg for ForeignKey in Django 2.0. Set it
to models.CASCADE on models and in existing migrations if you want to
maintain the current default behavior.
See https://docs.djangoproject.com/en/1.11/ref/models/fields/#django.db.models.ForeignKey.on_delete
2017-06-13 15:13:45 -07:00
Steve Howell c96d58734f migrations: Index mentioned messages.
This index is basically free, since it only consumes space for
messages that mention the user, but results in a massive performance
improvement when querying messages that mention a user.
2017-06-13 13:58:58 -07:00
Steve Howell 52f7d1a3e2 migrations: Index starred messages.
This index is essentially free, since it only consumes resources for
starred messages, and massively improves the performance of the
"starred messages" narrow.
2017-06-13 13:58:58 -07:00
Rick Chern 70d68f7e71 Refactoring: Replace get_user_profile_by_email() in lib/upload.py 2017-05-23 12:37:49 -07:00
Cory Lynch b1bfe9f42e Add migration to force lowercase existing realm emoji.
Since realm emoji are now required to be lowercase,
an appropriate migration was added to retroactively
fix any emoji that might have contained uppercase
letters.

Also, the validator on the model was changed to
reject uppercase letters.
2017-05-11 19:10:21 -07:00
Tim Abbott 223624be25 settings: Add support for longer, markdown-powered realm descriptions.
This makes it possible to create much prettier login pages.

Further work on styling may be necessary.
2017-05-11 13:59:46 -07:00
Tim Abbott 0e05f3f4ee emails: Remove pre-email-migration scheduled jobs.
This fixes an issue introduced when we migrated the format of all of
our emails, which caused any old ScheduledJob rows to be corrupted.
2017-05-10 09:45:40 -07:00
Aditya Bansal ce4c4f4e0f pep8: Add compliance with rule E261 to migrations/0001_initial.py. 2017-05-07 23:21:50 -07:00
rahuldeve 413c5f93bb Outgoing Webhook System: Add Service model. 2017-05-02 08:20:31 -07:00
K.Kanakhin f13d6a18eb realm-emoji: Add realm emoji uploading instead url providing.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.

Fixes #1134
2017-05-01 14:50:20 -07:00
Tim Abbott 427629ca4c emoji: Fix strings in migration 0076.
It's arguably a bug that Django puts the value strings into the
migration object, but this was causing test failures.
2017-04-24 22:50:19 -07:00
Harshit Bansal fb3fda031d models: Add `emojiset` field to `UserProfile` model.
Includes a database migration.
2017-04-24 22:30:06 -07:00
Tim Abbott b92385ea8e attachment: Make path_id field unique.
This will help avoid future bugs similar to that fixed in migration
0072.
2017-04-18 21:27:28 -07:00
Tim Abbott 0f855b84cc attachment: Add migration to fix duplicate attachment objects.
The comment in the migration explains this change in detail.
2017-04-18 21:27:27 -07:00
Umair Khan cf3b6c6ca9 profile: Support custom profile data.
Implements backend for #1760.
2017-04-18 15:20:59 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Rishi Gupta b396be4c98 models: Add index on RealmAuditLog.event_time.
Useful for the upcoming check_realmauditlog_by_user_query, if nothing else.

But I suspect it will indeed get use; looking for events around or within a
certain time is pretty natural for an audit log.

The main argument against I would say is that this should actually be a
joint index with something else. I'm not sure what that something else
should be, so just optimizing for what I think
check_realmauditlog_by_user_query will need for now.
2017-04-14 11:41:07 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Amy Liu 6f061beb46 hotspots: Add backend support for tutorial hotspots.
This commit adds the backend support for a new style of tutorial which
allows for highlighting of multiple areas of the page with hotspots that
disappear when clicked by the user.
2017-03-29 11:34:32 -07:00
Tim Abbott 141469399b RealmAuditLog: Add an extra_data field.
This will be useful for logging any additional details that we might
want to display.
2017-03-27 13:22:58 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
K.Kanakhin fe3213798d retention: Add models to store expired messages.
This addresses part of #106.
2017-03-25 18:12:06 -07:00
Ayush Jain bddcfb1c96 Add realm-level settings to control inline image and url preview.
This gives users more control in case they don't want previews,
especially for the "previews of linked websites" feature.

Fixes: #2640.
2017-03-21 15:46:17 -07:00
adnrs96 8ae35211b5 migration: Sync sizes for existing Attachment objects with actual files.
Tweaked by tabbott to add a nop reverse migration.

Fixes #3883.
2017-03-21 00:53:22 -07:00
Sarah Stringer aa880b0419 Add organization description field to realm settings.
This adds an organization description field to the Realm model, as well as
an input field to the organization settings template. Added three tests.
Set the max length of the field to 100 characters.

Fixes #3962.
2017-03-19 14:05:01 -07:00
Umair Khan 645d7c295c Timezone should default to empty string.
Related to #1506
2017-03-17 21:57:16 -07:00
Umair Khan 242d3ffaf4 Add timezone field in UserProfile.
Implements backend of #1506.
2017-03-15 11:18:24 -07:00
Rishi Gupta 3645bb9225 Change if(realm.domain == mit.edu) to use Realm.is_zephyr_mirror_realm. 2017-03-13 14:17:14 -07:00
adnrs96 556dde8e3e avatar migration: Handle missing avatars for S3 gracefully.
In this commit we start to handle missing avatars gracefully in case
migration was being run on the AWS.
2017-03-08 22:24:07 -08:00
Tim Abbott 7c9ee2f677 avatar migration: Remove now-unnecessary debugging print. 2017-03-07 23:17:11 -08:00
Tim Abbott a32f666f5c avatar migration: Handle missing avatars gracefully.
This makes the outcome if a user didn't have an avatar due to a past
email change reasonable; the user will just be bumped back to
gravatar, fixing their invalid state.
2017-03-07 23:10:15 -08:00
Tim Abbott 0afd4d5740 avatar: Add error handling for avatar migraton. 2017-03-07 23:10:15 -08:00
adnrs96 ae31c6cc0d avatar: Move avatar data from email based to user id based.
This commit introduces a migration for moving avatars from email based
to user id based storage.

This is in responce to change in behaviour of user_avatar_path to
return path comprising of realm id and a hash based on user id. Also
we fix test_helpers accordingly.

Fixes #3776.
2017-03-07 22:44:26 -08:00
Tim Abbott 5488de11e6 migrations: Renumber 0059_userprofile_quota to fix conflict. 2017-03-04 18:38:27 -08:00
Philip Skomorokhov 866a7b06b2 upload: Limit total size of files uploaded by a user to 1GB.
Fixes #3884.
2017-03-04 18:08:30 -08:00
Raghav Jajodia cd2d798498 admin: Added realm option to prevent users from changing their email.
A realm option to prevent users from changing their email address is added.
Fixes #3777.
2017-03-04 17:32:48 -08:00
Tim Abbott d25bfb88d3 lint: Ban importing zerver.models in database migrations.
This doesn't work correctly, but in a subtle way.
2017-03-04 17:32:48 -08:00
Tim Abbott 219c519186 migrations: Fix upgrade process for RealmAuditLog migration.
You aren't allowed to import from `zerver.models` in migrations.
2017-03-04 17:24:04 -08:00
Rishi Gupta 51b7677db7 Add RealmAuditLog table and record user activation/deactivation events.
The RealmAuditLog will make it easier for server admins to replay history.
2017-03-04 16:45:44 -08:00
Sourav Badami 4616ee7762 Enable display of emoji as their alt codes in reactions.
This currently only supports this in emoji reactions, not in actual
emoji in message bodies, but it's a great start for people who want a
text-only view.

Tweaked to update the text by tabbott.

Fixes #3169.
2017-03-03 15:19:34 -08:00
PhilSk 53f3d84af2 attachment: Add 'size' field tracking size of uploaded files.
This tracking will make it possible in the future to limit the total
size of uploads on a per-user or per-organization basis.

Fixes #3774.
2017-03-01 15:58:21 -08:00
K.Kanakhin 257bb40698 realm-icon: Add realm icon feature.
- Add realm icon fields to realm model.
- Add migration for new realm model's field.
- Add views for icon uploading and deleting.
- Add routes for realm icons views.
- Add JS widget for realm icon upload setting.
- Add realm icon upload to administration
  organization setting.
- Add tests for realm icons.

Fixes #3660.
2017-02-26 12:16:07 -08:00
Umair Khan 5bf83f9e0a change-email: Implement confirmation flow.
This adds to Zulip support for a user changing their own email
address.

It's backed by a huge amount of work by Steve Howell on making email
changes actually work from a UI perspective.

Fixes #734.
2017-02-23 03:15:17 -08:00
Tim Abbott 923f4ddd80 models: Remove nullable property from RealmAlias.realm. 2017-02-10 23:53:44 -08:00
Harshit Bansal 7d10cbc32b Add RealmAlias.allow_subdomains to model, frontend, and API.
Includes a database migration.

Fixes #1868.
2017-02-08 22:03:27 -08:00
Yago González 7fe090f902 translations: Improve some poorly-worded strings. 2017-01-28 18:04:17 -08:00
Tim Abbott d6e38e2a5c lint: Clean up E123 PEP-8 rule. 2017-01-23 21:34:26 -08:00
Adarsh S ec44edb305 Add avatar_version to UserProfile.
avatar_version is used to keep track of the avatar version of the
user and to update the cache, everytime the user updates their avatar.
2017-01-23 23:29:07 +05:30
Prabod Rathnayaka 6f087e468e Add setting hiding private message content in desktop notifications.
Tweaked by tabbott to fix a refactoring bug, set the default to True,
fix the real-time sync, and add tests for this.

Fixes #2355.
2016-12-30 14:10:34 -08:00
Rishi Gupta 823659848c Change UserProfile.enter_sends to default to False.
Adds a database migration.
2016-12-29 15:55:23 -08:00
Tim Abbott f46d99ba85 lint: Remove zerver/migrations from main exclude list. 2016-12-27 19:45:33 -08:00
paxapy 8e7fa6b2de emoji: Add add_emoji_by_admins_only realm setting.
This setting controls whether normal users can add realm emoji.

Fixes #978.
2016-12-27 15:46:14 -08:00
Igor Tokarev ffa724f8fc emoji: Add author field to RealmEmoji table and track. 2016-12-27 15:42:04 -08:00
vaibhav 75bf501553 Add optional waiting period for users to create streams.
This adds support for only allowing normal users with account age
equal or greater than a "waiting period" threshold to create streams;
this is useful for open organizations that want new members to
understand the community before creating streams.

If create_stream_by_admins_only setting is set to True, only admin users
were able to create streams. Now normal users with account age greater
or equal than waiting period threshold can also create streams.

Account age is defined as number of days passed since the user had
created his account.

Fixes: #2308.

Tweaked by tabbott to clean up the actual can_create_streams logic and
the tests.
2016-12-15 16:54:30 -08:00
Arpith Siromoney 001847ac5b Add backend support for emoji reactions.
This commit adds the following:

1. A reaction model that consists of a user, a message and an emoji that
are unique together (a user cannot react to a particular message more
than once with the same emoji)
2. A reaction event that looks like:
    {
        'type': 'reaction',
	'op': 'add',
	'message_id': 3,
	'emoji_name': 'doge',
	'user': {
	    'user_id': 1,
            'email': 'hamlet@zulip.com',
            'full_name': 'King Hamlet'
	}
    }
3. A new API endpoint, /reactions, that accepts POST requests to add a
reaction to a message
4. A migration to add the new model to the database
5. Tests that check that
   (a) Invalid requests cannot be made
   (b) The reaction event body contains all the info
   (c) The reaction event is sent to the appropriate users
   (d) Reacting more than once fails

It is still missing important features like removing emoji and
fetching them alongside messages.
2016-11-26 18:10:21 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
Tim Abbott e6f4cb2635 attachment: Remove DB limit on length of filenames.
This fixes an issue where we threw on error on files with long names.
2016-11-15 21:12:05 -08:00
paxapy 4438a09b30 Add migration to create attachments for old uploads.
This migration ensures that all historically uploaded files from
before we started tracking files in the Zulip database via the
Attachment model have Attachment objects.

This has been tested by tabbott against a production server to ensure
that it results in all old uploaded files having corresponding
attachment objects.

Merging this change is a key prerequisite for making our adding
attachment access controls in an enforcing fashion.
2016-11-10 12:14:08 -08:00
umkay 21c024fc29 auth: Make supported authentication backends a bitfield on realm.
This makes it possible to configure only certain authentication
methods to be enabled on a per-realm basis.

Note that the authentication_methods_dict function (which checks what
backends are supported on the realm) requires an in function import
due to a circular dependency.
2016-11-06 16:16:24 -08:00
Rishi Gupta 1944ac844a RealmAlias: Drop uniqueness constraint when REALMS_HAVE_SUBDOMAINS.
Removes the uniqueness constraint on RealmAlias.domain, and adds a function
can_add_alias that checks for uniqueness conditional on
settings.REALMS_HAVE_SUBDOMAINS.
2016-11-03 13:53:01 -07:00
Rishi Gupta c0f1b5d9f6 Change Realm defaults to Community defaults.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.

We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.

This commit adds a database migration.
2016-11-03 13:53:01 -07:00
Rishi Gupta 9ef8536cc6 models.Realm: Require Realm.string_id to be non-NULL.
Adds a database migration, adds a new string_id argument to the management
realm creation command, and adds a short name field to the web realm
creation form when REALMS_HAVE_SUBDOMAINS is False.
2016-11-02 22:46:34 -07:00
Rishi Gupta 64bcd71d6e models.Realm: Rename subdomain to string_id.
Does a database migration to rename Realm.subdomain to
Realm.string_id, and makes Realm.subdomain a property.  Eventually,
Realm.string_id will replace Realm.domain as the handle by which we
retrieve Realm objects.
2016-11-02 22:46:34 -07:00
K.Kanakhin 39e0886361 retention-policy: Add tool to determine expired messages.
This is a first step towards implementing a message retention policy
feature.

- Add Realm model message_retention_days field to setup
  messages expired period for realm.
- Add migration.
- Add tool to get expired messages for each Realm.
- Add tests to cover tool for getting expired messages.
2016-10-25 15:38:08 -07:00
Tomasz Kolek 4790316b57 Add user setting option to always send push notifications.
Add option in user's settings for getting mobile push notifications
even if a Zulip browser is online.  Default is False.

Fixes: #1596.
2016-10-25 10:52:29 -07:00
Rishi Gupta 537ee75761 Create RealmAlias entries for Realm.domain.
This is a preliminary step towards eliminating the realm.domain field
in favor of realm.subdomain.  Includes a database migration to create
these for existing realms.
2016-10-25 10:06:39 -07:00
Brock Whittaker fca61b2031 Add medium size avatars for use on the user's own settings page.
This adds a medium (500px) size avatar thumbnail, that can be
referenced as `{name}-medium.png`.  It is intended to be used on the
user's own settings page, though we may come up with other use cases
for high-resolution avatars in the future.

This will automatically generate and upload the medium avatar images
when a new avatar original is uploaded, and contains a migration
(contributed by Kirill Kanakhin) to ensure all pre-existing avatar
images have a medium avatar.

Note that this implementation does not provide an endpoint for
fetching the medium-size avatar for another user.

[substantially modified by tabbott]
2016-10-25 09:42:14 -07:00
Tim Abbott 893679baff Add migration to remove system avatar source.
Fixes the last commit having broken master.
2016-10-22 20:07:21 -07:00
Rishi Gupta 777fcaa6a0 Add new organization type field to Realm objects.
Adds a new field org_type to Realm.  Defaults for restricted_to_domain
and invite_required are now controlled by org_type at time of realm
creation (see zerver.lib.actions.do_create_realm), rather than at the
database level.  Note that the backend defaults are all
org_type=corporate, since that matches the current assumptions in the
codebase, whereas the frontend default is org_type=community, since if
a user isn't sure they probably want community.

Since we will likely in the future enable/disable various
administrative features based on whether an organization is corporate
or community, we discuss those issues in the realm creation form.
Before we actually implement any such features, we'll want to make
sure users understand what type of organization they are a member of.

Choice of org_type (via radio button) has been added to the realm
creation flow and the realm creation management command, and the
open-realm option removed.

The database defaults have not been changed, which allows our testing code
to work unchanged.

[includes some HTML/CSS work by Brock Whittaker to make it look nice]
2016-10-05 17:01:46 -07:00
hackerkid ea39fb2556 Add option for hosting each realm on its own subdomain.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.

This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
  for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
  enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
  (but does not restrict the API; that will be tightened in a future commit).

Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).

[substantially modified by tabbott as part of merging]
2016-09-27 23:24:14 -07:00
Rishi Gupta 708b416ca1 Terms of Service: Fix corner cases around new users being created.
This fixes a few bugs in 7910a6e134,
related to automatically created user accounts.
2016-08-13 00:19:54 -07:00
Rishi Gupta 7910a6e134 Terms of Service: Add ability to update TOS and have users re-sign.
Most directly useful for the migration to zulipchat.com.

Creates a new field in UserProfile to store the tos_version, as well as two
new settings TOS_VERSION and FIRST_TIME_TOS_TEMPLATE. We check for a version
mismatch between what the user has signed and the current
settings.TOS_VERSION whenever the user hits the home page, and redirect them
if needed.

Note that accounts_accept_terms.html and
zerver.views.accounts_accept_terms were unused before this commit
(they date from c327446537)
2016-08-11 23:37:48 -07:00
Rag Sagar 2fef36f15a Add realm-level default language setting.
Adds a new field default language in the zerver_realm model.
This realm level default language will be used as default language
for newly created users. Realm level default language can be
changed from the administration page.

Fixes #1372.
2016-08-09 17:38:29 -07:00
Tim Abbott df525ad1c5 Remove old MitUser model and related code.
The MitUser model caused a constant series of little problems for
users with mit.edu email addresses trying to sign up for different
Zulip servers.

The new implementation just uses conditionals on the realm object when
selecting the confirmation template to use.
2016-07-26 20:30:12 -07:00
Rishi Gupta d529a94e4d Add realm setting to time-limit editing of message content.
This is controlled through the admin tab and a new field in the Realms table.
Notes:
* The admin tab setting takes a value in minutes, whereas the backend stores it
  in seconds.
* This setting is unused when allow_message_editing is false.
* There is some generosity in how the limit is enforced. For instance, if the
  user sees the hovering edit button, we ensure they have at least 5 seconds to
  click it, and if the user gets to the message edit form, we ensure they have
  at least 10 seconds to make the edit, by relaxing the limit.
* This commit also includes a countdown timer in the message edit form.

Resolves #903.
2016-07-15 13:55:49 -07:00
Eklavya Sharma 958335bdb3 Change all default values in models to unicode. 2016-07-11 21:30:32 -07:00
Rishi Gupta 43c2f35776 Add realm setting to disable message editing.
This is controlled through the admin tab and a new field in the Realms
table.  This mirrors the behavior of the old hardcoded setting
feature_flags.disable_message_editing.  Partially resolves #903.
2016-07-10 11:57:24 -07:00
Eklavya Sharma d8de54abe5 zerver/migrations: Squash an AlterField from 0002 into 0001.
Squash the AlterField on UserProfile.groups in 0002 into the
AddField in 0001.  This is done to avoid a probable bug in Django,
where running migrations in python 3 sometimes led to a KeyError.
2016-07-06 11:40:12 -07:00
Eklavya Sharma 2d4dcd1698 Replace bytes by strings in zerver migrations 2, 13, 18. 2016-07-04 12:08:37 -07:00
Eklavya Sharma 6553e16b89 Replace bytes by strings in zerver/migrations/0001. 2016-07-04 12:08:37 -07:00
Umair Khan 035fceb814 Add dynamically loaded language dropdown. 2016-07-04 11:56:02 -07:00
Kartik Maji f8bb7503e6 Add ability to pin streams to top of the streams sidebar list.
Based on work by Lauren Long, with some tweaks by tabbott.
2016-06-30 22:26:09 -07:00
Tim Abbott 56e7a2f6f3 Annotate zerver.migrations. 2016-06-28 16:38:33 -07:00
rahuldeve 8cecb37743 Modify Attachment model to track file access permissions. 2016-06-23 17:46:16 -07:00
Vishnu Ks ad1c3894d9 Add interface for creating new realms.
This is controlled by settings.OPEN_REALM_CREATION; if that setting is
off, this feature doesn't do anything.
2016-06-17 16:15:28 -07:00
Tim Abbott f88e5b7438 Add missing no-op migration for realm_emoji.
Because of how Django's migration system works, changing the error
message attached to a model field's validator results in an extra
migration.
2016-06-15 09:26:01 -07:00
Tomasz Kolek 8c18b8947f Add bot_type field to UserProfile.
This is intended to support creating different types of bots with
potentially limited permissions.
2016-05-19 22:37:37 -07:00
Aristeidis Fkiaras 3ee210d9e8 Add setting to only allow admins create new streams.
Fixes: #691.

Thanks to Preston Hansen for work on this feature!
2016-05-18 18:53:13 -07:00
rahuldeve dde832b158 Add Attachment model to keep track of uploads.
This commit adds the capability to keep track and remove uploaded
files.  Unclaimed attachments are files that have been uploaded to the
server but are not referred in any messages.  A management command to
remove old unclaimed files after a week is also included.

Tests for getting the file referred in messages are also included.
2016-05-02 22:14:47 -07:00
Tim Abbott d9e4968d6f Increase maximum URL length for RealmEmoji to 1000.
The default of 200 was shorter than the Camo URLs use by Zulip.
2016-05-02 19:02:56 -07:00
Vladislav Manchev f5e6176aea Add custom realm emoji UI to administration page. 2016-04-26 13:15:54 -07:00
Tim Abbott 79297898f1 Remove obsolete AppleDeviceToken model. 2016-04-20 21:51:52 -07:00
Tim Abbott 49799440a4 Replace use of django-guardian with fields on UserProfile.
As documented in https://github.com/zulip/zulip/issues/441, Guardian
has quite poor performance, and in fact almost 50% of the time spent
running the Zulip backend test suite on my laptop was inside Guardian.

As part of this migration, we also clean up the old API_SUPER_USERS
variable used to mark EMAIL_GATEWAY_BOT as an API super user; now that
permission is managed entirely via the database.

When rebasing past this commit, developers will need to do a
`manage.py migrate` in order to apply the migration changes before the
server will run again.

We can't yet remove Guardian from INSTALLED_APPS, requirements.txt,
etc. in this release, because otherwise the reverse migration won't
work.

Fixes #441.
2016-04-20 21:51:52 -07:00
Tim Abbott b61d73fc93 Delete unused old StreamColor model. 2016-04-08 13:06:04 -07:00
Tim Abbott 0d40473818 Add type: ignore for empty list mypy bug. 2016-04-03 15:40:24 -07:00
Tim Abbott 5ef57a07e1 Add missing migrations present in models.py.
89a2765553 didn't include the database
migration corresponding to the change, which means it didn't take full
effect when it was merged.

I noticed this because `manage.py makemigrations` would generate these
migrations; that suggests a good idea for a test to add.
2016-01-26 20:38:46 -08:00
Tim Abbott dd4ca2f934 Add case-insensitive index on PreregistrationUser.email.
This fixes a performance issue joining a server with a large number of
users.

Thanks to @dbiollo for the suggestion!
2016-01-09 20:01:37 -08:00
Tim Abbott 2ea0fce47e Add UserProfile indexes on is_active and is_bot.
Since we frequently do filters for only active users, these indexes
may help performance in some cases.
2016-01-09 20:01:37 -08:00
Tim Abbott ebcb569c96 Add case-insensitive index on UserProfile.email.
This fixes a performance issue looking up UserProfile objects for
realms with a large number of users in the case that a UserProfile
object is not in the cache.

Thanks to @dbiollo for the suggestion!
2016-01-09 20:01:37 -08:00
Tim Abbott 858d0a984b Move API super users configuration into the database.
(imported from commit 3cc702f93e7252b42930dba4bde93a915b6dbf44)
2015-09-20 11:13:20 -07:00
David Roe 3ac95ddc1a Add UserProfile flag to control whether we have a left side userlist.
Previously this was hardcoded for a single customer.

(imported from commit a6b7095050aa10cef976541505d9b09a35453f48)
2015-09-19 23:22:59 -07:00
Reid Barton 36716fe518 Add remaining custom indexes that were created by South migrations
(imported from commit 9798afa8161af4ae6b3fa0c5f4894a3211b77cd4)
2015-08-23 21:47:12 -07:00
Reid Barton 827a825c06 Schema migrations for Django 1.8
These are the result of either the upgrade to Django 1.8 itself
(username max length increased to 254), or the changes needed for
Django 1.8 compatibility.

(imported from commit 6b1d7e73c85e9a2f7de9e5b91d851977eb4959e8)
2015-08-22 13:51:35 -07:00
Reid Barton 599742536b Switch from South to native Django migrations
This commit loses some indexes, unique constraints etc. that were
manually added by the old migrations. I plan to add them to a new
migration in a subsequent commit.

(imported from commit 4bcbf06080a7ad94788ac368385eac34b54623ce)
2015-08-20 23:01:26 -07:00
Kate Buckner 4d0f7c7ea4 Add a user-visible setting for 24-hour time display.
(imported from commit d934824fd6b72e64a455aac9ff4585b262145f02)
2015-08-20 17:33:16 -07:00
David Roe 086d8eee22 Add name_changes_disabled realm field.
(imported from commit 6b04ba2f7ad64c44f2ef18302f0fbd819259d632)
2015-08-20 15:29:46 -07:00
David Roe 5b7f3466ba Add feature where only admins can invite new users.
This is controlled through the admin tab and a new field in the Realms table.

(imported from commit e78a6f48160e2a1bbc68d278beb726fe31515266)
2015-08-20 15:29:46 -07:00
David Roe 809efc4f2b Add realm option to show or not show digest e-mail.
(imported from commit e9cfe519aa9f2857a1dcc7c75f5e1889834fcb86)
2015-08-20 15:29:45 -07:00
David Roe 90e2f5053f Add mandatory topics as a realm option.
(imported from commit 929a884b8610669aa24a167367b899683e33a045)
2015-08-20 15:29:45 -07:00
David Roe 472898cfc6 Allow adding users to realms more easily in Dev VM.
Include new field on Realm to control whether e-mail invitations are required
separately from whether the e-mail domain must match.
Allow control of these fields from admin panel.
Update logic in registration page to use these fields.

(imported from commit edc7f0a4c43b57361d9349e258ad4f217b426f88)
2015-08-19 22:24:53 -07:00
Leo Franchi 1c22b48bb1 Add an optional ios_app_id field to PushDeviceToken
(imported from commit 32d6efd1ecc58297b83e515350e6daac3bce8469)
2015-02-11 06:57:22 +00:00
Zev Benjamin 298e008cc2 [manual] Turn off GIN fast update for zephyr_message_search_tsvector
Apply this commit after hours!

To apply this commit, first run the migration and then run the following as the
zulip user on staging:

$ echo 'VACUUM zerver_message' | python manage.py dbshell

The above VACUUM is needed to clean out the existing fast update pending list.
It might take a long time and block new message inserts!

See discussion near Zulip message 18377486 for why we're turning off the fast
update mechanism for zephyr_message_search_tsvector.

The high level overview is:
As a consequence of the high work_mem setting on our postgres server, the
fastupdate pending list for zephyr_message_search_tsvector can grow very large.
This leads to the occasional INSERT or UPDATE taking inordinately long (many
minutes) as the pending list is flushed, blocking other inserts.

One other possible solution for preventing the list from growing too large is to
set the autovacuum storage parameters on the table such that the autovacuum
process will run after a reasonable number of INSERTs or UPDATEs.  However, the
table is mostly INSERT-only.  Therefore, only the autovacuum_analyze_*
parameters will actually do anything to affect when the autovacuumer will run,
but when it does, it will do a VACUUM ANALYZE instead of a plain VACUUM.  We
don't particularly need the table to be re-analyzed that often.

Turning off fast update will eventually cause the index to become less
efficient, but we can always rebuild it later if we notice it starting to get
too slow.

(imported from commit f280c193c3bc0a3f312960510c5a7dcf97f30c3d)
2014-05-20 22:38:48 -07:00
Jason Michalski f3180b774b [schema] Add default user_profile options for register events
Allow bot owners to set which streams their will receive events for
without needing to change a configuration file.

(imported from commit 2b69e519dbc12ffbdba072031a7f7196c9e50e33)
2014-03-05 14:16:18 -05:00
Jason Michalski de545d5fa0 [schema] Add a default to stream option to user profile
This allows bot owners to configure which streams messages are delivered
to without needing to change webhook URLs or configuration files.

(imported from commit 32a0c26657c145b001cd8cb3ce0a0364d48902ce)
2014-03-05 14:16:18 -05:00
Jessica McKellar 600786983c [schema] Split Subscription notifications field into audible + desktop.
(imported from commit e160b9eb958fa5a06f990fccdeb25fa39e13e753)
2014-03-03 16:08:31 -05:00
Jessica McKellar df39a7bde2 [schema] Add separate notification settings for streams.
(imported from commit 2122a4cc35e9a1f019eea004cd27ce703ba5a55a)
2014-03-03 16:08:30 -05:00
Zev Benjamin db4770a577 [schema] Create new Message indices for development machines
This migration will do nothing on staging/prod since the indices already exist.
It is only for creating the indices in dev.

(imported from commit ac26a23641191ba73fbccc2eebc4a261ece6c624)
2014-03-01 12:41:45 -05:00
Steve Howell 0f96f5064e [schema] Add has_attachment/has_image/has_link to Message.
We will need to run these commands manually when deploying to staging:

    CREATE INDEX CONCURRENTLY "zerver_message_has_attachment" ON "zerver_message" ("has_attachment");
    CREATE INDEX CONCURRENTLY "zerver_message_has_image" ON "zerver_message" ("has_image");
    CREATE INDEX CONCURRENTLY "zerver_message_has_link" ON "zerver_message" ("has_link");

(imported from commit 84808dc6b1af887ddf784cb8a875ae462f4df985)
2014-03-01 12:41:00 -05:00
Jessica McKellar f3d33c1aa0 Add missing stream field to schema migration 0067.
(imported from commit 005743859f03a1a8a154099a72bfbf50d5db890e)
2014-02-05 13:35:41 -05:00
Jessica McKellar 99d58fe8b5 [schema] at a `deactivated` field to Stream.
From a user's perpsective, the stream has been deleted. From the
database's perspective, the stream has been deactivated -- the stream
messages still exist.

(imported from commit b08b30b2a822663e17d64182af1fb160c2193344)
2014-01-29 12:41:20 -05:00
Jason Michalski 4104f00229 [schema] Add description to streams and display it
A description was added to the streams and it is now displayed on the
subscriptions page. It can not be set in the UI yet.

(imported from commit 81d08b65eee42dba87cd99dd5bd30106c4eb6c6a)
2014-01-24 14:47:44 -05:00
Jason Michalski 4c90392261 [schema] Add an per-user option to disable notification on new streams
Added a default_desktop_notifications boolean to userprofile with a UI
in Zulip Labs. This flag is used to default the notification flag on new
subscriptions.

(imported from commit a25223cc5ecf09980cf877991e25034bb3fd4046)
2014-01-16 20:13:34 -05:00
Tim Abbott 47500d8352 [schema] Add a bit for whether inactive users are mirror dummies.
(imported from commit bb21bb2c62ac09742484d7a4ca8907e7d864b982)
2014-01-09 11:08:35 -05:00
Jessica McKellar a06b5bd720 [schema] Add a deactivated field to Realm.
(imported from commit 8b873296d45dbacc0865c8d101a872be0b65ed46)
2014-01-07 20:24:20 -05:00
acrefoot 0fd492a817 [schema] Add enable_autoscroll_past_unread setting
This is for the CUSTOMER28 folks, so that they can turn Zulip into a more "chat client" thing.

(imported from commit 373a8afae4998fce5560e7b2bd13804c8fbb39fc)
2013-12-13 11:50:03 -05:00
Kevin Mehall e4589700b6 [schema] Modify device token to support both iOS and Android
This replaces the AppleDeviceToken table with a generic
PushDeviceToken with a `kind` field to make it easier to add functionality
like per-device/per-stream settings that share code between Android and
iOS devices.

The schema must continue to work on prod with the old table name, so we
add the new table in parallel and can drop the old table once this code
hits prod and any necessary data is copied.

(imported from commit 0209a7013f2850ac6311f23c3d6f92c65ffd19e3)
2013-12-11 15:37:47 -05:00
Leo Franchi 2c4efecee2 [schema] Add data model for managing realm filters via the database.
(imported from commit c624cdda9c23d9ca42706bf64b4c8a4e69e95c50)
2013-12-11 14:39:09 -05:00
Jessica McKellar 1b068d5433 [schema] add boolean field enable_digest_emails to UserProfile.
We'll use this to track the user preference for receiving digest
e-mails.

(imported from commit a56644b1a359444ded45f9437c954194b9ae47a1)
2013-12-02 18:58:03 -05:00
acrefoot f7b5a10da0 [schema] Add ScheduledJob table, and update mandrill related code
ScheduledJobs with type Email displace the usual mandrill codepaths
in the Zulip Enterprise deploys

* Email-specific helper functions will appear in deliver_email.py
* 0058_auto__add_scheduledjob.py

(imported from commit 8db08d8a279600322acfdbed792dc1a676f7a0ab)
2013-11-13 16:41:36 -05:00
Leo Franchi 62ba16a829 [schema] Allow a realm to have aliases for multiple other domains
(imported from commit 5d0340afd4c77f50db63550a3ec8d81e885c285c)
2013-11-13 14:17:43 -05:00
Tim Abbott 2de31ee287 [schema] Remove the UserMessage.archived field.
This was a precursor to UserMessage.flags.read that never got used
because we decided to use django-bitfield.

(imported from commit 868754723c07ee9b85ae951aee785e571ccfef97)
2013-11-04 07:31:11 -05:00
Zev Benjamin 7b6f6823a5 [manual] Change references to the humbug user, schema, and database to zulip
This commit must be simultaneously deployed on both staging and
prod0.  It also requires completely taking down the app.

To deploy these changes, do:
* check out this commit at /root/zulip on postgres0, postgres1, staging, and prod0

* stop the process_fts_updates job on postgres0 and postgres1
* stop the app on staging and prod0

* do a puppet apply on postgres0, postgres1, staging, and prod0
* move the new client certificates into place on staging and app
* move the new server certificates into place on postgres0 and postgres1
* reload the database config on postgres0 and postgres1 (this might
  actually require a restart)
* run tools/migrate-db on postgres0 as root

* do a deploy through this commit on staging and prod0
* start the process_fts_updates job on postgres0 and postgres1

* do a puppet apply on nagios

(imported from commit 819bdd14326c1425e2d3041a491a8ca3b9716506)
2013-10-26 04:16:27 -04:00
Leo Franchi 05564ec0e2 [schema] Add a config option for enabling/disabling push notifications
(imported from commit 5649188cd5e0e4edb6e71559ec7d7fc5e92efaf8)
2013-10-24 14:54:30 -04:00
Leo Franchi 1d82704b8d [schema] Add a last_updated field to AppleDeviceToken
(imported from commit 2ea91e462773c50591585a610b0de63625ad2b7e)
2013-10-24 14:54:30 -04:00
Tim Abbott eaaf0fec04 [schema] Add case-insensitive index on stream names.
We need to run the schema migration manually using

"CREATE INDEX CONCURRENTLY upper_stream_name_idx ON zerver_stream ((upper(name)));"

since we need CONCURRENTLY and I seem to recall that doesn't work with South.

This significantly improves the uncached performance of get_stream()
(e.g. from 32ms to 9ms).  At present, this codepath is not used
particularly heavily since we do cache the stream names and do most of
our filtering by recipient ID, but the index isn't expensive and does
provide a significant improvement in the uncached case.

(imported from commit 4d28dc2e9a02d0602861b165393d90ed18f5f4c8)
2013-10-18 16:38:31 -04:00
Tim Abbott 4d94d4d6f6 [schema] Add case-insensitive index on message subject field.
We need to run the schema migration manually using

"CREATE INDEX CONCURRENTLY upper_subject_idx ON zerver_message ((upper(subject)));"

since we need CONCURRENTLY and I seem to recall that doesn't work with South.

Apparently our existing indexes on subject/topic weren't being used in
our narrowing queries, because we do case-insensitive search.

This substantially improves our database performance around
stream+topic narrows.  See before and after query plans below from my
test instance.

humbug=# explain analyze SELECT "zerver_message"."id" FROM "zerver_message" WHERE ("zerver_message"."recipient_id" = 38  AND UPPER(zerver_message.subject) = 'TEST'  AND "zerver_message"."id" <= 348495 ) ORDER BY "zerver_message"."id" DESC LIMIT 50;
                                                                       QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=13510.61..13510.71 rows=41 width=4) (actual time=32.952..32.958 rows=2 loops=1)
   ->  Sort  (cost=13510.61..13510.71 rows=41 width=4) (actual time=32.946..32.947 rows=2 loops=1)
         Sort Key: id
         Sort Method: quicksort  Memory: 25kB
         ->  Bitmap Heap Scan on zerver_message  (cost=237.99..13509.51 rows=41 width=4) (actual time=2.357..32.912 rows=2 loops=1)
               Recheck Cond: (recipient_id = 38)
               Filter: ((id <= 348495) AND (upper((subject)::text) = 'TEST'::text))
               ->  Bitmap Index Scan on zephyr_message_recipient_id  (cost=0.00..237.98 rows=8221 width=0) (actual time=1.178..1.178 rows=10354 loops=1)
                     Index Cond: (recipient_id = 38)
 Total runtime: 33.049 ms
(10 rows)

humbug=# explain analyze SELECT "zerver_message"."id" FROM "zerver_message" WHERE ("zerver_message"."recipient_id" = 38  AND UPPER(zerver_message.subject) = 'TEST'  AND "zerver_message"."id" <= 348495 ) ORDER BY "zerver_message"."id" DESC LIMIT 50;
                                                                          QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=435.11..435.22 rows=41 width=4) (actual time=4.998..4.999 rows=2 loops=1)
   ->  Sort  (cost=435.11..435.22 rows=41 width=4) (actual time=4.997..4.997 rows=2 loops=1)
         Sort Key: id
         Sort Method: quicksort  Memory: 25kB
         ->  Bitmap Heap Scan on zerver_message  (cost=275.63..434.02 rows=41 width=4) (actual time=4.981..4.984 rows=2 loops=1)
               Recheck Cond: ((upper((subject)::text) = 'TEST'::text) AND (recipient_id = 38))
               Filter: (id <= 348495)
               ->  BitmapAnd  (cost=275.63..275.63 rows=41 width=0) (actual time=4.954..4.954 rows=0 loops=1)
                     ->  Bitmap Index Scan on upper_subject_idx  (cost=0.00..37.38 rows=1744 width=0) (actual time=2.972..2.972 rows=27457 loops=1)
                           Index Cond: (upper((subject)::text) = 'TEST'::text)
                     ->  Bitmap Index Scan on zephyr_message_recipient_id  (cost=0.00..237.98 rows=8221 width=0) (actual time=0.855..0.855 rows=10354 loops=1)
                           Index Cond: (recipient_id = 38)
 Total runtime: 5.049 ms
(13 rows)

(imported from commit 1f4815ccb0691053ff8d505149482dbc74153fb3)
2013-10-18 16:38:31 -04:00
Jessica McKellar 5c5ffd6ea3 [schema] Add a name field to zerver_realm.
(imported from commit 5b5d7a40d5dff11fe9ca6624b3794cb9c8a3520a)
2013-10-17 13:47:51 -04:00
Leo Franchi f7386c3f58 [schema] Add support for keeping track of iOS APNS device tokens
In order to support iOS Push Notifications, we need to keep track
of a device's unique APNS Token. These are delivered to our iOS
code after registering for remote notifications

(imported from commit bbe34483e1380dc20a1c93e3ffa1fcfdb9087e67)
2013-10-16 12:54:28 -04:00
Tim Abbott 32861af323 [schema] Increase maximum stream name length to 60.
(imported from commit 83e9434178909d18481692a9e7fe3b8e1802bd7e)
2013-10-15 09:13:35 -04:00
Leo Franchi 72c7c5b836 [schema] Give new realms a default notifications stream, and send signups/creations to it
(imported from commit 19ad761f7d9eabb865ac3526bf66dff8b7ed0224)
2013-10-07 15:46:13 -04:00
Jessica McKellar ed0fd51160 [schema] Add a date_created field to zerver_stream.
(imported from commit 1360da91b81fad59d49068fdf05205eef982e64a)
2013-09-27 16:23:56 -04:00
Jessica McKellar f22023d4cb [schema] Add a date_created field to Realm.
(imported from commit 1953f7c1f8af7f81d50d7362c704f6a81637d72e)
2013-09-24 16:19:52 -04:00
Steve Howell 21456091e4 [schema] Add muted_topics to UserProfile
(imported from commit 5d804259be94ba83dc4f7541e50443dd1a42cc6b)
2013-09-11 16:47:37 -04:00
Tim Abbott 9fcca3df4e [schema] Collect data on when users are active on site.
These engagement data will be useful both for making pretty graphs of
how addicted our users are as well as for allowing us to check whether
a new deployment is actually using the product or not.

This measures "number of minutes during which each user had checked
the app within the previous 15 minutes".  It should correctly not
count server-initiated reloads.

It's possible that we should use something less aggressive than
mousemove; I'm a little torn on that because you really can check the
app for new messages without doing anything active.

This is somewhat tested but there are a few outstanding issues:

* Mobile apps don't report these data.  It should be as easy as having
  them send in update_active_status queries with new_user_input=true.

* The semantics of this should be better documented (e.g. the
  management script should print out the spec above)x.

(imported from commit ec8b2dc96b180e1951df00490707ae916887178e)
2013-09-10 13:25:59 -04:00
Leo Franchi b2ddd670e4 [schema] Add backend support for per-user alert words
(imported from commit 7a9c596a010cbedbddf594c5d9c68bb9ed46d122)
2013-09-05 10:18:40 -04:00
Kevin Mehall 32aa0bb72d [migration] Add RealmEmoji model for realm-specific emoji.
(imported from commit d0a08b748f458c0ab68985b4cf00c78950ab40c4)
2013-08-23 21:39:29 -04:00
Jessica McKellar befd34d111 [schema][manual] Make Stream.email_token required.
This commit CANNOT be deployed until the previous schema change
([schema] models: add an email_token field to Streams) is on prod.

Before applying this schema change, run the populate-stream-tokens
management command to generate tokens for streams that need them.

(imported from commit 7adc81c8c317ec5d59dd59ba42a4dc1a46174007)
2013-08-13 14:28:47 -04:00
Jessica McKellar c020545e02 [schema] models: add an email_token field to Streams.
The e-mail forwarder will use this. Set it to nullable temporarily to
accomodate existing streams; later commits will a) provide a script to
give all streams a token, and b) make the field non-null.

Realm administrators will eventually have a UI to regenerate stream
tokens.

(imported from commit a084d0a7012eb9665e4da095cbc46aa9ef354eaa)
2013-08-09 14:59:26 -04:00
Tim Abbott e111a2f9a5 [manual] Rename Django app from zephyr to zerver.
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).

At the time it is deployed, we need to make a few changes directly in
the database:

(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';

(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
2013-08-06 07:39:36 -04:00