Commit Graph

273 Commits

Author SHA1 Message Date
neiljp (Neil Pilgrim) f3ef2a186c mypy: Enable strict-optional for calculate_first_visible_message_id. 2019-05-01 10:49:24 -07:00
Vishnu Ks ecdd3bea43 billing: Add cron job to run invoice_plans once a day.
Fixes #11960
2019-04-29 11:23:17 -07:00
Rishi Gupta e8741c448d refactoring: Move set_default_streams into do_create_realm.
After the commits leading up to this, the only meaningful use of this
function was in the realm creation process.
2019-03-21 12:33:19 -07:00
Rishi Gupta e71a1a2b4e onboarding: Remove initial streams other than general and core team.
The hope is that by having a shorter list of initial streams, it'll
avoid some potential confusion confusion about the value of topics.
At the very least, having 5 streams each with 1 topic was not a good
way to introduce Zulip.

This commit minimizes changes to the message content in
`send_initial_realm_messages` to keep the diff readable. Future commits will
reshape the content.
2019-03-21 12:30:14 -07:00
Tim Abbott 12756b48a0 populate_db: Fix help text for --test-suite. 2019-02-20 17:42:30 -08:00
Eeshan Garg 179b747769 streams: Refactor multi-option helpers into separate functions.
For internal stream messages, most of the time, we have access to
a Stream object. For the few corner cases where we don't, it is a
much cleaner approach to have a separate function that accepts a
stream name than having one multi-option helper that accepts both
names and objects.
2019-02-12 11:10:26 -08:00
Eeshan Garg c240008edb internal_prep_stream_message: Support accepting a Stream object.
If the caller has access to a Stream object, it is wasteful to
query a database for a stream by ID or name. In addition, not
having to go through stream names eliminates various classes of
possible bugs involved with getting a Stream object back.
2019-02-08 08:53:16 -08:00
Anders Kaseorg f5197518a9 analytics/zilencer/zproject: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:31:45 -08:00
Anders Kaseorg 79d888223f profiling: Use mkstemp for profile.data filename.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-01-15 16:08:48 -08:00
Tim Abbott e603237010 email: Convert accounts code to use delivery_email.
A key part of this is the new helper, get_user_by_delivery_email.  Its
verbose name is important for clarity; it should help avoid blind
copy-pasting of get_user (which we'll also want to rename).
Unfortunately, it requires detailed understanding of the context to
figure out which one to use; each is used in about half of call sites.

Another important note is that this PR doesn't migrate get_user calls
in the tests except where not doing so would cause the tests to fail.
This probably deserves a follow-up refactor to avoid bugs here.
2018-12-06 16:21:38 -08:00
Vishnu Ks 2889de6d90 populate_db: Extract subscribe_users_to_streams function. 2018-11-29 12:15:58 -08:00
Tim Abbott 9c224ccdd3 populate_db: Suppress logging output for push notifications.
This fixes a lot of spammy output of the form:

2018-11-27 17:46:48.279 INFO [zerver.lib.push_notifications] Sending push notification to user 46

when running populate_db, which is both confusing (since we're not
actually sending push notifications here) and spammy.
2018-11-27 09:50:17 -08:00
Tim Abbott e8290ccd6d add_new_user: Fix mention of running "python manage.py".
This won't work on Python 3.
2018-11-06 13:28:18 -08:00
Vishnu Ks c01ebaa8d8 billing: Move billing management commands from zilencer to corporate. 2018-10-01 15:43:55 -07:00
Vishnu Ks 93b398bc0a billing: Move zilencer/lib/stripe to corporate. 2018-10-01 15:43:44 -07:00
Rishi Gupta bebd93b571 billing: Fix idempotency bug in setup_stripe management command. 2018-08-31 17:49:34 -07:00
Rishi Gupta eec07072ee billing: Add coupons. 2018-08-24 17:27:26 -07:00
Rishi Gupta c3a912f8b6 management commands: Add sleep_forever to lib/management. 2018-08-17 09:20:51 -07:00
Rishi Gupta 27d4b1a5f0 billing: Add RealmAuditLogProcessor. 2018-08-13 17:12:50 -07:00
Yago González 6a192ac84c utils: Move random API key generator as generate_api_key.
random_api_key, the function we use to generate random tokens for API
keys, has been moved to zerver/lib/utils.py because it's used in more
parts of the codebase (apart from user creation), and having it in
zerver/lib/create_user.py was prone to cyclic dependencies.

The function has also been renamed to generate_api_key to have an
imperative name, that makes clearer what it does.
2018-08-08 16:45:25 -07:00
Yago González f6219745de users: Get all API keys via wrapper method.
Now reading API keys from a user is done with the get_api_key wrapper
method, rather than directly fetching it from the user object.

Also, every place where an action should be done for each API key is now
using get_all_api_keys. This method returns for the moment a single-item
list, containing the specified user's API key.

This commit is the first step towards allowing users have multiple API
keys.
2018-08-08 16:35:17 -07:00
Roman Godov c0806917ec models: Rename Realm.restricted_to_domain field.
This renames Realm.restricted_to_domain field to
emails_restricted_to_domains, for greater clarity as to what it does
just from seeing the setting name, without having to look it up.

Fixes part of #10042.
2018-07-31 09:28:33 -07:00
Vishnu Ks ff4c1ca2c1 models: Add SUBSCRIPTION_CREATED event type constant to RealmAuditLog. 2018-07-10 15:42:26 +05:30
Shubham Dhama 7cbacdb8c2 populate_db: Integrate populate_analytics_db.
As detailed in the documentation changes, this simplifies the
development workflow for doing UI work on the /stats pages.

The cost is a ~10% increase the time it takes to run `populate_db`,
which doesn't happen very often (and for most purposes manifests as a
1% increase in the time it takes to rebuild the database from scratch).
2018-07-09 15:05:48 +05:30
Rishi Gupta 16334a1ba7 billing: Update billing system. 2018-07-09 14:33:07 +05:30
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Yashashvi Dave 8909cb1d15 custom fields: Allow list of users in user type of custom fields.
Allow user to add more than one user-value in user type of custom
fields.

Tweaked by tabbott to improve the models.py code and type annotations.
2018-06-16 09:37:49 -07:00
Rishi Gupta 634c4516a7 portico: Add svgs of sea creatures. 2018-06-06 13:57:55 -07:00
Tim Abbott 27e7770540 populate_db: Initialize the outgoing webhook bot properly.
Previously, we created the bot, but didn't initialize its service
object.
2018-06-01 08:48:17 -07:00
Robert Hönig 497cd4a1f9 outgoing webhooks: Change default URL in all tests.
This reflects the changes to the default URL publicly
displayed to the user. It also changes the default
URL of the default test server outgoing webhook, which
prevented the test server flaskbotrc from working out
of the box.
2018-05-28 10:40:44 -07:00
Yashashvi Dave e82c879b85 custom fields: Add user type of custom fields.
Fixes #8878
2018-05-27 23:01:21 -07:00
Aditya Bansal 8cfb437677 provision: Don't run create_realm_internal_bots management command.
We essentially stop running create_realm_internal_bots during
every provisioing and move its operations to run from populate db.
In fact to speed things up a bit we actually make populate db call the
funcs which create_realm_internal_bots calls behind the scenes.

Fixes: #9467.
2018-05-23 13:34:52 +05:30
Tim Abbott 1b47b2e7cd populate_db: Fix an out-of-date mypy annotation. 2018-05-22 14:46:03 -07:00
Tim Abbott 8a64d8ef06 populate_db: Fix stream colors being generated wrong.
Previously, the stream colors index i was accidentally a function only
of the user, so each user got the same color for all their streams.

This should provide a lot nicer-looking development environment
experience.
2018-05-22 14:29:43 -07:00
Tim Abbott 4c8a69e3fb populate_db: Add an initial stream for the default guest user. 2018-05-22 14:29:43 -07:00
Sampriti Panda 4ce5a85a49 populate_db: Make test_suite data generation more deterministic. 2018-05-22 14:07:55 -07:00
Shubham Padia 4d70d032bc populate_db: Make is_web_public default to False.
Explicit mentions of `is_web_public` as False in the stream dict to be
passed to `bulk_create_streams` have been removed.
2018-05-16 13:40:22 -07:00
Shubham Padia 315047a38b populate_db: Make invite_only default to False.
Explicit mentions of `invite_only` as False in the stream dict to be
passed to `bulk_create_streams` have been removed.
2018-05-16 13:40:10 -07:00
Shubham Padia 7b4f6e0058 tests: Make is_announcement_only true for `announce` stream in populate_db.
Makes announce stream `is_announcement_only` for the dev db for easier
manual testing. The default value for `is_announcement_only` in
`bulk_create_streams` is False.
2018-05-16 13:35:45 -07:00
Aditya Bansal 67bf71472a zilencer: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Aditya Bansal 9629be689b populate_db: Add a web public stream to dev database.
We flip the Stream "Rome" to be a web public stream. Also we add
attribute is_web_public in various stream dicts and in the
bulk_create_streams function of bulk_create.py responsible for
default stream creation in dev environment.
2018-05-02 15:23:33 -07:00
Yashashvi Dave 0d7d94d0db custom fields: Add support for custom URL field type. 2018-04-30 10:53:23 -07:00
Yashashvi Dave 4033f210af custom fields: Add support for custom date field type. 2018-04-30 23:04:25 +05:30
Tim Abbott 8344bd171e populate_db: Properly initialize the email_gateway_bot.
Apparently, this bot account was not properly being tagged as an API
super user in the test database; resulting in incorrect behavior if we
tried to send to a private stream in a test.

(Note that there seems to also be a similar issue in production, that
we don't understand the cause of; that is unrelated).
2018-04-27 13:56:04 -07:00
Umair Khan cf2f6b38dd profile: Add choice field.
Fixes part of #8878
2018-04-26 00:35:51 -07:00
Shubham Dhama 26d2ffa821 populate_db: Add new user of "guest" type.
The purpose of this user is to act as a guest.
(This is a preliminary step in adding the guest type of user
and is a part of #8385.)
2018-04-20 16:20:00 -07:00
Preston Hansen 76d6c71595 tests: Move zerver/fixtures to zerver/tests/fixtures for clarity.
Fixes #9153.
2018-04-19 21:50:17 -07:00
Umair Khan 0420b89468 profile: Add hint attribute to custom profile fields.
This is the model for #8876.
2018-04-19 11:32:35 -07:00
Umair Khan c30a282dd9 profile: Remove integer and float fields. 2018-04-02 09:46:21 -07:00
sinwar 79672fffc6 populate_db: Pre-populate open graph cache to avoid network activity.
This prevents annoying network errors when trying to populate the
database without Internet access.

Cleaned up a bit by tabbott.

Fixes #8289.
2018-03-27 16:42:59 -07:00
Jack Weatherilt 3396cfc2ef refactoring: Replaced occurences of create_stream_if_needed.
Issue #2088 asked for a wrapper to be created for
`create_stream_if_needed` (called `ensure_stream`) for the 25 times that
`create_stream_if_needed` is called and ignores whether the stream was
created. This commit replaces relevant occurences of
`create_stream_if_needed` with `ensure_stream`, including imports.

The changes weren't significant enough to add any tests or do any
additional manual testing.

The refactoring intended to make the API easier to use in most cases.
The majority of uses of `create_stream_if_needed` ignored the second
parameter.

Fixes: #2088.
2018-03-21 16:47:36 -07:00
YJDave 6ac687790c populate_db: Remove the `test_suite` check for custom profile fields.
To ensure that we have some basic data for custom profile settings,
in the `populate_db` data set, remove `options['test_suite']` check
for adding intial custom profile data.
2018-03-21 16:05:31 -07:00
Harshit Bansal a49655e0d4 emoji: Migrate realm emoji to be addressed by `id` rather than `name`.
This commit migrates realm emoji to be addressed by their `id` rather
than their name. This fixes a long standing issue which was causing
an error on uploading an emoji with same name as a deactivated realm
emoji.

Fixes: #6977.
2018-03-20 22:24:44 -07:00
Eeshan Garg af56df7723 webhooks: Enable custom topics and default PM notifications.
This commit adds a generic function called check_send_webhook_message
that does the following:
* If a stream is specified in the webhook URL, it sends a stream
  message, otherwise sends a PM to the owner of the bot.
* In the case of a stream message, if a custom topic is specified
  in the webhook URL, it uses that topic as the subject of the
  stream message.

Also, note that we need not test this anywhere except for the
helloworld webhook. Since helloworld is our default example for
webhooks, it is here to stay and it made sense that tests for a
generic function such as check_send_webhook_message be tested
with an actual generic webhook!

Fixes #8607.
2018-03-16 11:34:20 -07:00
Tim Abbott 56b1239ad5 populate_db: Include some initial data for custom profile fields.
This is currently only for manual testing; creating these would break
the existing custom profile test suite.
2018-03-11 18:08:17 -07:00
Nick Al cab3c8c32c populate_db: Spread message timestamps over a few days.
This makes it easier to work on features that depend on messages
having been sent in the past (E.g. the date parts of recipient bars).

The new feature only works with --threads=1; since with the ~100
default messages that populate_db generates, the multi-threaded
feature shouldn't have significant performance impact (and it would be
tricky to make increasing timestamps work with the multi-threaded
model), it's reasonable to just set the default number of threads to 1
for now and have this timestamp-spreading feature only supported with
--threads=1.

Fixes #8277.
2018-02-24 08:11:27 -08:00
neiljp (Neil Pilgrim) bed56b253e mypy: Add clarifying variable in add_new_user.py. 2018-02-13 11:40:52 -08:00
Vishnu Ks a11b742984 messages: Calculate value of first visible message ID using cron job.
[greg: Fixed buggy time conversion in estimate_recent_messages.]
2018-01-24 17:15:08 -08:00
Greg Price 9a4cd54a25 zilencer: Add a simple management command to create a RemoteZulipServer.
This saves us from having to go into a `manage.py shell` to do this,
and adds a bit more structure like the usage message.
2018-01-08 17:46:46 -08:00
Marco Burstein 6f8792729d manage.py: Move the maximum digits calculation to outside of a loop. 2018-01-02 17:21:28 -05:00
Marco Burstein ff5a73e027 manage.py: Make it easy to create lots of streams with `populate_db`.
Fix #7968.
2018-01-02 16:48:03 -05:00
rht 92d62f62e1 zilencer/management: Use Python 3 syntax for typing. 2017-12-12 17:41:12 -08:00
Greg Price 50e451cd7b populate_db: Create an email-reusing Cordelia in the "lear" realm. 2017-11-28 16:23:10 -08:00
Greg Price a753f49e6a populate_db: Add a "lear" realm for testing cross-realm email sharing.
We don't yet actually allow sharing emails across realms, but we
will soon.  This lets us start testing some pieces of those changes.
2017-11-28 16:23:10 -08:00
rht 138c486548 zilencer: Use python 3 syntax for typing. 2017-11-22 12:29:03 -08:00
Rishi Gupta 42652713c8 onboarding: Refactor away setup_initial_private_stream.
It is a confusing name, now that the stream creation part has been taken
away.
2017-11-21 17:39:50 -08:00
Tim Abbott 708872f361 populate_db: Use call_consume_in_tests when generating test data.
We'd rather this work be just executed immediately, rather than
queued, since queued events can confuse the queue workers if the
database is dropped and recreated repeatedly.
2017-11-21 09:43:54 -08:00
Tim Abbott 64230b3bd5 populate_db: Remove the 'simple' realm.
The 'simple' realm was super broken and confusing for new users.  We
should replace this with having an easy way to make a new realm in
development, done properly.

Fixes #6116.
2017-11-15 22:42:05 -08:00
Tim Abbott 2b43a0302a python: Sort imports in smaller apps. 2017-11-15 15:55:49 -08:00
Umair Khan ad8f5650a3 user-groups: Generate test data of user groups. 2017-11-13 13:08:15 +05:00
rht 548c96ad86 zilencer: Text-wrap long lines exceeding 110. 2017-11-07 17:24:06 -08:00
rht 995b300840 zilencer: Remove unused imports (F401). 2017-11-07 16:37:01 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
rht ec5120e807 refactor: Remove six.moves.zip import. 2017-11-07 10:46:42 -08:00
rht 0e71b243ce zilencer: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 6c995db231 mypy: Use Iterator[Any] as return type of queryset_iterator. 2017-11-04 19:47:45 -07:00
Harshit Bansal 233595b2f7 actions.py: Rename `do_add_reaction()` to `do_add_reaction_legacy()`. 2017-10-31 15:42:35 -07:00
Vishnu Ks 1d94119d31 actions: Call send_initial_pms from process_new_human_user. 2017-10-25 14:14:59 -07:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
rht 2f3ae84e5a py3: Remove all `__future__ import division`. 2017-10-17 23:09:12 -07:00
Tim Abbott 51006f4a43 populate_db: Create bots before sending messages.
Apparently, the order we'd just created in
cbbdd539f4 broke populate_db; this is a
better and more sustainable fix to the overall problem.
2017-10-11 23:27:00 -07:00
Tim Abbott cbbdd539f4 populate_db: Fix creating welcome-bot at the wrong place.
While it might be useful to have created welcome-bot earlier in a
certain sense, it's definitely not a good idea in this populate_db
implementation, because doing so threw off the random initial
assignment of users to streams and thus broke the casper tests.
2017-10-11 22:17:09 -07:00
derAnfaenger 5ddc336844 tests: Add welcome bot as user. 2017-10-11 20:45:42 -07:00
Brock Whittaker 835b6c7059 components: Make checkboxes darker.
This makes the standard checkboxes 7% darker and makes the disabled
ones about 12% darker + 7% darker than they were before, to
increase visibility.

Fixes: #6331.
2017-10-11 13:15:44 -07:00
rht 8d289b8228 zilencer: Remove absolute_import. 2017-09-27 20:20:07 -07:00
rht ed9b3ca440 zilencer: Remove import print_function. 2017-09-27 18:06:47 -07:00
Robert Hönig c77b245944 backend tests: Add 'AARON' as bot owner of OUTGOING_WEBHOOK_BOT. 2017-09-26 16:11:27 -07:00
Rishi Gupta 0727f045f5 zilencer: Add management commands to create new dev users and realms. 2017-09-22 06:35:43 -07:00
Rishi Gupta b32fe9f2a0 features: Update screenshot and add tool to generate.
This adds a new management command that should make generating or
updating screenshots a bit easier in the future.
2017-08-22 11:32:22 -07:00
Tim Abbott d7f8781b06 populate_db: Remove unnecessary print statement. 2017-08-15 12:55:56 -07:00
Tim Abbott 2858c13513 populate_db: Use caching helpers to get users. 2017-08-15 12:55:00 -07:00
Tim Abbott 27b79e7189 populate_db: Fix duplicate test data when run multithreaded.
The previous implementation resulted in each of the 10 threads using
the same random seed and thus same messages.

Fixes #6028.
2017-08-05 11:22:11 -07:00
Darren Fix 69767a43cb Create complex sample messages for dev VM.
Create a generator script to pull lines from a play, enhancing
random lines with emoji, Markdown and other flair.

With numerous contributions from Rein Zustand and Tim Abbott to finish
the project.

Fixes: #1666.
2017-08-01 20:54:39 -07:00
Aditya Bansal 9e372c465c populate_db: Set pointer as message_id of last user sent message. 2017-07-28 11:57:52 -07:00
Harshit Bansal eeb25b6b4b populate_db: Add a test realm emoji while populating databases.
Add `green_tick` realm emoji which can be used while performing tests.
2017-07-27 18:57:06 +00:00
Vishnu Ks 7eaec543f1 zilencer: Use get_user in populate_db.py. 2017-07-18 17:14:05 -07:00
Vishnu Ks df664db7b4 zilencer: Remove unused imports from populate_db.py. 2017-07-18 17:14:05 -07:00
Vishnu Ks 9a44794d14 zilencer: Eliminate get_user_profile_by_email from sync_api_key. 2017-07-18 17:14:05 -07:00
Vishnu Ks b0b809b195 zilencer: Eliminate get_user_profile_by_email from print_initial_password. 2017-07-18 17:14:05 -07:00
Vishnu Ks 754db8ec9f zilencer: Eliminate get_user_profile_by_email from profile_request. 2017-07-18 17:14:05 -07:00