Commit Graph

225 Commits

Author SHA1 Message Date
Umair Khan c30a282dd9 profile: Remove integer and float fields. 2018-04-02 09:46:21 -07:00
sinwar 79672fffc6 populate_db: Pre-populate open graph cache to avoid network activity.
This prevents annoying network errors when trying to populate the
database without Internet access.

Cleaned up a bit by tabbott.

Fixes #8289.
2018-03-27 16:42:59 -07:00
Jack Weatherilt 3396cfc2ef refactoring: Replaced occurences of create_stream_if_needed.
Issue #2088 asked for a wrapper to be created for
`create_stream_if_needed` (called `ensure_stream`) for the 25 times that
`create_stream_if_needed` is called and ignores whether the stream was
created. This commit replaces relevant occurences of
`create_stream_if_needed` with `ensure_stream`, including imports.

The changes weren't significant enough to add any tests or do any
additional manual testing.

The refactoring intended to make the API easier to use in most cases.
The majority of uses of `create_stream_if_needed` ignored the second
parameter.

Fixes: #2088.
2018-03-21 16:47:36 -07:00
YJDave 6ac687790c populate_db: Remove the `test_suite` check for custom profile fields.
To ensure that we have some basic data for custom profile settings,
in the `populate_db` data set, remove `options['test_suite']` check
for adding intial custom profile data.
2018-03-21 16:05:31 -07:00
Harshit Bansal a49655e0d4 emoji: Migrate realm emoji to be addressed by `id` rather than `name`.
This commit migrates realm emoji to be addressed by their `id` rather
than their name. This fixes a long standing issue which was causing
an error on uploading an emoji with same name as a deactivated realm
emoji.

Fixes: #6977.
2018-03-20 22:24:44 -07:00
Eeshan Garg af56df7723 webhooks: Enable custom topics and default PM notifications.
This commit adds a generic function called check_send_webhook_message
that does the following:
* If a stream is specified in the webhook URL, it sends a stream
  message, otherwise sends a PM to the owner of the bot.
* In the case of a stream message, if a custom topic is specified
  in the webhook URL, it uses that topic as the subject of the
  stream message.

Also, note that we need not test this anywhere except for the
helloworld webhook. Since helloworld is our default example for
webhooks, it is here to stay and it made sense that tests for a
generic function such as check_send_webhook_message be tested
with an actual generic webhook!

Fixes #8607.
2018-03-16 11:34:20 -07:00
Tim Abbott 56b1239ad5 populate_db: Include some initial data for custom profile fields.
This is currently only for manual testing; creating these would break
the existing custom profile test suite.
2018-03-11 18:08:17 -07:00
Nick Al cab3c8c32c populate_db: Spread message timestamps over a few days.
This makes it easier to work on features that depend on messages
having been sent in the past (E.g. the date parts of recipient bars).

The new feature only works with --threads=1; since with the ~100
default messages that populate_db generates, the multi-threaded
feature shouldn't have significant performance impact (and it would be
tricky to make increasing timestamps work with the multi-threaded
model), it's reasonable to just set the default number of threads to 1
for now and have this timestamp-spreading feature only supported with
--threads=1.

Fixes #8277.
2018-02-24 08:11:27 -08:00
neiljp (Neil Pilgrim) bed56b253e mypy: Add clarifying variable in add_new_user.py. 2018-02-13 11:40:52 -08:00
Vishnu Ks a11b742984 messages: Calculate value of first visible message ID using cron job.
[greg: Fixed buggy time conversion in estimate_recent_messages.]
2018-01-24 17:15:08 -08:00
Greg Price 9a4cd54a25 zilencer: Add a simple management command to create a RemoteZulipServer.
This saves us from having to go into a `manage.py shell` to do this,
and adds a bit more structure like the usage message.
2018-01-08 17:46:46 -08:00
Marco Burstein 6f8792729d manage.py: Move the maximum digits calculation to outside of a loop. 2018-01-02 17:21:28 -05:00
Marco Burstein ff5a73e027 manage.py: Make it easy to create lots of streams with `populate_db`.
Fix #7968.
2018-01-02 16:48:03 -05:00
rht 92d62f62e1 zilencer/management: Use Python 3 syntax for typing. 2017-12-12 17:41:12 -08:00
Greg Price 50e451cd7b populate_db: Create an email-reusing Cordelia in the "lear" realm. 2017-11-28 16:23:10 -08:00
Greg Price a753f49e6a populate_db: Add a "lear" realm for testing cross-realm email sharing.
We don't yet actually allow sharing emails across realms, but we
will soon.  This lets us start testing some pieces of those changes.
2017-11-28 16:23:10 -08:00
rht 138c486548 zilencer: Use python 3 syntax for typing. 2017-11-22 12:29:03 -08:00
Rishi Gupta 42652713c8 onboarding: Refactor away setup_initial_private_stream.
It is a confusing name, now that the stream creation part has been taken
away.
2017-11-21 17:39:50 -08:00
Tim Abbott 708872f361 populate_db: Use call_consume_in_tests when generating test data.
We'd rather this work be just executed immediately, rather than
queued, since queued events can confuse the queue workers if the
database is dropped and recreated repeatedly.
2017-11-21 09:43:54 -08:00
Tim Abbott 64230b3bd5 populate_db: Remove the 'simple' realm.
The 'simple' realm was super broken and confusing for new users.  We
should replace this with having an easy way to make a new realm in
development, done properly.

Fixes #6116.
2017-11-15 22:42:05 -08:00
Tim Abbott 2b43a0302a python: Sort imports in smaller apps. 2017-11-15 15:55:49 -08:00
Umair Khan ad8f5650a3 user-groups: Generate test data of user groups. 2017-11-13 13:08:15 +05:00
rht 548c96ad86 zilencer: Text-wrap long lines exceeding 110. 2017-11-07 17:24:06 -08:00
rht 995b300840 zilencer: Remove unused imports (F401). 2017-11-07 16:37:01 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
rht ec5120e807 refactor: Remove six.moves.zip import. 2017-11-07 10:46:42 -08:00
rht 0e71b243ce zilencer: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
neiljp (Neil Pilgrim) 6c995db231 mypy: Use Iterator[Any] as return type of queryset_iterator. 2017-11-04 19:47:45 -07:00
Harshit Bansal 233595b2f7 actions.py: Rename `do_add_reaction()` to `do_add_reaction_legacy()`. 2017-10-31 15:42:35 -07:00
Vishnu Ks 1d94119d31 actions: Call send_initial_pms from process_new_human_user. 2017-10-25 14:14:59 -07:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
rht 2f3ae84e5a py3: Remove all `__future__ import division`. 2017-10-17 23:09:12 -07:00
Tim Abbott 51006f4a43 populate_db: Create bots before sending messages.
Apparently, the order we'd just created in
cbbdd539f4 broke populate_db; this is a
better and more sustainable fix to the overall problem.
2017-10-11 23:27:00 -07:00
Tim Abbott cbbdd539f4 populate_db: Fix creating welcome-bot at the wrong place.
While it might be useful to have created welcome-bot earlier in a
certain sense, it's definitely not a good idea in this populate_db
implementation, because doing so threw off the random initial
assignment of users to streams and thus broke the casper tests.
2017-10-11 22:17:09 -07:00
derAnfaenger 5ddc336844 tests: Add welcome bot as user. 2017-10-11 20:45:42 -07:00
Brock Whittaker 835b6c7059 components: Make checkboxes darker.
This makes the standard checkboxes 7% darker and makes the disabled
ones about 12% darker + 7% darker than they were before, to
increase visibility.

Fixes: #6331.
2017-10-11 13:15:44 -07:00
rht 8d289b8228 zilencer: Remove absolute_import. 2017-09-27 20:20:07 -07:00
rht ed9b3ca440 zilencer: Remove import print_function. 2017-09-27 18:06:47 -07:00
Robert Hönig c77b245944 backend tests: Add 'AARON' as bot owner of OUTGOING_WEBHOOK_BOT. 2017-09-26 16:11:27 -07:00
Rishi Gupta 0727f045f5 zilencer: Add management commands to create new dev users and realms. 2017-09-22 06:35:43 -07:00
Rishi Gupta b32fe9f2a0 features: Update screenshot and add tool to generate.
This adds a new management command that should make generating or
updating screenshots a bit easier in the future.
2017-08-22 11:32:22 -07:00
Tim Abbott d7f8781b06 populate_db: Remove unnecessary print statement. 2017-08-15 12:55:56 -07:00
Tim Abbott 2858c13513 populate_db: Use caching helpers to get users. 2017-08-15 12:55:00 -07:00
Tim Abbott 27b79e7189 populate_db: Fix duplicate test data when run multithreaded.
The previous implementation resulted in each of the 10 threads using
the same random seed and thus same messages.

Fixes #6028.
2017-08-05 11:22:11 -07:00
Darren Fix 69767a43cb Create complex sample messages for dev VM.
Create a generator script to pull lines from a play, enhancing
random lines with emoji, Markdown and other flair.

With numerous contributions from Rein Zustand and Tim Abbott to finish
the project.

Fixes: #1666.
2017-08-01 20:54:39 -07:00
Aditya Bansal 9e372c465c populate_db: Set pointer as message_id of last user sent message. 2017-07-28 11:57:52 -07:00
Harshit Bansal eeb25b6b4b populate_db: Add a test realm emoji while populating databases.
Add `green_tick` realm emoji which can be used while performing tests.
2017-07-27 18:57:06 +00:00
Vishnu Ks 7eaec543f1 zilencer: Use get_user in populate_db.py. 2017-07-18 17:14:05 -07:00
Vishnu Ks df664db7b4 zilencer: Remove unused imports from populate_db.py. 2017-07-18 17:14:05 -07:00
Vishnu Ks 9a44794d14 zilencer: Eliminate get_user_profile_by_email from sync_api_key. 2017-07-18 17:14:05 -07:00
Vishnu Ks b0b809b195 zilencer: Eliminate get_user_profile_by_email from print_initial_password. 2017-07-18 17:14:05 -07:00
Vishnu Ks 754db8ec9f zilencer: Eliminate get_user_profile_by_email from profile_request. 2017-07-18 17:14:05 -07:00
Aditya Bansal f2d6194ae1 actions: Start logging subscription activities in RealmAuditLog. 2017-07-17 17:23:41 -07:00
Aditya Bansal 0c4165a5bd pep8: Add compliance with rule E261 to populate_db.py. 2017-07-11 11:56:50 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
vaibhav 9cf9837f12 webhooks: Add outgoing webhook bot user to development database. 2017-05-29 16:01:23 -07:00
Aditya Bansal b07273c2d0 pep8: Add compliance with rule E261 to commands/profile_request.py. 2017-05-07 23:21:50 -07:00
Tim Abbott 60225012e6 unread: Add tool for marking all messages as unread for testing.
This tool can save a lot of manual work in testing our unread counts
logic.
2017-04-25 15:40:12 -07:00
Tim Abbott 8ae052a9d8 populate_db: Set a description for the default realm. 2017-04-17 21:59:21 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Rishi Gupta 8617c2d594 populate_db: Make Iago staff.
Will make it easier to test the /activity page.
2017-03-27 16:43:35 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Tim Abbott 3617ebfd41 api: Rename get_old_messages to get_messages in the backend.
Fixes #1315.
2017-03-23 23:52:44 -07:00
Rishi Gupta 871c754369 bulk_create: Remove unused function bulk_create_realms. 2017-03-13 14:42:55 -07:00
Tim Abbott 950e06cf32 populate_db: Remove unused Deployment import. 2017-03-06 00:11:53 -08:00
Tim Abbott 10a52147fc zilencer: Remove obsolete create_deployment command. 2017-03-06 00:10:03 -08:00
Rishi Gupta 1453a5bfda Change string_id of test zephyr realm from mit to zephyr.
Also changes Realm.is_zephyr_mirror_realm to use string_id=zephyr instead of
domain=mit.edu.
Part of a larger migration away from Realm.domain.
2017-03-04 12:18:01 -08:00
Rishi Gupta 15d60fa7ed Change now() to timezone.now() throughout codebase.
Change `from django.utils.timezone import now` to
`from django.utils import timezone`.

This is both because now() is ambiguous (could be datetime.datetime.now),
and more importantly to make it easier to write a lint rule against
datetime.datetime.now().
2017-02-28 16:03:37 -08:00
Tim Abbott 8691f1466a populate_db: Disable use of memcached when populating database.
This fixes an issue where one would get errors of the form:

`ValueError: unsupported pickle protocol: 3`

in a `run-dev.py` server run against Python 2 if you ran `provision`.
Provision currently runs `populate_db` with Python 3, storing Python 3
based data in memcached, which then can't be read by Python 2.
2017-02-26 21:08:47 -08:00
Tim Abbott 08a7e0db44 render_messages: Fix buggy handling of edit history. 2017-02-18 22:33:26 -08:00
Tim Abbott 080182ad17 populate_db: Fix broken initial presence data.
Apparently, the initial data just set everyone to "API" in a broken way.
2017-02-10 23:52:32 -08:00
Tim Abbott 7e8d3d9a46 populate_db: Add --extra-bots option to create extra bots. 2017-02-10 17:40:45 -08:00
Steve Howell 4be2d6577d Add a new community realm to our test databases.
The realm with string_id of "simple" just has three users
named alice, bob, and cindy for now.  It is useful for testing
scenarios where realms don't have special zulip.com exception
handling.
2017-01-26 19:25:34 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tim Abbott d6e38e2a5c lint: Clean up E123 PEP-8 rule. 2017-01-23 21:34:26 -08:00
Tim Abbott c0c9dfb66d populate_db: Fix crash by removing do_send_message.
This old helper has for years been used only by populate_db, and got
buggy (as of a recent refactoring).  So we just call do_send_messages
directly instead.

Fixes the provisioning error we currently get in Travis CI.
2017-01-21 22:07:36 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rafid Aslam 50afe5a6c2 sync_api_key: Fix `ConfigParser` deprecation warnings.
Fix `ConfigParser` deprecation warnings in
`zilencer/management/commands/sync_api_key.py`.

Fixes #2727.
2016-12-30 11:52:22 -08:00
Rafid Aslam aece2c7c45 populate_db: Fix naive datetime warnings
Fix naive datetime warnings in
`zilencer/management/commands/populate_db.py`.

Fixes #2729.
2016-12-20 17:48:22 -08:00
Tim Abbott d3b1f6e273 populate_db: Add notifications stream for Zulip realm. 2016-12-15 22:43:08 -08:00
Rishi Gupta 6b39f3222c Remove Realm.domain from populate_db.py.
Fixes #2286.
2016-12-15 13:22:17 -08:00
Rishi Gupta fc188de8e0 bulk_create: Refactor bulk_create_users to take Realm instead of domains.
Previously bulk_create_users would figure out a user's realm from their
email domain. Now require that a realm be passed explicitly.
2016-12-15 13:22:17 -08:00
Rishi Gupta 56c8b58cbc populate_db: Remove create_streams.
Is extraneous indirection.
2016-12-15 13:22:17 -08:00
Rishi Gupta ee9b59321c populate_db: Refactor create_streams.
Remove unused field from stream_dict.
2016-12-15 13:22:17 -08:00
Rishi Gupta fbf48f7ec6 bulk_create: Refactor bulk_create_streams to not take domain dictionary.
First step in cleaning up populate_db.create_streams and
bulk_create.bulk_create_streams. Part of a series of commits to remove
Realm.domain from populate_db.
2016-12-15 13:22:17 -08:00
reyha 82e32ad255 Access realm by `string_id` in management commands.
`Realm.string_id` replaces 'Realm.domain'
in the management commands.

Fixes #2325.
2016-12-14 10:38:03 -08:00
Juan Verhook 9869fb8d87 Add UserPresence rows to test databases.
Fixes: #2426

mypy: Added annotations to populate_db.py
2016-12-09 15:41:21 -08:00
Joy Chen dd95a5e03f streams: Add default stream description population data. 2016-12-07 21:02:17 -08:00
anirudhjain75 beaa62cafa mypy: Convert several directories to use typing.Text.
Specifically, these directories are converted: [analytics/, scripts/,
tools/, zerver/management/, zilencer/, zproject/]
2016-12-07 20:51:05 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Steve Howell 78474c8bee Add test users with non-lowercase emails.
We are prone to case-sensitivity bugs, so I added AARON and ZOE.
Also, for good measure, I insert them in non-alphabetical order
to try to drive out bugs from non-consistent sorting of user ids.
2016-11-26 11:48:52 -08:00
Tim Abbott 61816f6151 populate_db: Remove replay_old_messages code.
Now that we have the data import/export tool, that's a better
mechanism for importing data than this old and likely buggy code was.
2016-11-15 22:34:44 -08:00
paxapy b90057d557 render_messages.py: render first version of edited messages.
This decreases the number of spurious differences in rendered content
significantly.
2016-11-15 13:39:11 -08:00
Rishi Gupta 5681ee90fa models.py: Rename split_email_to_domain to email_to_domain.
In order to better match models.email_to_username.
2016-11-11 15:26:51 -08:00
paxapy 93965a8e89 Add tools for dumping and comparing markdown renderings.
This adds a couple new tools that can be used to determine whether a
particular change in Zulip's backend markdown processor would impact
the rendering of historical messages, without a human actually looking
at the message content.  This is a useful way to verify whether a
change to our markdown syntax is likely to create problems.

[commit message and code tweaked by tabbott]
2016-11-07 07:49:56 -08:00