Jun 25, 2024

Using ChatGPT? Why is it a bad idea for your organization

ChatGPT may be great for personal use but this blog explains you why organizations are banning ChatGPT and why you should do it too!

ChatGPT for Enterprises, AI for everyone, ChatGPT for organizations

OpenAI’s ChatGPT, despite its impressive language generation capabilities, has shown several instances of failure that organizations should consider before implementing this technology. This blog post will delve into some of these issues, including hallucinations, data privacy concerns, and specific use cases where ChatGPT has failed.

Understanding ChatGPT

ChatGPT is a language model developed by OpenAI. It’s trained on a vast amount of internet text and can generate human-like text based on the prompts it’s given1. While this can be incredibly useful, it also opens the door to several potential issues.

The Issue of Hallucinations

Hallucinations in ChatGPT refer to instances where the model generates false or absurd responses that aren’t based on its training data. These hallucinations can occur due to various reasons, such as data sparsity, model limitations, or adversarial attacks.

For example, ChatGPT might generate an incorrect date for a historical event or attribute an invention to the wrong person. These inaccuracies can pose challenges to the reliability and security of ChatGPT and its applications.

Data Privacy Concerns

Data privacy is another significant concern with ChatGPT. The model is trained on vast amounts of data, and users have no way of knowing which of their data it contains. This lack of transparency can lead to potential privacy violations.

Moreover, any information entered into ChatGPT may become part of its training dataset. This means that sensitive, proprietary, or confidential information used in prompts may be incorporated into responses for users outside the enterprise.

Use Cases of ChatGPT Failures

Despite its impressive capabilities, ChatGPT has shown several instances of failure. Here are some notable examples:

  1. Mathematical Errors: In a study conducted by researchers at Stanford and UC Berkeley, GPT-4’s accuracy in identifying prime numbers dropped from 97.6% in March 2023 to a mere 2.4% in June 20235.
  2. Code Generation: Both GPT-4 and GPT-3.5 showed more formatting mistakes in code generation in June than in March.
  3. Sensitive Questions: GPT-4 was less willing to answer sensitive questions in June than in March.
  4. Jailbreaking Attacks: While GPT-4’s update was more robust to jailbreaking attacks than that of GPT-3.5, the risk still exists. Jailbreaking is a form of manipulation in which a prompt is crafted to conceal a malicious question and surpass protection boundaries.

Companies Not Using ChatGPT

Several companies have chosen not to use ChatGPT due to various concerns. Here are some examples:

  1. Samsung: In May 2023, Samsung prohibited the use of ChatGPT and other generative AI tools.
  2. Commonwealth Bank of Australia: In June 2023, the Commonwealth Bank of Australia restricted the use of ChatGPT
  3. JPMorgan Chase & Co.: JPMorgan Chase & Co. has also banned the use of ChatGPT.

These companies have cited various reasons for their decisions, including concerns about data privacy, the potential for the technology to generate incorrect or misleading information, and the risk of sensitive company information being unintentionally shared with other users65.

Decision pointsOpen-Source LLMClose-Source LLM
AccessibilityThe code behind the LLM is freely available for anyone to inspect, modify, and use. This fosters collaboration and innovation.The underlying code is proprietary and not accessible to the public. Users rely on the terms and conditions set by the developer.
CustomizationLLMs can be customized and adapted for specific tasks or applications. Developers can fine-tune the models and experiment with new techniques.Customization options are typically limited. Users might have some options to adjust parameters, but are restricted to the functionalities provided by the developer.
Community & DevelopmentBenefit from a thriving community of developers and researchers who contribute to improvements, bug fixes, and feature enhancements.Development is controlled by the owning company, with limited external contributions.
SupportSupport may come from the community, but users may need to rely on in-house expertise for troubleshooting and maintenance.Typically comes with dedicated support from the developer, offering professional assistance and guidance.
CostGenerally free to use, with minimal costs for running the model on your own infrastructure, & may require investment in technical expertise for customization and maintenance.May involve licensing fees, pay-per-use models or require cloud-based access with associated costs.
Transparency & BiasGreater transparency as the training data and methods are open to scrutiny, potentially reducing bias.Limited transparency makes it harder to identify and address potential biases within the model.
IPCode and potentially training data are publicly accessible, can be used as a foundation for building new models.Code and training data are considered trade secrets, no external contributions
SecurityTraining data might be accessible, raising privacy concerns if it contains sensitive information & Security relies on the communityThe codebase is not publicly accessible, control over the training data and stricter privacy measures & Security depends on the vendor's commitment
ScalabilityUsers might need to invest in their own infrastructure to train and run very large models & require leveraging community experts resourcesCompanies often have access to significant resources for training and scaling their models and can be offered as cloud-based services
Deployment & Integration ComplexityOffers greater flexibility for customization and integration into specific workflows but often requires more technical knowledgeTypically designed for ease of deployment and integration with minimal technical setup. Customization options might be limited to functionalities offered by the vendor.
10 ponits you need to evaluate for your Enterprise Usecases


While ChatGPT offers many benefits, organizations need to be aware of the potential issues surrounding hallucinations, data privacy, and specific use cases where the model has failed. By understanding these concerns, organizations can make informed decisions about whether or not to implement this technology.

Remember, while AI can be a powerful tool, it’s essential to use it responsibly and ethically. As with any technology, the key is to understand its limitations and use it to benefit your organization while minimizing potential risks.

At Fluid AI, we stand at the forefront of this AI revolution, helping organizations kickstart their AI journey giving utmost importance to security. If you’re seeking a solution for your organization, look no further. We’re committed to making your organization future-ready, just like we’ve done for many others.

Take the first step towards this exciting journey by booking a free demo call with us today. Let’s explore the possibilities together and unlock the full potential of AI for your organization. Remember, the future belongs to those who prepare for it today.

Didn't find specific use-case you're looking for?

Talk to our Gen AI Expert !

Book your free 1-1 strategic call

- Outline your AI strategic roadmap and identify high-impact use cases.
- Craft an optimal data architecture, tailor models, & bring your most ambitious AI projects to life.
- Scope with simple internal pilot journey instantly in just 1-day.
- Easily Scale-to-Production, & achieve seamless integration with your existing financial systems.
- Holistic end-to-end support, insights & performance evaluation for successful journey.